MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

Overview

MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

PyPI version test Docker Cloud Build Status Documentation Status Maintainability codecov MIT

MINERVA is an out-of-the-box GUI tool for offline deep reinforcement learning, designed for everyone including non-programmers to do reinforcement learning as a tool.

Documentation: https://minerva-ui.readthedocs.io

Chat: Gitter

key features

⚡ All You Need Is Dataset

MINERVA only requires datasets to start offline deep reinforcement learning. Any combinations of vector observations and image observations with discrete actions and continuous actions are supported.

đź”° Stunning GUI

MINERVA provides designed with intuitive GUI to let everyone lerverage extremely powerful algorithms without barriers. The GUI is developed as a Single Page Application (SPA) to make it work in the eye-opening speed.

🚀 Powerful Algorithm

MINERVA is powered by d3rlpy, a powerful offline deep reinforcement learning library for Python, to provide extremely powerful algorithms in an out-of-the-box way. The trained policy can be exported as TorchScript and ONNX.

installation

PyPI

$ pip install minerva-ui

Docker

$ docker run -d --gpus all -p 9000:9000 --name minerva takuseno/minerva:latest

update guide

If you update MINERVA, the database schema should be also updated as follows:

$ pip install -U minerva-ui
$ minerva upgrade-db

usage

run server

At the first time, ~/.minerva will be automatically created to store database, uploaded datasets and training metrics.

$ minerva run

By default, you can access to MINERVA interface at http://localhost:9000 . You can change the host and port with --host and --port arguments respectively.

delete data

You can delete entire data (~/.minerva) as follows:

$ minerva clean

contributions

build

$ npm install
$ npm run build

coding style

This repository is fully formatted with yapf and standard. You can format the entire scripts as follows:

$ ./scripts/format

lint

This repository is fully analyzed with Pylint, ESLint and sass-lint. You can run analysis as follows:

$ ./scripts/lint

test

The unit tests are provided as much as possible. This repository is using pytest-cov instead of pytest. You can run the entire tests as follows:

$ ./scripts/test

acknowledgement

This work is supported by Information-technology Promotion Agency, Japan (IPA), Exploratory IT Human Resources Project (MITOU Program) in the fiscal year 2020.

The concept of the GUI software for deep reinforcement learning is inspired by DeepAnalyzer from Ghelia inc. I'm showing the great respect to the team here.

Comments
  • Bump y18n from 4.0.0 to 4.0.3

    Bump y18n from 4.0.0 to 4.0.3

    Bumps y18n from 4.0.0 to 4.0.3.

    Changelog

    Sourced from y18n's changelog.

    4.0.3 (2021-04-07)

    Bug Fixes

    • release: 4.x.x should not enforce Node 10 (#126) (1e21a53)

    4.0.1 (2020-11-30)

    Bug Fixes

    Commits
    Maintainer changes

    This version was pushed to npm by oss-bot, a new releaser for y18n since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump pillow from 7.1.2 to 8.1.1

    Bump pillow from 7.1.2 to 8.1.1

    Bumps pillow from 7.1.2 to 8.1.1.

    Release notes

    Sourced from pillow's releases.

    8.1.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.1.html

    8.1.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.0.html

    Changes

    Dependencies

    Deprecations

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.1.1 (2021-03-01)

    • Use more specific regex chars to prevent ReDoS. CVE-2021-25292 [hugovk]

    • Fix OOB Read in TiffDecode.c, and check the tile validity before reading. CVE-2021-25291 [wiredfool]

    • Fix negative size read in TiffDecode.c. CVE-2021-25290 [wiredfool]

    • Fix OOB read in SgiRleDecode.c. CVE-2021-25293 [wiredfool]

    • Incorrect error code checking in TiffDecode.c. CVE-2021-25289 [wiredfool]

    • PyModule_AddObject fix for Python 3.10 #5194 [radarhere]

    8.1.0 (2021-01-02)

    • Fix TIFF OOB Write error. CVE-2020-35654 #5175 [wiredfool]

    • Fix for Read Overflow in PCX Decoding. CVE-2020-35653 #5174 [wiredfool, radarhere]

    • Fix for SGI Decode buffer overrun. CVE-2020-35655 #5173 [wiredfool, radarhere]

    • Fix OOB Read when saving GIF of xsize=1 #5149 [wiredfool]

    • Makefile updates #5159 [wiredfool, radarhere]

    • Add support for PySide6 #5161 [hugovk]

    • Use disposal settings from previous frame in APNG #5126 [radarhere]

    • Added exception explaining that repr_png saves to PNG #5139 [radarhere]

    • Use previous disposal method in GIF load_end #5125 [radarhere]

    ... (truncated)

    Commits
    • 741d874 8.1.1 version bump
    • 179cd1c Added 8.1.1 release notes to index
    • 7d29665 Update CHANGES.rst [ci skip]
    • d25036f Credits
    • 973a4c3 Release notes for 8.1.1
    • 521dab9 Use more specific regex chars to prevent ReDoS
    • 8b8076b Fix for CVE-2021-25291
    • e25be1e Fix negative size read in TiffDecode.c
    • f891baa Fix OOB read in SgiRleDecode.c
    • cbfdde7 Incorrect error code checking in TiffDecode.c
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 1
  • Bump ini from 1.3.5 to 1.3.8

    Bump ini from 1.3.5 to 1.3.8

    Bumps ini from 1.3.5 to 1.3.8.

    Commits
    • a2c5da8 1.3.8
    • af5c6bb Do not use Object.create(null)
    • 8b648a1 don't test where our devdeps don't even work
    • c74c8af 1.3.7
    • 024b8b5 update deps, add linting
    • 032fbaf Use Object.create(null) to avoid default object property hazards
    • 2da9039 1.3.6
    • cfea636 better git push script, before publish instead of after
    • 56d2805 do not allow invalid hazardous string as section name
    • See full diff in compare view
    Maintainer changes

    This version was pushed to npm by isaacs, a new releaser for ini since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump axios from 0.19.2 to 0.21.1

    Bump axios from 0.19.2 to 0.21.1

    Bumps axios from 0.19.2 to 0.21.1.

    Release notes

    Sourced from axios's releases.

    v0.21.1

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    v0.21.0

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Changelog

    Sourced from axios's changelog.

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • What does Eposide id correspond to in a recommendation dataset

    What does Eposide id correspond to in a recommendation dataset

    I have a recommender dataset where I have userID, user_feature1,user_featureX...., ItemID,Item_feature1...Item_featureY.., reward(sales amount),datetime.. how should I restructure this data to be used/tested in Minerva? In the tutorial, https://minerva-ui.readthedocs.io/en/v0.40/getting_started.html, I am unable to correlate what will be the eposide ID in my dataset.

    opened by ayush488 0
  • ERROR: (sqlite3.OperationalError) unable to open database file.

    ERROR: (sqlite3.OperationalError) unable to open database file.

    Hi,

    I just installed the Minerva. However when I execute minerva run command I receive following error:

    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file (Background on this error at: http://sqlalche.me/e/14/e3q8)

    Can you please tell me what is causing this error and how can I fix it? Thanks in advance.

    bug 
    opened by Zaharah 3
Releases(v0.40)
  • v0.40(May 29, 2021)

    Support d3rlpy v0.90!

    MINERVA now supports d3rlpy v0.90 and there are the highlights for MINERVA.

    • more precise CQL and BEAR implementations
    • predict_value is fixed for unscaled actions
    • CRR algorithm is available
    Source code(tar.gz)
    Source code(zip)
  • v0.32(Mar 6, 2021)

    a way to upload images is changed

    Previously, the image files are uploaded via selecting the directory that contains all the files. However this is an easy way, the browser limits the number of files due to the efficiency issue. So, from this version, you can upload zipped file that contains the image files. This is the more efficient and fast way. image

    Source code(tar.gz)
    Source code(zip)
  • v0.31(Mar 6, 2021)

  • v0.30(Mar 5, 2021)

    Support d3rlpy v0.70!

    MINERVA now supports d3rlpy v0.70, which has a lot of progress from the v0.41 that is the previous dependency.

    These are some highlights related to MINERVA.

    • the continuous action is automatically normalized based on the dataset statistics
    • discrete action option has been removed since the action-space is automatically detected
    • many other internal enhancements
    Source code(tar.gz)
    Source code(zip)
  • v0.20(Dec 21, 2020)

    Support d3rlpy v0.41!

    MINERVA now supports d3rlpy v0.41, which has a lot of progress from the v0.23 that is the previous dependency.

    These are the some highlights related to MINERVA.

    • extremely fast mini-batch creation
    • extremely fast frame stacking for image observation
    • extremely fast N-step TD calculation
    • new metrics
    • etc

    Algorithm Selection

    MINERVA now provides many many algorithms for both discrete and continuous control datasets. You can choose an algorithm at the project creation dialog.

    discrete algorithms

    • DQN
    • Double DQN
    • AWR
    • CQL
    • BCQ
    • SAC

    continuous algorithms

    • DDPG
    • TD3
    • SAC
    • BCQ
    • BEAR
    • CQL
    • AWR
    • AWAC
    • PLAS

    Of course, there is the Q function option to incorporate arbitrary algorithms with the powerful distributional Q functions.

    Q functions

    • mean
    • Quantile Regression
    • Implicit Quantile Network
    • Fully parameterized Quantile Function
    Source code(tar.gz)
    Source code(zip)
  • v0.12(Sep 9, 2020)

  • v0.11(Sep 9, 2020)

    Dataset Dashboard

    • upload datasets
      • discrete/continuous action-space
      • vector/image observations

    Project Dashboard

    • train with CQL with detailed configurations.

    Export Policy

    • Support TorchScript and ONNX.
    Source code(tar.gz)
    Source code(zip)
Owner
Takuma Seno
Machine learning engineer at Sony AI / Ph.D CS student at Keio University.
Takuma Seno
Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection

Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection (NimPme) The official implementation of Novel Instances Mining with

12 Sep 08, 2022
Explainable Zero-Shot Topic Extraction

Zero-Shot Topic Extraction with Common-Sense Knowledge Graph This repository contains the code for reproducing the results reported in the paper "Expl

D2K Lab 56 Dec 14, 2022
Spectral Temporal Graph Neural Network (StemGNN in short) for Multivariate Time-series Forecasting

Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting This repository is the official implementation of Spectral Temporal Gr

Microsoft 306 Dec 29, 2022
DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Differentiable Model Compression via Pseudo Quantization Noise DiffQ performs differentiable quantization using pseudo quantization noise. It can auto

Facebook Research 145 Dec 30, 2022
Safe Bayesian Optimization

SafeOpt - Safe Bayesian Optimization This code implements an adapted version of the safe, Bayesian optimization algorithm, SafeOpt [1], [2]. It also p

Felix Berkenkamp 111 Dec 11, 2022
Building blocks for uncertainty-aware cycle consistency presented at NeurIPS'21.

UncertaintyAwareCycleConsistency This repository provides the building blocks and the API for the work presented in the NeurIPS'21 paper Robustness vi

EML TĂĽbingen 19 Dec 12, 2022
Pytorch implementation of NeurIPS 2021 paper: Geometry Processing with Neural Fields.

Geometry Processing with Neural Fields Pytorch implementation for the NeurIPS 2021 paper: Geometry Processing with Neural Fields Guandao Yang, Serge B

Guandao Yang 162 Dec 16, 2022
code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren*, Raymond A. Yeh*, Alexander G. Schwing.

Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning Overview This code is for paper: Not All Unlabeled Data are Equa

Jason Ren 22 Nov 23, 2022
DockStream: A Docking Wrapper to Enhance De Novo Molecular Design

DockStream Description DockStream is a docking wrapper providing access to a collection of ligand embedders and docking backends. Docking execution an

AstraZeneca - Molecular AI 72 Jan 02, 2023
Jarvis Project is a basic virtual assistant that uses TensorFlow for learning.

Jarvis_proyect Jarvis Project is a basic virtual assistant that uses TensorFlow for learning. Latest version 0.1 Features: Good morning protocol Tell

Anze Kovac 3 Aug 31, 2022
Official implementation of the paper "AAVAE: Augmentation-AugmentedVariational Autoencoders"

AAVAE Official implementation of the paper "AAVAE: Augmentation-AugmentedVariational Autoencoders" Abstract Recent methods for self-supervised learnin

Grid AI Labs 48 Dec 12, 2022
Tensorflow AffordanceNet and AffContext implementations

AffordanceNet and AffContext This is tensorflow AffordanceNet and AffContext implementations. Both are implemented and tested with tensorflow 2.3. The

Beatriz Pérez 6 Dec 01, 2022
Boosted CVaR Classification (NeurIPS 2021)

Boosted CVaR Classification Runtian Zhai, Chen Dan, Arun Sai Suggala, Zico Kolter, Pradeep Ravikumar NeurIPS 2021 Table of Contents Quick Start Train

Runtian Zhai 4 Feb 15, 2022
Only valid pull requests will be allowed. Use python only and readme changes will not be accepted.

❌ This repo is excluded from hacktoberfest This repo is for python beginners and contains lot of beginner python projects for practice. You can also s

Prajjwal Pathak 50 Dec 28, 2022
DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection

DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection Code for our Paper DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Obje

Steven Lang 58 Dec 19, 2022
Semiconductor Machine learning project

Wafer Fault Detection Problem Statement: Wafer (In electronics), also called a slice or substrate, is a thin slice of semiconductor, such as a crystal

kunal suryawanshi 1 Jan 15, 2022
A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.

python_graphs This package is for computing graph representations of Python programs for machine learning applications. It includes the following modu

Google Research 258 Dec 29, 2022
You Only Look One-level Feature (YOLOF), CVPR2021, Detectron2

You Only Look One-level Feature (YOLOF), CVPR2021 A simple, fast, and efficient object detector without FPN. This repo provides a neat implementation

qiang chen 273 Jan 03, 2023
Doge-Prediction - Coding Club prediction ig

Doge-Prediction Coding Club prediction ig Basically: Create an application that

1 Jan 10, 2022
Official implementation for the paper "Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection"

Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection PyTorch code release of the paper "Attentive Prototypes for Sour

Deepti Hegde 23 Oct 17, 2022