The CARL Benchmark Library
CARL (context adaptive RL) provides highly configurable contextual extensions to several well-known RL environments. It's designed to test your agent's generalization capabilities in all scenarios where intra-task generalization is important.
Benchmarks include:
-
OpenAI gym classic control suite extended with several physics context features like gravity or friction
-
OpenAI gym Box2D BipedalWalker, LunarLander and CarRacing, each with their own modification possibilities like new vehicles to race
-
All Brax locomotion environments with exposed internal features like joint strength or torso mass
-
Super Mario (TOAD-GAN), a procedurally generated jump'n'run game with control over level similarity
-
RNADesign, an environment for RNA design given structure constraints with structures from different datasets to choose from
Installation
We recommend you use a virtual environment (e.g. Anaconda) to install CARL and its dependencies. We recommend and test with python 3.9 under Linux.
First, clone our repository and install the basic requirements:
git clone https://github.com/automl/CARL.git --recursive
cd CARL
pip install .
This will only install the basic classic control environments, which should run on most operating systems. For the full set of environments, use the install options:
pip install -e .[box2d, brax, rna, mario]
These may not be compatible with Windows systems. Box2D environment may need to be installed via conda on MacOS systems:
conda install -c conda-forge gym-box2d
In general, we test on Linux systems, but aim to keep the benchmark compatible with MacOS as much as possible. Mario at this point, however, will not run on any operation system besides Linux
To install the additional requirements for ToadGAN:
javac src/envs/mario/Mario-AI-Framework/**/*.java
If you want to use the RNA design environment:
cd src/envs/rna/learna
make requirements
make data
In case you want to run our experiments or use our training files, also install the experiment dependencies:
pip install -e .[experiments]
Train an Agent
To get started with CARL, you can use our 'train.py' script. It will train a PPO agent on the environment of your choice with custom context variations that are sampled from a standard deviation.
To use MetaCartPole with variations in gravity and friction by 20% compared to the default, run:
python train.py
--env CARLCartPoleEnv
--context_args gravity friction
--default_sample_std_percentage 0.2
--outdir <result_location>
You can use the plotting scripts in src/eval to view the results.
CARL's Contextual Extension
CARL contextually extends the environment by making the context visible and configurable. During training we therefore can encounter different contexts and train for generalization. We exemplarily show how Brax' Fetch is extended and embedded by CARL. Different instiations can be achieved by setting the context features to different values.
Cite Us
@misc{CARL,
author = {C. Benjamins and
T. Eimer and
F. Schubert and
A. Biedenkapp and
B. Rosenhahn and
F. Hutter and
M. Lindauer},
title = {CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning},
howpublished = {https://github.com/automl/CARL},
year = {2021},
month = aug,
}
References
OpenAI gym, Brockman et al., 2016. arXiv preprint arXiv:1606.01540
TOAD-GAN: Coherent Style Level Generation from a Single Example, Awiszus et al., AIIDE 2020
Learning to Design RNA, Runge et al., ICRL 2019
License
CARL falls under the Apache License 2.0 (see file 'LICENSE') as is permitted by all work that we use. This includes CARLMario, which is not based on the Nintendo Game, but on TOAD-GAN and TOAD-GUI running under an MIT license. They in turn make use of the Mario AI framework (https://github.com/amidos2006/Mario-AI-Framework). This is not the original game but a replica, explicitly built for research purposes and includes a copyright notice (https://github.com/amidos2006/Mario-AI-Framework#copyrights ).
15 Aug 20, 2022
1.2k Jan 04, 2023
328 Dec 17, 2022
79 Jan 05, 2023
44 Dec 14, 2022
381 Nov 11, 2022
501 Jan 08, 2023
268 Jan 07, 2023
338 Dec 29, 2022
19 Dec 28, 2022
2.6k Jan 04, 2023
6 Jan 25, 2022
41 Dec 10, 2022
68 Jan 04, 2023
10 Dec 07, 2022
112 Nov 07, 2022
37 Dec 12, 2022
62 Oct 25, 2022
41 Oct 21, 2022
37 Oct 25, 2022