Skip to content

CAROL: Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretation

License

Notifications You must be signed in to change notification settings

chenxi-yang/CAROL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CAROL

This repository contains the main implementation for CAROL: Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretation. (SaTML'24)

Training Requirements

The main code is tested with Python 3.8.12. The required packages are listed in requirements.txt and can be installed with

pip install -r requirements.txt

pip install auto_LiRPA==0.3

pip install hydra-core==1.3.1

Additionally, MuJoCo 2.1 is required.

To run the code, you need to add this project's path in the PYTHONPATH with

export PYTHONPATH=$PYTHONPATH:/path/to/carol

Running

To run the training for experiments in the paper,

cd carol

python run.py overrides=hopper

python run.py overrides=walker2d

python run.py overrides=halfcheetah

python run.py overrides=ant

Test

We attach an example in /example_models with the policy and models for Hopper. overall indicates the example policy and the model trained together with this policy. seperate contains examples of the separately trained models used for the evaluation in Figure 3.

For the provability part, a new environment is needed and the packages can be intalled with

pip install -r requirements_proof.txt

The main change is for the PyTorch version.

To run provability experiments for this example (with the proof packages):

cd evaluation

python test_provability.py evaluation=provability overrides=hopper

To run an empirical attack on this example:

cd evaluation

python test_attack.py evaluation=attack overrides=hopper

About

CAROL: Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretation

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages