RPR: Robust Policy Regularization for Offline Reinforcement Learning

Implementation of the RPR algorithm.

Environment

Paper results were collected with MuJoCo 210 (and mujoco-py 2.1.2.14) in OpenAI gym 0.23.1 with the D4RL datasets. Networks are trained using PyTorch 2.1.1 and Python 3.9.

Usage

Offline RL Training

Use the following command to train offline RL on D4RL, including Gym locomotion and Antmaze tasks, and save the models.

python main.py --env halfcheetah-medium-v2 --nu 0.1 --delta 1  --save_model
python main.py --env antmaze-umaze-v2 --nu 0.5 --delta 3  --no_normalize --save_model

For all AntMaze tasks you must add the --no_normalize flag.