Implemented Algorithms#

robomimic includes several high-quality implementations of offline learning algorithms, and offers tools to easily build your own learning algorithms.

Imitation Learning#

BC#

  • Vanilla Behavioral Cloning (see this paper), consisting of simple supervised regression from observations to actions. Implemented in the BC class in algo/bc.py, along with some variants such as BC_GMM (stochastic GMM policy) and BC_VAE (stochastic VAE policy)

BC-RNN#

  • Behavioral Cloning with an RNN network. Implemented in the BC_RNN and BC_RNN_GMM (recurrent GMM policy) classes in algo/bc.py.

BC-Transformer#

  • Behavioral Cloning with an Transformer network. Implemented in the BC_Transformer and BC_Transformer_GMM (transformer GMM policy) classes in algo/bc.py.

HBC#

  • Hierarchical Behavioral Cloning - the implementation is largely based off of this paper. Implemented in the HBC class in algo/hbc.py.

Offline Reinforcement Learning#

IRIS#

  • A recent batch offline RL algorithm from this paper. Implemented in the IRIS class in algo/iris.py.

BCQ#

  • A recent batch offline RL algorithm from this paper. Implemented in the BCQ class in algo/bcq.py.

CQL#

  • A recent batch offline RL algorithm from this paper. Implemented in the CQL class in algo/cql.py.

IQL#

  • A recent batch offline RL algorithm from this paper. Implemented in the IQL class in algo/iql.py.

TD3-BC#

  • A recent algorithm from this paper. We implemented it as an example (see section below on building your own algorithm). Implemented in the TD3_BC class in algo/td3_bc.py.