Внутри директории raisimLib запускаем

xhost si:localuser:root

export PYTORCH_CUDA_VERSION=$(ls /usr/local | grep -E 'cuda-[0-9]+\.[0-9]+$' | head -n1 | sed 's/cuda-//')
docker compose up --build

Внутри контейнера:

chmod +x /workspace/rma_entry.sh
/workspace/rma_entry.sh

Запустить обучение:

cd /workspace/rma/raisimGymTorch/raisimGymTorch/env/envs/rsg_go1_task
python runner.py --name random --gpu 0 --exptid 1 --overwrite

Run the following commands to install the training environments

# You might want to create a new conda environment if you did not do it already for the vision part
conda create --name cms python=3.8
conda activate cms
pip install -e .

# installation of the environments
python setup.py develop

You can use the following code to train a policy with access to priviledged information about the robot (e.g. mass, velocity, and motor strenght) and about the environment (e.g. the terrain geometry). The optimization will be guided by trajectories generated by a previous policy we provide. You can control the strenght of the imitation by changing the parameter RL_coeff in this file.

To start training, you can use the following commands:

cd raisimLib/raisimGymTorch/raisimGymTorch/env/envs/rsg_go1_task
python runner.py --name random --gpu 1 --exptid 1

It will take approximately 4K iterations to train a good enough policy. If you want to make any changes to the training environment, feel free to edit this file. Note that every time you make changes, you need to recompile the file by running this commands:

cd raisimLib/raisimGymTorch
python setup.py develop

If you wish to continue a previous run, use the following commands:

cd raisimLib/raisimGymTorch/raisimGymTorch/env/envs/rsg_go1_task
python runner.py --name random --gpu 1 --exptid 1 --loadid ITR_NBR --overwrite

You can use the following code to see if your policy training worked. First run the unity renderer:

cd raisimLib/raisimUnity/linux
./raisimUnity.x86_64

In a separate terminal, run the policy

conda activate cms
cd raisimLib/raisimGymTorch/raisimGymTorch/env/envs/rsg_go1_task
python viz_policy.py ../../../../data/rsg_go1_task/EXPT_ID POLICY_ID

You can now analysize the behaviour!

If you want to know how the policy performs over a set of controlled experiments, use the following commands:

conda activate cms
cd raisimLib/raisimGymTorch/raisimGymTorch/env/envs/rsg_go1_task
python evaluate_policy.py ../../../../data/rsg_go1_task/EXPT_ID POLICY_ID

This will generate a json file in the experiment folder. You can visualize the results in a nice table using this script:

python compute_results.py ../data/rsg_go1_task/EXPT_ID/evaluation_results.csv

If you want to make any changes to the evaluation, feel free to edit this file. Note that in the evaluation, the flag Eval is set to true. Remeber to recompile any time you edit the environment file!

A policy trained with priviledged information cannot be used on a physical robot. Therefore, we distill a priviledged policy using a slightly different version of RMA optimized for walking on complex terrains.

To start training, you can use the following commands:

cd raisimLib/raisimGymTorch/raisimGymTorch/env/envs/dagger_go1
python dagger.py --name cms_dagger --exptid 1 --loadpth ../../../../data/rsg_go1_task/EXPT_ID --loadid PRIV_POLICY_ID --gpu 1

It will take approximately 2K iterations to train a good enough policy. If you want to make any changes to the training environment, feel free to edit this file. Note that every time you make changes, you need to recompile the environment (see above).

You can follow exactly the same steps as for the priviledged policy (but now running commands from this folder) to visualize and benchmark a blind policy.