强化学习包装器

强化学习包装器#

我们提供对不同强化学习库的包装器。这些包装器将环境中的数据转换为各自库的函数参数和返回类型。

Stable-Baselines3#

在 Isaac-Cartpole-v0 上使用 Stable-Baselines3 训练一个智能体:

Linux

# install python module (for stable-baselines3)
./isaaclab.sh -i sb3
# run script for training
# note: we set the device to cpu since SB3 doesn't optimize for GPU anyway
./isaaclab.sh -p source/standalone/workflows/sb3/train.py --task Isaac-Cartpole-v0 --headless --device cpu
# run script for playing with 32 environments
./isaaclab.sh -p source/standalone/workflows/sb3/play.py --task Isaac-Cartpole-v0 --num_envs 32 --checkpoint /PATH/TO/model.zip
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p source/standalone/workflows/sb3/play.py --task Isaac-Cartpole-v0 --headless --video --video_length 200

Windows

:: install python module (for stable-baselines3)
isaaclab.bat -i sb3
:: run script for training
:: note: we set the device to cpu since SB3 doesn't optimize for GPU anyway
isaaclab.bat -p source\standalone\workflows\sb3\train.py --task Isaac-Cartpole-v0 --headless --device cpu
:: run script for playing with 32 environments
isaaclab.bat -p source\standalone\workflows\sb3\play.py --task Isaac-Cartpole-v0 --num_envs 32 --checkpoint /PATH/TO/model.zip
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p source\standalone\workflows\sb3\play.py --task Isaac-Cartpole-v0 --headless --video --video_length 200

SKRL#

使用 SKRL 在 Isaac-Reach-Franka-v0 上训练智能体:

PyTorch

Linux

# install python module (for skrl)
./isaaclab.sh -i skrl
# run script for training
./isaaclab.sh -p source/standalone/workflows/skrl/train.py --task Isaac-Reach-Franka-v0 --headless
# run script for playing with 32 environments
./isaaclab.sh -p source/standalone/workflows/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --checkpoint /PATH/TO/model.pt
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p source/standalone/workflows/skrl/play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200

Windows

:: install python module (for skrl)
isaaclab.bat -i skrl
:: run script for training
isaaclab.bat -p source\standalone\workflows\skrl\train.py --task Isaac-Reach-Franka-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p source\standalone\workflows\skrl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --checkpoint /PATH/TO/model.pt
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p source\standalone\workflows\skrl\play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200

JAX

# install python module (for skrl)
./isaaclab.sh -i skrl
# install skrl dependencies for JAX. Visit https://skrl.readthedocs.io/en/latest/intro/installation.html for more details
./isaaclab.sh -p -m pip install skrl["jax"]
# run script for training
./isaaclab.sh -p source/standalone/workflows/skrl/train.py --task Isaac-Reach-Franka-v0 --headless --ml_framework jax
# run script for playing with 32 environments
./isaaclab.sh -p source/standalone/workflows/skrl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32  --ml_framework jax --checkpoint /PATH/TO/model.pt
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p source/standalone/workflows/skrl/play.py --task Isaac-Reach-Franka-v0 --headless --ml_framework jax --video --video_length 200

用skrl训练多智能体环境 Isaac-Shadow-Hand-Over-Direct-v0 。

Linux

# install python module (for skrl)
./isaaclab.sh -i skrl
# run script for training with the MAPPO algorithm (IPPO is also supported)
./isaaclab.sh -p source/standalone/workflows/skrl/train.py --task Isaac-Shadow-Hand-Over-Direct-v0 --headless --algorithm MAPPO
# run script for playing with 32 environments with the MAPPO algorithm (IPPO is also supported)
./isaaclab.sh -p source/standalone/workflows/skrl/play.py --task Isaac-Shadow-Hand-Over-Direct-v0 --num_envs 32 --algorithm MAPPO --checkpoint /PATH/TO/model.pt

Windows

:: install python module (for skrl)
isaaclab.bat -i skrl
:: run script for training with the MAPPO algorithm (IPPO is also supported)
isaaclab.bat -p source\standalone\workflows\skrl\train.py --task Isaac-Shadow-Hand-Over-Direct-v0 --headless --algorithm MAPPO
:: run script for playing with 32 environments with the MAPPO algorithm (IPPO is also supported)
isaaclab.bat -p source\standalone\workflows\skrl\play.py --task Isaac-Shadow-Hand-Over-Direct-v0 --num_envs 32 --algorithm MAPPO --checkpoint /PATH/TO/model.pt

RL-Games#

使用 RL-Games 在 Isaac-Ant-v0 上训练智能体

Linux

# install python module (for rl-games)
./isaaclab.sh -i rl_games
# run script for training
./isaaclab.sh -p source/standalone/workflows/rl_games/train.py --task Isaac-Ant-v0 --headless
# run script for playing with 32 environments
./isaaclab.sh -p source/standalone/workflows/rl_games/play.py --task Isaac-Ant-v0 --num_envs 32 --checkpoint /PATH/TO/model.pth
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p source/standalone/workflows/rl_games/play.py --task Isaac-Ant-v0 --headless --video --video_length 200

Windows

:: install python module (for rl-games)
isaaclab.bat -i rl_games
:: run script for training
isaaclab.bat -p source\standalone\workflows\rl_games\train.py --task Isaac-Ant-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p source\standalone\workflows\rl_games\play.py --task Isaac-Ant-v0 --num_envs 32 --checkpoint /PATH/TO/model.pth
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p source\standalone\workflows\rl_games\play.py --task Isaac-Ant-v0 --headless --video --video_length 200

RSL-RL#

用 RSL-RL 在 Isaac-Reach-Franka-v0 上训练一个agent:

Linux

# install python module (for rsl-rl)
./isaaclab.sh -i rsl_rl
# run script for training
./isaaclab.sh -p source/standalone/workflows/rsl_rl/train.py --task Isaac-Reach-Franka-v0 --headless
# run script for playing with 32 environments
./isaaclab.sh -p source/standalone/workflows/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --load_run run_folder_name --checkpoint model.pt
# run script for recording video of a trained agent (requires installing `ffmpeg`)
./isaaclab.sh -p source/standalone/workflows/rsl_rl/play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200

Windows

:: install python module (for rsl-rl)
isaaclab.bat -i rsl_rl
:: run script for training
isaaclab.bat -p source\standalone\workflows\rsl_rl\train.py --task Isaac-Reach-Franka-v0 --headless
:: run script for playing with 32 environments
isaaclab.bat -p source\standalone\workflows\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --num_envs 32 --load_run run_folder_name --checkpoint model.pt
:: run script for recording video of a trained agent (requires installing `ffmpeg`)
isaaclab.bat -p source\standalone\workflows\rsl_rl\play.py --task Isaac-Reach-Franka-v0 --headless --video --video_length 200

所有上述脚本将训练进度记录到存储库根目录中的 logs 目录中的 Tensorboard 。logs目录遵循模式 logs/<library>/<task>/<date-time> ，其中 <library> 为学习框架的名称， <task> 为任务名称， <date-time> 为执行训练脚本的时间戳。

要查看日志，请运行:

Linux

# execute from the root directory of the repository
./isaaclab.sh -p -m tensorboard.main --logdir=logs

Windows

:: execute from the root directory of the repository
isaaclab.bat -p -m tensorboard.main --logdir=logs