Quickstart Guide

Quickstart Guide#

This guide is written for those who just can’t wait to get their hands dirty and will touch on the most common concepts you will encounter as you build your own projects with Isaac Lab! This includes installation, running RL, finding environments, creating new projects, and more!

The power of Isaac Lab comes from from a few key features that we will very briefly touch on in this guide.

Vectorization: Reinforcement Learning requires attempting a task many times. Isaac Lab speeds this process along by vectorizing the environment, a process by which training can be run in parallel across many copies of the same environment, thus reducing the amount of time spent on collecting data before the weights of the model can be updated. Most of the codebase is devoted to defining those parts of the environment that need to be touched by this vectorization system
Modular Design: Isaac Lab is designed to be modular, meaning that you can design your projects to have various components that can be swapped out for different needs. For example, suppose you want to train a policy that supports a specific subset of robots. You could design the environment and task to be robot agnostic by writing a controller interface layer in the form of one of our Manager classes (the ActionManager in this specific case). Most of the rest of the codebase is devoted to defining those parts of your project that need to be touched by this manager system.

To get started, we will first install Isaac Lab and launch a training script.

Launch Training#

The various backends of Isaac Lab are accessed through their corresponding train.py and play.py scripts located in the isaaclab/scripts/reinforcement_learning directory. Invoking these scripts will require a Task Name and a corresponding Entry Point to the gymnasium API. For example

python scripts/reinforcement_learning/skrl/train.py --task=Isaac-Ant-v0

This will train the mujoco ant to “run”. You can see the various launch option available to you with the --help flag. Note specifically the --num_envs option and the --headless flag, both of which can be useful when trying to develop and debug a new environment. Options specified at this level automatically overwrite any configuration equivalent that may be defined in the code (so long as those definitions are part of a @configclass, see below).

List Available Environments#

Above, Isaac-Ant-v0 is the task name and skrl is the RL framework being used. The Isaac-Ant-v0 environment has been registered with the Gymnasium API, and you can see how the entry point is defined by calling the list_envs.py script, which can be found in isaaclab/scripts/environments/lsit_envs.py. You should see entries like the following

$> python scripts/environments/list_envs.py

+--------------------------------------------------------------------------------------------------------------------------------------------+
|  Available Environments in Isaac Lab
+--------+----------------------+--------------------------------------------+---------------------------------------------------------------+
| S. No. | Task Name            | Entry Point                                | Config
.
.
.
+--------+----------------------+--------------------------------------------+---------------------------------------------------------------+
|   2    | Isaac-Ant-Direct-v0  |  isaaclab_tasks.direct.ant.ant_env:AntEnv  |  isaaclab_tasks.direct.ant.ant_env:AntEnvCfg
+--------+----------------------+--------------------------------------------+---------------------------------------------------------------+
.
.
.
+--------+----------------------+--------------------------------------------+---------------------------------------------------------------+
|   48   | Isaac-Ant-v0         | isaaclab.envs:ManagerBasedRLEnv            |   isaaclab_tasks.manager_based.classic.ant.ant_env_cfg:AntEnvCfg
+--------+----------------------+--------------------------------------------+---------------------------------------------------------------+

Notice that there are two different Ant tasks, one for a Direct environment and one for a ManagerBased environment. These are the two primary workflows that you can use with Isaac Lab out of the box. The Direct workflow will give you the shortest path to a working custom environment for reinforcement learning, but the Manager based workflow will give your project the modularity required for more generalized development. For the purposes of this quickstart guide, we will only focus on the Direct workflow.

Generate Your Own Project#

Getting a new project started with Isaac Lab can seem daunting at first, but this is why we provide the template generator, to rapidly boilerplate a new project via the command line.

./isaaclab.sh --new

This will create a new project for you based on the settings you choose

External vs Internal: Determines if the project is meant to be built as a part of the isaac lab repository, or if it is meant to be loaded as an external extension.
Direct vs Manager: A direct task primarily contains all the implementation details within the environment definition, while a manager based project is meant to use our modular definitions for the different “parts” of an environment.
Framework: You can select more than one option here. This determines which RL frameworks you intend to natively use with your project (which specific algorithm implementations you want to use for training).

Once created, navigate to the installed project and run

python -m pip install -e source/<given-project-name>

to complete the installation process and register the environment. Within the directories created by the template generator, you will find at least one __init__.py file with something that looks like the following

import gymnasium as gym

gym.register(
    id="Template-isaaclabtutorial_env-v0",
    entry_point=f"{__name__}.isaaclabtutorial_env:IsaaclabtutorialEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.isaaclabtutorial_env_cfg:IsaaclabtutorialEnvCfg",
        "skrl_cfg_entry_point": f"{agents.__name__}.skrl_ppo_cfg:PPORunnerCfg",
    },
)

This is the function that actually registers an environment for future use. Notice that the entry_point is literally just the python module path to the environment definition. This is why we need to install the project as a package: the module path is the entry point for the gymnasium API.

Configurations#

Regardless of what you are going to be doing with Isaac Lab, you will need to deal with Configurations. Configurations can all be identified by the inclusion of the @configclass decorator above their class definition and the lack of an __init__ function. For example, consider this configuration class for the cartpole environment.

@configclass
class CartpoleEnvCfg(DirectRLEnvCfg):
    # env
    decimation = 2
    episode_length_s = 5.0
    action_scale = 100.0  # [N]
    action_space = 1
    observation_space = 4
    state_space = 0

    # simulation
    sim: SimulationCfg = SimulationCfg(dt=1 / 120, render_interval=decimation)

    # robot
    robot_cfg: ArticulationCfg = CARTPOLE_CFG.replace(prim_path="/World/envs/env_.*/Robot")
    cart_dof_name = "slider_to_cart"
    pole_dof_name = "cart_to_pole"

    # scene
    scene: InteractiveSceneCfg = InteractiveSceneCfg(num_envs=4096, env_spacing=4.0, replicate_physics=True)

    # reset
    max_cart_pos = 3.0  # the cart is reset if it exceeds that position [m]
    initial_pole_angle_range = [-0.25, 0.25]  # the range in which the pole angle is sampled from on reset [rad]

    # reward scales
    rew_scale_alive = 1.0
    rew_scale_terminated = -2.0
    rew_scale_pole_pos = -1.0
    rew_scale_cart_vel = -0.01
    rew_scale_pole_vel = -0.005

Notice that the entire class definition is just a list of value fields and other configurations. Configuration classes are necessary for anything that needs to care about being vectorized by the lab during training. If you want to be able to copy an environment thousands of times, and manage the data from each asynchronously, you need to somehow “label” what parts of the scene matter to this copying process (vectorization). This is what the configuration classes accomplish!

In this case, the class defines the configuration for the entire training environment! Notice also the num_envs variable in the InteractiveSceneCfg. This actually gets overwritten by the CLI argument from within the train.py script. Configurations provide a direct path to any variable in the configuration hierarchy, making it easy to modify anything “configured” by the environment at launch time.

Robots#

Robots are entirely defined as instances of configurations within Isaac Lab. If you examine source/isaaclab_assets/isaaclab_assets/robots, you will see a number of files, each of which contains configurations for the robot in question. The purpose of these individual files is to better define scope for all the different robots, but there is nothing preventing you from adding your own to your project or even to the isaaclab repository! For example, consider the following configuration for the Dofbot

import isaaclab.sim as sim_utils
from isaaclab.actuators import ImplicitActuatorCfg
from isaaclab.assets.articulation import ArticulationCfg
from isaaclab.utils.assets import ISAAC_NUCLEUS_DIR

DOFBOT_CONFIG = ArticulationCfg(
    spawn=sim_utils.UsdFileCfg(
        usd_path=f"{ISAAC_NUCLEUS_DIR}/Robots/Dofbot/dofbot.usd",
        rigid_props=sim_utils.RigidBodyPropertiesCfg(
            disable_gravity=False,
            max_depenetration_velocity=5.0,
        ),
        articulation_props=sim_utils.ArticulationRootPropertiesCfg(
            enabled_self_collisions=True, solver_position_iteration_count=8, solver_velocity_iteration_count=0
        ),
    ),
    init_state=ArticulationCfg.InitialStateCfg(
        joint_pos={
            "joint1": 0.0,
            "joint2": 0.0,
            "joint3": 0.0,
            "joint4": 0.0,
        },
        pos=(0.25, -0.25, 0.0),
    ),
    actuators={
        "front_joints": ImplicitActuatorCfg(
            joint_names_expr=["joint[1-2]"],
            effort_limit_sim=100.0,
            velocity_limit_sim=100.0,
            stiffness=10000.0,
            damping=100.0,
        ),
        "joint3_act": ImplicitActuatorCfg(
            joint_names_expr=["joint3"],
            effort_limit_sim=100.0,
            velocity_limit_sim=100.0,
            stiffness=10000.0,
            damping=100.0,
        ),
        "joint4_act": ImplicitActuatorCfg(
            joint_names_expr=["joint4"],
            effort_limit_sim=100.0,
            velocity_limit_sim=100.0,
            stiffness=10000.0,
            damping=100.0,
        ),
    },
)

This completely defines the dofbot! You could copy this into a .py file and import it as a module and you would be able to use the dofbot in your own lab sims. One common feature you will see in any config defining things with state is the presence of an InitialStateCfg. Remember, the configurations are what informs vectorization, and it’s the InitialStateCfg that describes the state of the joints of our robot when it gets created in each environment. The ImplicitActuatorCfg defines the joints of the robot using the default actuation model determined by the joint time. Not all joints need to be actuated, but you will get warnings if you don’t. If you aren’t planning on using those undefined joints, you can generally ignore these.

Apps and Sims#

Using the simulation means launching the Isaac Sim app to provide simulation context. If you are not running a task defined by the standard workflows, then you are responsible for creating the app, managing the context, and stepping the simulation forward through time. This is the “third workflow”: a Standalone app, which is what we call the scripts for the frameworks, demos, benchmarks, etc…

The Standalone workflow gives you total control over everything in the app and simulation context. Developing standalone apps is discussed at length in the Isaac Sim documentation but there are a few points worth touching on that can be incredibly useful.

import argparse

from isaaclab.app import AppLauncher
# add argparse arguments
parser = argparse.ArgumentParser(
    description="This script demonstrates adding a custom robot to an Isaac Lab environment."
)
parser.add_argument("--num_envs", type=int, default=1, help="Number of environments to spawn.")
# append AppLauncher cli args
AppLauncher.add_app_launcher_args(parser)
# parse the arguments
args_cli = parser.parse_args()

# launch omniverse app
app_launcher = AppLauncher(args_cli)
simulation_app = app_launcher.app

The AppLauncher is the entrypoint to any and all Isaac Sim applications, like Isaac Lab! Many Isaac Lab and Isaac Sim modules cannot be imported until the app is launched!. This is done on the second to last line of the code above, when the AppLauncher is constructed. The app_launcher.app is our interface to the Kit App Framework; the broader interstitial code that binds the simulation to things the extension management system, or the GUI, etc… In the standalone workflow, this interface, often called the simulation_app is predominantly used to check if the simulation is running, and cleanup after the simulation finishes.