cuMotion Integration

This page provides a high-level overview of the cuMotion integration to the Isaac Sim Motion Generation API. Detailed tutorials for each component are linked at the end of this page.

What is cuMotion?

cuMotion is a high-performance, GPU-accelerated Motion Generation library for robotic manipulation. This integration provides access to cuMotion’s planning and control algorithms within Isaac Sim:

• RMPflow: Real-time, reactive motion policies for smooth, collision-free motions

• Graph-Based Motion Planning: Sampling-based algorithms (RRT variants) for global path planning

• Trajectory Generation: Time-optimal collision-unaware trajectory generation from waypoints or path specifications

• Trajectory Optimization: Collision-free and kinematically constrained global optimization-based trajectory planning (new to cuMotion, did not exist in Lula)

Note

Windows Support: The Trajectory Optimizer is not currently available on Windows.

These algorithms make use of two main classes that manage the cuMotion robot and world:

• CumotionRobot - encapsulates the robot description, kinematics, and configuration

• CumotionWorldInterface - manages obstacles and world state for collision-aware planning

The cuMotion integration follows a minimal wrapping philosophy, directly exposing cuMotion’s powerful Python API while maintaining compatibility with Isaac Sim’s Motion Generation interfaces. cuMotion is descended from the Lula library, though its integration into Isaac Sim is substantially different. It is recommended to complete the full cuMotion tutorials even if you are familiar with the previous Lula integration.

Key Architectural Principles

Centralized World State

All world state (obstacles, robot base transforms) is managed by a single CumotionWorldInterface instance that is shared across all algorithms. This ensures consistency and simplifies obstacle management. The CumotionWorldInterface meets the WorldInterface defined by the Motion Generation API (see Scene Interaction), allowing automatic scene initialization and synchronization using the Motion Generation WorldBinding.

Minimal Wrapping

The integration only wraps what’s necessary for Isaac Sim compatibility. cuMotion’s native Python API is used directly for:

• Kinematics - Direct access to cumotion.Kinematics objects

• Path specifications - Use cuMotion’s native path specification API

• Parameter configuration - Full parameter structs are exposed for direct modification according to cuMotion’s documentation

Coordinate Frame Conversion

The cuMotion integration provides utility functions to convert between Isaac Sim world frame coordinates and cuMotion base frame coordinates. These helpers simplify working with the cuMotion Python API while using Isaac Sim coordinates. The general workflow is:

• Create the world coordinate you need in Isaac Sim (full pose, translation, or rotation)

• Convert to cuMotion base frame coordinates using the utility function

• Use the cuMotion python API with the base frame coordinates (call kinematics functions, create path specifications, etc.)

• If needed, convert back to Isaac Sim coordinates using the utility function

Integration with Motion Generation API

The cuMotion integration is built on top of the Isaac Sim Motion Generation API, providing:

• WorldInterface Implementation: CumotionWorldInterface implements the WorldInterface interface, enabling use with WorldBinding and SceneQuery

• BaseController Implementation: RmpFlowController implements the BaseController interface, enabling composition with other controllers

• Trajectory Interface: cuMotion trajectories implement the Trajectory interface, enabling use with TrajectoryFollower

Next Steps

For detailed information on each component, see the following sections:

Tutorials

• Robot Configuration Tutorial
  • Key Concepts
  • Loading Supported Robots
  • Creating Custom Robot Configurations
  • Robot Configuration Directory Structure
  • Accessing Robot Description and Kinematics
  • Loading Configuration Files
  • Summary
• cuMotion World Interface Tutorial
  • Key Concepts
  • Searching for Obstacles
  • Configuring Obstacle Representations
  • Creating the World Interface and World Binding
  • Synchronizing the World Binding
  • Updating Robot Base Transforms
  • Exploring the Tutorial
  • Summary
• RMPflow Tutorial
  • Key Concepts
  • Setting Up the Controller
  • Creating RobotState Objects
  • Resetting the Controller
  • Running the Controller
  • Updating World State
  • Accessing cuMotion Parameters
  • Example Usage
  • Summary
• Graph-Based Motion Planner Tutorial
  • Key Concepts
  • Initializing the Planner
  • Configuring Planner Parameters
  • Updating World State
  • Planning to C-Space Targets
  • Planning to Task-Space Targets
  • Planning to Translation-Only Targets
  • Executing Planned Paths
  • Example Usage
  • Summary
• Trajectory Generator Tutorial
  • Key Concepts
  • Initializing the Generator
  • Configuring Trajectory Parameters
  • Generating from C-Space Waypoints
  • Generating from Path Specifications
  • Task-Space Path Specifications
  • Executing Trajectories
  • Example Usage
  • Summary
• Trajectory Optimizer Tutorial
  • Key Concepts
  • Initializing the Optimizer
  • Accessing cuMotion Parameters
  • Updating World State
  • Planning to Configuration Space Targets
  • Planning to Task-Space Targets
  • Optional: Checking for Collisions
  • Executing Trajectories
  • Example Usage
  • Summary