Multi-Backend Architecture

Isaac Lab 3.0 introduced a multi-backend architecture that enables running simulations with different physics backends (PhysX, Newton, and OvPhysX) while maintaining a unified API. This page explains how the backend system works and how to extend it.

Overview

Instead of hard-coding a single physics engine, Isaac Lab uses a factory pattern to dispatch object creation to backend-specific implementations at runtime. When you write:

from isaaclab.assets import Articulation

robot = Articulation(cfg)

The Articulation class is a factory that automatically creates an instance of the active backend implementation, such as PhysX Articulation, Newton Articulation, or OvPhysX Articulation. Your code never needs to import backend-specific modules directly.

This pattern applies across simulation components, though not every backend implements every component yet:

Component	Core API (isaaclab)	PhysX (isaaclab_physx)	Newton (isaaclab_newton)	OvPhysX (isaaclab_ovphysx)
Physics Manager	PhysicsManager	PhysxManager	NewtonManager	OvPhysxManager
Articulation	Articulation	Articulation	Articulation	Articulation
Rigid Object	RigidObject	RigidObject	RigidObject	RigidObject
Deformable Object	DeformableObject	DeformableObject	DeformableObject	Not supported
Contact Sensor	ContactSensor	ContactSensor	ContactSensor	ContactSensor
Renderer	Renderer	IsaacRtxRenderer	NewtonWarpRenderer	Not supported
Scene Data Backend	SceneDataBackend	PhysxSceneDataBackend (in isaaclab_physx.physics)	NewtonSceneDataBackend (in isaaclab_newton.physics)	OvPhysxSceneDataBackend (in isaaclab_ovphysx.physics)
Cloner	usd_replicate()	physx_replicate()	newton_physics_replicate()	ovphysx_replicate()

The Factory Pattern

All factories inherit from FactoryBase, which uses a convention-over-configuration approach to locate backend implementations:

1. The active physics backend is determined by inspecting SimulationContext.physics_manager.

2. The factory’s module path is used to derive the backend module path by replacing isaaclab with isaaclab_{backend}. For example, isaaclab.assets.articulation maps to isaaclab_physx.assets.articulation, isaaclab_newton.assets.articulation, or isaaclab_ovphysx.assets.articulation.

3. The backend module is lazily imported and the implementation class is cached in a registry.

User code: Articulation(cfg)
    │
    ▼
FactoryBase.__new__()
    │
    ├─ _get_backend()       → "physx", "newton", or "ovphysx"
    │    (reads SimulationContext.physics_manager)
    │
    ├─ _get_module_name()   → "isaaclab_physx.assets.articulation"
    │    (convention: isaaclab.X.Y → isaaclab_{backend}.X.Y)
    │
    ├─ importlib.import_module()
    │    (lazy load — only on first use)
    │
    └─ Return backend-specific instance

Custom backend resolution: Some factories override the default resolution. For example, the Renderer factory selects backends based on the renderer config type rather than the physics manager, because renderers and physics backends are independent:

class Renderer(FactoryBase, BaseRenderer):
    _backend_class_names = {
        "physx": "IsaacRtxRenderer",
        "newton": "NewtonWarpRenderer",
        "ov": "OVRTXRenderer",
    }

Similarly, visualizers select backends based on the visualizer_type field in their config, allowing any visualizer to work with any physics backend.

Backend Selection

The physics backend is selected via the physics field in SimulationCfg:

from isaaclab.sim import SimulationCfg
from isaaclab_newton.physics import MJWarpSolverCfg, NewtonCfg
from isaaclab_ovphysx.physics import OvPhysxCfg
from isaaclab_physx.physics import PhysxCfg

# Use PhysX (default)
sim_cfg = SimulationCfg(physics=PhysxCfg())

# Use Newton with MuJoCo-Warp solver
sim_cfg = SimulationCfg(physics=NewtonCfg(
    solver_cfg=MJWarpSolverCfg(),
    num_substeps=4,
))

# Use OvPhysX
sim_cfg = SimulationCfg(physics=OvPhysxCfg())

Once the SimulationContext is initialized, all subsequent factory instantiations automatically use the selected backend.

Multi-Backend Environments with Presets

Environments can support multiple backends simultaneously using the preset system. Each backend gets its own configuration variant. The example below shows only the physics-related fields:

from isaaclab.envs import DirectRLEnvCfg
from isaaclab.sim import SimulationCfg
from isaaclab.utils.configclass import configclass
from isaaclab_newton.physics import MJWarpSolverCfg, NewtonCfg
from isaaclab_ovphysx.physics import OvPhysxCfg
from isaaclab_physx.physics import PhysxCfg
from isaaclab_tasks.utils import PresetCfg

@configclass
class CartpolePhysicsCfg(PresetCfg):
    default: PhysxCfg = PhysxCfg()
    physx: PhysxCfg = PhysxCfg()
    newton_mjwarp: NewtonCfg = NewtonCfg(
        solver_cfg=MJWarpSolverCfg(njmax=5, nconmax=3)
    )
    ovphysx: OvPhysxCfg = OvPhysxCfg()

@configclass
class CartpoleEnvCfg(DirectRLEnvCfg):
    sim: SimulationCfg = SimulationCfg(physics=CartpolePhysicsCfg())

Users then select a physics backend at the command line:

# Default (PhysX)
./isaaclab.sh train --rl_library rsl_rl --task Isaac-Cartpole-Direct-v0

# MJWarp (Newton backend)
./isaaclab.sh train --rl_library rsl_rl --task Isaac-Cartpole-Direct-v0 physics=newton_mjwarp

# OvPhysX backend
./isaaclab.sh train --rl_library rsl_rl --task Isaac-Cartpole-Direct-v0 physics=ovphysx

The Physics Manager

Each backend implements PhysicsManager, the abstract base class that drives the simulation loop:

class PhysicsManager(ABC):
    @classmethod
    @abstractmethod
    def initialize(cls, sim_context: SimulationContext) -> None: ...

    @classmethod
    @abstractmethod
    def reset(cls, soft: bool = False) -> None: ...

    @classmethod
    @abstractmethod
    def forward(cls) -> None: ...

    @classmethod
    @abstractmethod
    def step(cls) -> None: ...

    @classmethod
    def close(cls) -> None: ...  # concrete; dispatches STOP event

The physics manager also provides a callback system via PhysicsEvent for cross-backend event handling:

from isaaclab.physics import PhysicsManager, PhysicsEvent

handle = PhysicsManager.register_callback(
    callback=my_setup_fn,
    event=PhysicsEvent.PHYSICS_READY,
    order=0,
    name="my_callback",
)

Available events: MODEL_INIT (during scene building), PHYSICS_READY (after physics initialization), and STOP (on simulation shutdown).

Asset and Sensor Interfaces

Assets and sensors follow the same pattern. Each has:

1. A base class in isaaclab defining the interface (e.g., BaseArticulation, BaseContactSensor)

2. A factory class that inherits from both FactoryBase and the base class

3. Backend implementations in isaaclab_physx, isaaclab_newton, and isaaclab_ovphysx where supported

The base classes define the public API contract — properties, methods, and data accessors that all backends must provide. Current backend implementations use wp.array (Warp arrays) as their primary data type for asset and sensor data.

Data classes follow the same pattern with their own factories (e.g., ArticulationData(FactoryBase, BaseArticulationData)).

Adding a New Physics Backend

To add a new physics backend (e.g., mybackend), create a new extension package following the established conventions:

1. Package structure:

source/isaaclab_mybackend/
└── isaaclab_mybackend/
    ├── __init__.py
    ├── physics/
    │   ├── __init__.py           # lazy_export()
    │   ├── __init__.pyi          # public exports
    │   ├── mybackend_manager.py
    │   └── mybackend_manager_cfg.py
    ├── assets/
    │   ├── articulation/
    │   │   ├── __init__.py
    │   │   ├── __init__.pyi
    │   │   ├── articulation.py
    │   │   └── articulation_data.py
    │   ├── rigid_object/
    │   │   └── ...
    │   ├── deformable_object/
    │   │   └── ...
    │   └── rigid_object_collection/
    │       └── ...
    ├── sensors/
    │   ├── contact_sensor/
    │   └── ...
    ├── renderers/
    │   └── ...
    └── cloner/
        └── ...

2. Implement the physics manager:

The manager must expose a SceneDataBackend so that SceneDataProvider can read your backend’s body transforms in a Warp-native format that renderers and visualizers consume directly.

# isaaclab_mybackend/physics/mybackend_manager.py
from isaaclab.physics import PhysicsManager
from isaaclab.scene_data import SceneDataBackend, SceneDataFormat


class MyBackendSceneDataBackend(SceneDataBackend):
    def __init__(self):
        self._scene_data = SceneDataFormat.Transform()

    @property
    def transforms(self) -> SceneDataFormat.Transform:
        # Return current world-space body transforms as a Warp ``transformf`` array.
        self._scene_data.transforms = ...  # backend-native tensor view
        return self._scene_data

    @property
    def transform_count(self) -> int:
        ...

    @property
    def transform_paths(self) -> list[str]:
        # Prim path per row of ``transforms``; used by ``SceneDataProvider.create_mapping``.
        ...


class MyBackendManager(PhysicsManager):
    _scene_data_backend: ClassVar[MyBackendSceneDataBackend | None] = None

    @classmethod
    def initialize(cls, sim_context):
        super().initialize(sim_context)
        cls._scene_data_backend = MyBackendSceneDataBackend()
        # Initialize your physics engine

    @classmethod
    def get_scene_data_backend(cls) -> SceneDataBackend:
        return cls._scene_data_backend

    @classmethod
    def step(cls):
        # Advance simulation by one timestep

    @classmethod
    def forward(cls):
        # Update kinematics without stepping

    @classmethod
    def reset(cls, soft=False):
        if not soft:
            cls.dispatch_event(PhysicsEvent.PHYSICS_READY)
        # Reset simulation state

    @classmethod
    def close(cls):
        super().close()
        # Clean up resources

3. Create the physics config:

# isaaclab_mybackend/physics/mybackend_manager_cfg.py
from isaaclab.physics import PhysicsCfg
from isaaclab.utils.configclass import configclass

@configclass
class MyBackendCfg(PhysicsCfg):
    class_type = "{DIR}.mybackend_manager:MyBackendManager"
    # Backend-specific settings here

4. Implement assets and sensors:

Each asset/sensor must extend the corresponding base class from isaaclab. The class name must match the factory’s expected name (by convention, the same name as the factory class). Use lazy_export() in __init__.py files — no manual registration needed.

# isaaclab_mybackend/assets/articulation/articulation.py
from isaaclab.assets.articulation import BaseArticulation

class Articulation(BaseArticulation):
    def __init__(self, cfg):
        super().__init__(cfg)
        # Set up backend-specific simulation structures

5. Module discovery is automatic. The FactoryBase convention maps isaaclab.assets.articulation to isaaclab_mybackend.assets.articulation based on the active physics manager name. As long as you follow the package structure above, your backend classes will be discovered automatically.

Key Design Principles

• Lazy loading: Backend modules are imported only when first instantiated, keeping startup fast and avoiding hard dependencies on unused backends.

• Convention over configuration: Module paths follow a strict pattern (isaaclab.X.Y → isaaclab_{backend}.X.Y), so no manual registration is needed.

• Independent selection: Physics backend, renderer, and visualizer are selected independently — you can use any combination.

• Warp-native data types: Backend implementations return wp.array for asset and sensor data. Use wp.to_torch() when interoperating with PyTorch-based code.

• Zero runtime overhead: Backend selection happens at instantiation time. There are no if-statements or dispatch logic on the hot path.

See Also

• Migrating to Isaac Lab 3.0 — migration guide from Isaac Lab 2.x to the multi-backend architecture

• Hydra Configuration System — preset system for multi-backend environment configurations

• Physics Backends — feature matrix and per-backend guides (PhysX, Newton, OvPhysX)

• Newton Backend — Newton backend guide

• Newton Manager Abstraction — adding Newton solver managers and coupled solvers

• Renderers — renderer backend architecture