Python API#

Warning

The API featured in this extension is experimental and subject to change without deprecation cycles. Although we will try to maintain backward compatibility in the event of a change, it may not always be possible.

The following table summarizes the available classes.

authoring (USD prim wrappers)

`Lidar`	High level class for creating/wrapping USD OmniLidar prims.
`Radar`	High level class for creating/wrapping USD OmniRadar prims.
`Acoustic`	High level class for creating/wrapping USD OmniAcoustic prims.
`RtxCamera`	High level class for creating/wrapping USD Camera prims as RTX sensors.
`StructuredLightCamera`	Structured light camera sensor: an `RtxCamera` with cycling projectors.

sensors (runtime)

`LidarSensor`	Runtime class for operating a single RTX-based lidar sensor.
`RadarSensor`	Runtime class for operating a single RTX-based radar sensor.
`AcousticSensor`	Runtime class for operating a single RTX-based acoustic sensor.
`CameraSensor`	High level class for creating/wrapping and operating single camera sensor.
`SingleViewDepthCameraSensor`	High level class for creating/wrapping and operating single view depth camera sensor.
`TiledCameraSensor`	High level class for creating/wrapping and operating tiled (batched) camera sensors.

utils

`parse_generic_model_output_data`	Parse generic model output structure from annotator data.
`parse_stable_id_map_data`	Parse Stable ID Map data from annotator data.

lidar configuration

SUPPORTED_LIDAR_CONFIGS
SUPPORTED_LIDAR_VARIANT_SET_NAME

Authoring#

Bases: _SensorAuthoring

High level class for creating/wrapping USD OmniLidar prims.

This class uses omni.replicator.core.functional.create.omni_lidar to create new lidar prims, which handles defining the prim, applying schemas, and setting attributes.

Note

This class creates or wraps (one of both) USD OmniLidar prims according to the following rules:

If the prim path exists, a wrapper is placed over the USD OmniLidar prim.
If the prim path does not exist, a USD OmniLidar prim is created at the path and a wrapper is placed over it.

Parameters:

path – Single path to existing or non-existing (one of both) USD OmniLidar prim. Can include regular expression for matching a prim.
accumulate_outputs – Set the omni:sensor:Core:accumulateOutputs attribute on the OmniLidar prim. When True (the default), the lidar model accumulates a full scan before generating an output. When None, the attribute is left untouched on the prim (useful for preserving values authored on a USD asset that is being wrapped).
aux_output_level – Auxiliary data level for GenericModelOutput. Valid values: "NONE" (default), "BASIC", "EXTRA", "FULL".
tick_rate – Sensor tick rate in Hz. When None (the default), the prim’s omni:sensor:tickRate attribute is left untouched, so any value already authored on the prim (e.g. from a USD asset) is preserved. For newly-created prims, the OmniSensorGenericLidarCoreAPI schema default of 10 Hz applies.
schemas – Additional API schemas to apply to the prim.
attributes – Attributes to set on the OmniLidar prim.
positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims to a standard set. See reset_xform_op_properties() for more details.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one prim.

Example:

>>> from isaacsim.sensors.experimental.rtx import Lidar
>>>
>>> # given a USD stage with the OmniLidar prim: /World/prim_0
>>> lidar = Lidar("/World/prim_0")  

Apply visual materials to the prims, and optionally, to their descendants.

Backends: usd.

Parameters:

materials – Visual materials to be applied to the prims (shape (N,)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
weaker_than_descendants – Boolean flags to indicate whether descendant materials should be overridden (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> from isaacsim.core.experimental.materials import OmniGlassMaterial
>>>
>>> # create a dark-red glass visual material
>>> material = OmniGlassMaterial("/World/material/glass")
>>> material.set_input_values("glass_ior", [1.25])
>>> material.set_input_values("depth", [0.001])
>>> material.set_input_values("thin_walled", [False])
>>> material.set_input_values("glass_color", [0.5, 0.0, 0.0])
>>>
>>> prims.apply_visual_materials(material)

Create a Lidar instance from a config name or USD file path.

Parameters:

path – Single path to existing or non-existing (one of both) USD OmniLidar prim.
accumulate_outputs – Set the omni:sensor:Core:accumulateOutputs attribute on the OmniLidar prim. When None (the default), the attribute authored on the loaded asset is preserved. Pass True/False to override.
aux_output_level – Auxiliary output level to author on the OmniLidar prim.
tick_rate – Sensor tick rate in Hz. When None (the default), the asset’s omni:sensor:tickRate attribute is preserved. Pass an explicit value to override.
schemas – Optional schema API names to apply to the created OmniLidar prim.
attributes – Attributes to set on the OmniLidar prim.
positions – Positions in the world frame (shape (N, 3)).
translations – Translations in the local frame (shape (N, 3)).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz).
scales – Scales to be applied to the prims (shape (N, 3)).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims.
config – Configuration name for the sensor (from SUPPORTED_LIDAR_CONFIGS).
usd_path – Path to a USD file containing the sensor asset.
variant – Variant name for the sensor configuration. Nested variants supported via dictionary; pairs applied in dict insertion order, so outer variant sets must come first.

Returns:

Lidar instance.

Raises:

ValueError – If both ‘config’ and ‘usd_path’ are provided.
ValueError – If the specified config is not found.

Example:

>>> from isaacsim.sensors.experimental.rtx import Lidar
>>>
>>> lidar = Lidar.create(
...     path="/World/lidar",
...     config="OS1",
...     variant="OS1_REV6_32ch20hz512res",
... )

static ensure_api(

prims: list[Usd.Prim],

api: type,

*args: Any,

**kwargs: Any,

) → list[type['UsdAPISchemaBase']]#

Ensure that all prims have the specified API schema applied.

Backends: usd.

If a prim doesn’t have the API schema, it will be applied. If it already has it, the existing API schema will be returned.

Parameters:

prims – List of USD Prims to ensure API schema on.
api – The API schema type to ensure.
*args – Additional positional arguments passed to API schema when applying it.
**kwargs – Additional keyword arguments passed to API schema when applying it.

Returns:

List of API schema objects, one for each input prim.

Example:

>>> import isaacsim.core.experimental.utils.prim as prim_utils
>>> from pxr import UsdPhysics
>>> from isaacsim.core.experimental.prims import Prim
>>>
>>> # given a USD stage with 3 prims at paths /World/prim_0, /World/prim_1, /World/prim_2,
>>> # ensure all prims have physics API schema
>>> usd_prims = [prim_utils.get_prim_at_path(f"/World/prim_{i}") for i in range(3)]
>>> physics_apis = Prim.ensure_api(usd_prims, UsdPhysics.RigidBodyAPI)

Get the applied visual materials.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: List of applied visual materials (shape (N,)). If a prim does not have a material, None is returned.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the applied visual material of the last wrapped prim
>>> prims.get_applied_visual_materials(indices=[2])[0]
<isaacsim.core.experimental.materials.impl.visual_materials.omni_glass.OmniGlassMaterial object at 0x...>

Get the default state (positions and orientations) of the prims.

Backends: usd.

Parameters:

indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Returns:

Two-elements tuple. 1) The default positions in the world frame (shape (N, 3)). 2) The default orientations in the world frame (shape (N, 4), quaternion wxyz). If the default state is not set using the set_default_state() method, None is returned.

Raises:

AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

get_local_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The translations in the local frame (shape (N, 3)). 2) The orientations in the local frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the local poses of all prims
>>> translations, orientations = prims.get_local_poses()
>>> translations.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the local pose of the first prim
>>> translations, orientations = prims.get_local_poses(indices=[0])
>>> translations.shape, orientations.shape
((1, 3), (1, 4))

get_local_scales( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Scales of the prims (shape (N, 3)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get local scales of all prims
>>> scales = prims.get_local_scales()
>>> scales.shape
(3, 3)

get_visibilities( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Boolean flags indicating the visibility state (shape (N, 1)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the visibility states of all prims
>>> visibilities = prims.get_visibilities()
>>> print(visibilities)
[[ True] [ True] [ True]]
>>>
>>> # get the visibility states of the first and last prims
>>> visibilities = prims.get_visibilities(indices=[0, 2])
>>> print(visibilities)
[[ True] [ True]]

get_world_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The positions in the world frame (shape (N, 3)). 2) The orientations in the world frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the world poses of all prims
>>> positions, orientations = prims.get_world_poses()
>>> positions.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the world pose of the first prim
>>> positions, orientations = prims.get_world_poses(indices=[0])
>>> positions.shape, orientations.shape
((1, 3), (1, 4))

reset_to_default_state( *, warn_on_non_default_state: bool = False, ) → None#

Reset the prims to the specified default state.

Backends: usd, usdrt, fabric.

This method applies the default state defined using the set_default_state() method.

Note

This method teleports prims to the specified default state (positions and orientations).

Warning

This method has no effect on non-root articulation links or when no default state is set. In this case, a warning message is logged if warn_on_non_default_state is True.

Parameters:: warn_on_non_default_state – Whether to log a warning message when no default state is set.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get default state (no default state set at this point)
>>> prims.get_default_state()
(None, None)
>>>
>>> # set default state
>>> # - random positions for each prim
>>> # - same fixed orientation for all prim
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> prims.set_default_state(positions, orientations=[1.0, 0.0, 0.0, 0.0])
>>>
>>> # get default state (default state is set)
>>> prims.get_default_state()
(array(shape=(3, 3), dtype=float32), array(shape=(3, 4), dtype=float32))
>>>
>>> # reset prims to default state
>>> prims.reset_to_default_state()

reset_xform_op_properties() → None#

Reset the transformation operation attributes of the prims to a standard set.

Backends: usd.

USD Xform schema supports a wide range of transformation operation types. This method ensures that each wrapped prim has only the following transformations in the specified order. Any other transformation operations are removed, so they are not consumed.

xformOp:translate (double precision)
xformOp:orient (double precision)
xformOp:scale (double precision)

Note

This method preserves the poses of the prims in the world frame while reorganizing the transformation operations.

Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # reset transform operations of all prims
>>> prims.reset_xform_op_properties()
>>>
>>> # verify transform operations of the first wrapped prim
>>> prims.prims[0].GetPropertyNames()
[... 'xformOp:orient', 'xformOp:scale', 'xformOp:translate', 'xformOpOrder']

static resolve_paths( paths: str | list[str], raise_on_mixed_paths: bool = True, ) → tuple[list[str], list[str]]#

Resolve paths to prims in the stage to get existing and non-existing paths.

Backends: usd.

Parameters:

paths – Single path or list of paths to USD prims. Paths may contain regular expressions to match multiple prims.
raise_on_mixed_paths – Whether to raise an error if resulting paths are mixed or invalid.

Returns:

Two-elements tuple. 1) List of existing paths. 2) List of non-existing paths.

Raises:

ValueError – If resulting paths are mixed or invalid and raise_on_mixed_paths is True.

Set the default state (positions and orientations) of the prims.

Backends: usd.

Hint

Prims can be reset to their default state by calling the reset_to_default_state() method.

Parameters:

positions – Default positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Default orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

Set the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the local frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither translations nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> translations = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_local_poses(translations, orientations)

Set the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:

scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # set random positive scales for all prims
>>> scales = np.random.uniform(low=0.5, high=1.5, size=(3, 3))
>>> prims.set_local_scales(scales)

Set the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:

visibilities – Boolean flags to set the visibility state (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # make all prims invisible
>>> prims.set_visibilities([False])
>>>
>>> # make first and last prims invisible
>>> prims.set_visibilities([True])  # restore visibility from previous call
>>> prims.set_visibilities([False], indices=[0, 2])

Set the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_world_poses(positions, orientations)

property aux_output_level: str#

The auxiliary output level configured on the GenericModelOutput RenderVar.

Returns:: The configured level (e.g. "NONE", "BASIC", "EXTRA", "FULL").

property is_non_root_articulation_link: bool#

Indicate if the wrapped prims are a non-root link in an articulation tree.

Backends: usd.

Warning

Transformation of the poses of non-root links in an articulation tree are not supported.

Returns:: Whether the prims are a non-root link in an articulation tree.

property paths: list[str]#

Prim paths in the stage encapsulated by the wrapper.

Returns:: List of prim paths as strings.

Example:

>>> prims.paths
['/World/prim_0', '/World/prim_1', '/World/prim_2']

property prims: list[pxr.Usd.Prim]#

USD Prim objects encapsulated by the wrapper.

Returns:: List of USD Prim objects.

Example:

>>> prims.prims
[Usd.Prim(</World/prim_0>), Usd.Prim(</World/prim_1>), Usd.Prim(</World/prim_2>)]

property valid: bool#

Whether all prims in the wrapper are valid.

Returns:: True if all prim paths specified in the wrapper correspond to valid prims in stage, False otherwise.

Example:

>>> prims.valid
True

Bases: _SensorAuthoring

High level class for creating/wrapping USD OmniRadar prims.

This class uses omni.replicator.core.functional.create.omni_radar to create new radar prims, which handles defining the prim, applying schemas, and setting attributes.

Note

RTX Radar requires Motion BVH to be enabled. The setting /renderer/raytracingMotion/enabled must be set to True before creating a radar prim.

Note

This class creates or wraps (one of both) USD OmniRadar prims according to the following rules:

If the prim path exists, a wrapper is placed over the USD OmniRadar prim.
If the prim path does not exist, a USD OmniRadar prim is created at the path and a wrapper is placed over it.

Parameters:

path – Single path to existing or non-existing (one of both) USD OmniRadar prim. Can include regular expression for matching a prim.
aux_output_level – Auxiliary data level for GenericModelOutput. Valid values: "NONE" (default), "BASIC".
tick_rate – Sensor tick rate in Hz. A value of 0 (the default) enables autotrigger mode.
attributes – Attributes to set on the OmniRadar prim.
positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims to a standard set. See reset_xform_op_properties() for more details.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one prim.
RuntimeError – If Motion BVH is not enabled when creating a new radar prim.

Example:

>>> from isaacsim.sensors.experimental.rtx import Radar
>>>
>>> # given a USD stage with the OmniRadar prim: /World/prim_0
>>> radar = Radar("/World/prim_0")  

Apply visual materials to the prims, and optionally, to their descendants.

Backends: usd.

Parameters:

materials – Visual materials to be applied to the prims (shape (N,)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
weaker_than_descendants – Boolean flags to indicate whether descendant materials should be overridden (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> from isaacsim.core.experimental.materials import OmniGlassMaterial
>>>
>>> # create a dark-red glass visual material
>>> material = OmniGlassMaterial("/World/material/glass")
>>> material.set_input_values("glass_ior", [1.25])
>>> material.set_input_values("depth", [0.001])
>>> material.set_input_values("thin_walled", [False])
>>> material.set_input_values("glass_color", [0.5, 0.0, 0.0])
>>>
>>> prims.apply_visual_materials(material)

Create a Radar instance from a config name or USD file path.

Parameters:

path – Single path to existing or non-existing (one of both) USD OmniRadar prim.
aux_output_level – Auxiliary data level for GenericModelOutput. Valid values: "NONE" (default), "BASIC".
tick_rate – Sensor tick rate in Hz. When None (the default), the asset’s omni:sensor:tickRate attribute is preserved. Pass an explicit value to override.
attributes – Attributes to set on the OmniRadar prim.
positions – Positions in the world frame (shape (N, 3)).
translations – Translations in the local frame (shape (N, 3)).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz).
scales – Scales to be applied to the prims (shape (N, 3)).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims.
config – Configuration name for the sensor (from SUPPORTED_RADAR_CONFIGS).
usd_path – Path to a USD file containing the sensor asset.
variant – Variant name for the sensor configuration. Nested variants supported via dictionary; pairs applied in dict insertion order, so outer variant sets must come first.

Returns:

Radar instance.

Raises:

ValueError – If both ‘config’ and ‘usd_path’ are provided.
ValueError – If the specified config is not found.
RuntimeError – If Motion BVH is not enabled.

Example:

>>> from isaacsim.sensors.experimental.rtx import Radar
>>>
>>> radar = Radar.create(path="/World/radar", config="IWRL6432AOP")

static ensure_api(

prims: list[Usd.Prim],

api: type,

*args: Any,

**kwargs: Any,

) → list[type['UsdAPISchemaBase']]#

Ensure that all prims have the specified API schema applied.

Backends: usd.

If a prim doesn’t have the API schema, it will be applied. If it already has it, the existing API schema will be returned.

Parameters:

prims – List of USD Prims to ensure API schema on.
api – The API schema type to ensure.
*args – Additional positional arguments passed to API schema when applying it.
**kwargs – Additional keyword arguments passed to API schema when applying it.

Returns:

List of API schema objects, one for each input prim.

Example:

>>> import isaacsim.core.experimental.utils.prim as prim_utils
>>> from pxr import UsdPhysics
>>> from isaacsim.core.experimental.prims import Prim
>>>
>>> # given a USD stage with 3 prims at paths /World/prim_0, /World/prim_1, /World/prim_2,
>>> # ensure all prims have physics API schema
>>> usd_prims = [prim_utils.get_prim_at_path(f"/World/prim_{i}") for i in range(3)]
>>> physics_apis = Prim.ensure_api(usd_prims, UsdPhysics.RigidBodyAPI)

Get the applied visual materials.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: List of applied visual materials (shape (N,)). If a prim does not have a material, None is returned.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the applied visual material of the last wrapped prim
>>> prims.get_applied_visual_materials(indices=[2])[0]
<isaacsim.core.experimental.materials.impl.visual_materials.omni_glass.OmniGlassMaterial object at 0x...>

Get the default state (positions and orientations) of the prims.

Backends: usd.

Parameters:

indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Returns:

Two-elements tuple. 1) The default positions in the world frame (shape (N, 3)). 2) The default orientations in the world frame (shape (N, 4), quaternion wxyz). If the default state is not set using the set_default_state() method, None is returned.

Raises:

AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

get_local_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The translations in the local frame (shape (N, 3)). 2) The orientations in the local frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the local poses of all prims
>>> translations, orientations = prims.get_local_poses()
>>> translations.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the local pose of the first prim
>>> translations, orientations = prims.get_local_poses(indices=[0])
>>> translations.shape, orientations.shape
((1, 3), (1, 4))

get_local_scales( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Scales of the prims (shape (N, 3)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get local scales of all prims
>>> scales = prims.get_local_scales()
>>> scales.shape
(3, 3)

get_visibilities( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Boolean flags indicating the visibility state (shape (N, 1)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the visibility states of all prims
>>> visibilities = prims.get_visibilities()
>>> print(visibilities)
[[ True] [ True] [ True]]
>>>
>>> # get the visibility states of the first and last prims
>>> visibilities = prims.get_visibilities(indices=[0, 2])
>>> print(visibilities)
[[ True] [ True]]

get_world_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The positions in the world frame (shape (N, 3)). 2) The orientations in the world frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the world poses of all prims
>>> positions, orientations = prims.get_world_poses()
>>> positions.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the world pose of the first prim
>>> positions, orientations = prims.get_world_poses(indices=[0])
>>> positions.shape, orientations.shape
((1, 3), (1, 4))

reset_to_default_state( *, warn_on_non_default_state: bool = False, ) → None#

Reset the prims to the specified default state.

Backends: usd, usdrt, fabric.

This method applies the default state defined using the set_default_state() method.

Note

This method teleports prims to the specified default state (positions and orientations).

Warning

This method has no effect on non-root articulation links or when no default state is set. In this case, a warning message is logged if warn_on_non_default_state is True.

Parameters:: warn_on_non_default_state – Whether to log a warning message when no default state is set.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get default state (no default state set at this point)
>>> prims.get_default_state()
(None, None)
>>>
>>> # set default state
>>> # - random positions for each prim
>>> # - same fixed orientation for all prim
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> prims.set_default_state(positions, orientations=[1.0, 0.0, 0.0, 0.0])
>>>
>>> # get default state (default state is set)
>>> prims.get_default_state()
(array(shape=(3, 3), dtype=float32), array(shape=(3, 4), dtype=float32))
>>>
>>> # reset prims to default state
>>> prims.reset_to_default_state()

reset_xform_op_properties() → None#

Reset the transformation operation attributes of the prims to a standard set.

Backends: usd.

USD Xform schema supports a wide range of transformation operation types. This method ensures that each wrapped prim has only the following transformations in the specified order. Any other transformation operations are removed, so they are not consumed.

xformOp:translate (double precision)
xformOp:orient (double precision)
xformOp:scale (double precision)

Note

This method preserves the poses of the prims in the world frame while reorganizing the transformation operations.

Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # reset transform operations of all prims
>>> prims.reset_xform_op_properties()
>>>
>>> # verify transform operations of the first wrapped prim
>>> prims.prims[0].GetPropertyNames()
[... 'xformOp:orient', 'xformOp:scale', 'xformOp:translate', 'xformOpOrder']

static resolve_paths( paths: str | list[str], raise_on_mixed_paths: bool = True, ) → tuple[list[str], list[str]]#

Resolve paths to prims in the stage to get existing and non-existing paths.

Backends: usd.

Parameters:

paths – Single path or list of paths to USD prims. Paths may contain regular expressions to match multiple prims.
raise_on_mixed_paths – Whether to raise an error if resulting paths are mixed or invalid.

Returns:

Two-elements tuple. 1) List of existing paths. 2) List of non-existing paths.

Raises:

ValueError – If resulting paths are mixed or invalid and raise_on_mixed_paths is True.

Set the default state (positions and orientations) of the prims.

Backends: usd.

Hint

Prims can be reset to their default state by calling the reset_to_default_state() method.

Parameters:

positions – Default positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Default orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

Set the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the local frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither translations nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> translations = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_local_poses(translations, orientations)

Set the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:

scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # set random positive scales for all prims
>>> scales = np.random.uniform(low=0.5, high=1.5, size=(3, 3))
>>> prims.set_local_scales(scales)

Set the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:

visibilities – Boolean flags to set the visibility state (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # make all prims invisible
>>> prims.set_visibilities([False])
>>>
>>> # make first and last prims invisible
>>> prims.set_visibilities([True])  # restore visibility from previous call
>>> prims.set_visibilities([False], indices=[0, 2])

Set the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_world_poses(positions, orientations)

property aux_output_level: str#

The auxiliary output level configured on the GenericModelOutput RenderVar.

Returns:: The configured level (e.g. "NONE", "BASIC", "EXTRA", "FULL").

property is_non_root_articulation_link: bool#

Indicate if the wrapped prims are a non-root link in an articulation tree.

Backends: usd.

Warning

Transformation of the poses of non-root links in an articulation tree are not supported.

Returns:: Whether the prims are a non-root link in an articulation tree.

property paths: list[str]#

Prim paths in the stage encapsulated by the wrapper.

Returns:: List of prim paths as strings.

Example:

>>> prims.paths
['/World/prim_0', '/World/prim_1', '/World/prim_2']

property prims: list[pxr.Usd.Prim]#

USD Prim objects encapsulated by the wrapper.

Returns:: List of USD Prim objects.

Example:

>>> prims.prims
[Usd.Prim(</World/prim_0>), Usd.Prim(</World/prim_1>), Usd.Prim(</World/prim_2>)]

property valid: bool#

Whether all prims in the wrapper are valid.

Returns:: True if all prim paths specified in the wrapper correspond to valid prims in stage, False otherwise.

Example:

>>> prims.valid
True

Bases: _SensorAuthoring

High level class for creating/wrapping USD OmniAcoustic prims.

Note

This class creates or wraps (one of both) USD OmniAcoustic prims according to the following rules:

If the prim path exists, a wrapper is placed over the USD OmniAcoustic prim.
If the prim path does not exist, a USD OmniAcoustic prim is created at the path and a wrapper is placed over it.

Parameters:

path – Single path to existing or non-existing (one of both) USD OmniAcoustic prim. Can include regular expression for matching a prim.
aux_output_level – Auxiliary data level for GenericModelOutput. Valid values: "NONE" (default), "BASIC".
tick_rate – Sensor tick rate in Hz. A value of 0 (the default) enables autotrigger mode.
attributes – Attributes to set on the OmniAcoustic prim.
positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims to a standard set. See reset_xform_op_properties() for more details.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one prim.

Example:

>>> from isaacsim.sensors.experimental.rtx import Acoustic
>>>
>>> # given a USD stage with the OmniAcoustic prim: /World/prim_0
>>> acoustic = Acoustic("/World/prim_0")  

Apply visual materials to the prims, and optionally, to their descendants.

Backends: usd.

Parameters:

materials – Visual materials to be applied to the prims (shape (N,)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
weaker_than_descendants – Boolean flags to indicate whether descendant materials should be overridden (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> from isaacsim.core.experimental.materials import OmniGlassMaterial
>>>
>>> # create a dark-red glass visual material
>>> material = OmniGlassMaterial("/World/material/glass")
>>> material.set_input_values("glass_ior", [1.25])
>>> material.set_input_values("depth", [0.001])
>>> material.set_input_values("thin_walled", [False])
>>> material.set_input_values("glass_color", [0.5, 0.0, 0.0])
>>>
>>> prims.apply_visual_materials(material)

Create an Acoustic instance from a config name or USD file path.

Parameters:

path – Single path to existing or non-existing (one of both) USD OmniAcoustic prim.
aux_output_level – Auxiliary data level for GenericModelOutput. Valid values: "NONE" (default), "BASIC".
tick_rate – Sensor tick rate in Hz. When None (the default), the asset’s omni:sensor:tickRate attribute is preserved. Pass an explicit value to override.
attributes – Attributes to set on the OmniAcoustic prim.
positions – Positions in the world frame (shape (N, 3)).
translations – Translations in the local frame (shape (N, 3)).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz).
scales – Scales to be applied to the prims (shape (N, 3)).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims.
config – Configuration name for the sensor (from SUPPORTED_ACOUSTIC_CONFIGS).
usd_path – Path to a USD file containing the sensor asset.
variant – Variant name for the sensor configuration. Nested variants supported via dictionary; pairs applied in dict insertion order, so outer variant sets must come first.

Returns:

Acoustic instance.

Raises:

ValueError – If both ‘config’ and ‘usd_path’ are provided.
ValueError – If the specified config is not found.

Example:

>>> from isaacsim.sensors.experimental.rtx import Acoustic
>>>
>>> acoustic = Acoustic.create(path="/World/acoustic")

static ensure_api(

prims: list[Usd.Prim],

api: type,

*args: Any,

**kwargs: Any,

) → list[type['UsdAPISchemaBase']]#

Ensure that all prims have the specified API schema applied.

Backends: usd.

If a prim doesn’t have the API schema, it will be applied. If it already has it, the existing API schema will be returned.

Parameters:

prims – List of USD Prims to ensure API schema on.
api – The API schema type to ensure.
*args – Additional positional arguments passed to API schema when applying it.
**kwargs – Additional keyword arguments passed to API schema when applying it.

Returns:

List of API schema objects, one for each input prim.

Example:

>>> import isaacsim.core.experimental.utils.prim as prim_utils
>>> from pxr import UsdPhysics
>>> from isaacsim.core.experimental.prims import Prim
>>>
>>> # given a USD stage with 3 prims at paths /World/prim_0, /World/prim_1, /World/prim_2,
>>> # ensure all prims have physics API schema
>>> usd_prims = [prim_utils.get_prim_at_path(f"/World/prim_{i}") for i in range(3)]
>>> physics_apis = Prim.ensure_api(usd_prims, UsdPhysics.RigidBodyAPI)

Get the applied visual materials.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: List of applied visual materials (shape (N,)). If a prim does not have a material, None is returned.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the applied visual material of the last wrapped prim
>>> prims.get_applied_visual_materials(indices=[2])[0]
<isaacsim.core.experimental.materials.impl.visual_materials.omni_glass.OmniGlassMaterial object at 0x...>

Get the default state (positions and orientations) of the prims.

Backends: usd.

Parameters:

indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Returns:

Two-elements tuple. 1) The default positions in the world frame (shape (N, 3)). 2) The default orientations in the world frame (shape (N, 4), quaternion wxyz). If the default state is not set using the set_default_state() method, None is returned.

Raises:

AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

get_local_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The translations in the local frame (shape (N, 3)). 2) The orientations in the local frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the local poses of all prims
>>> translations, orientations = prims.get_local_poses()
>>> translations.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the local pose of the first prim
>>> translations, orientations = prims.get_local_poses(indices=[0])
>>> translations.shape, orientations.shape
((1, 3), (1, 4))

get_local_scales( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Scales of the prims (shape (N, 3)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get local scales of all prims
>>> scales = prims.get_local_scales()
>>> scales.shape
(3, 3)

get_visibilities( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Boolean flags indicating the visibility state (shape (N, 1)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the visibility states of all prims
>>> visibilities = prims.get_visibilities()
>>> print(visibilities)
[[ True] [ True] [ True]]
>>>
>>> # get the visibility states of the first and last prims
>>> visibilities = prims.get_visibilities(indices=[0, 2])
>>> print(visibilities)
[[ True] [ True]]

get_world_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The positions in the world frame (shape (N, 3)). 2) The orientations in the world frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the world poses of all prims
>>> positions, orientations = prims.get_world_poses()
>>> positions.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the world pose of the first prim
>>> positions, orientations = prims.get_world_poses(indices=[0])
>>> positions.shape, orientations.shape
((1, 3), (1, 4))

reset_to_default_state( *, warn_on_non_default_state: bool = False, ) → None#

Reset the prims to the specified default state.

Backends: usd, usdrt, fabric.

This method applies the default state defined using the set_default_state() method.

Note

This method teleports prims to the specified default state (positions and orientations).

Warning

This method has no effect on non-root articulation links or when no default state is set. In this case, a warning message is logged if warn_on_non_default_state is True.

Parameters:: warn_on_non_default_state – Whether to log a warning message when no default state is set.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get default state (no default state set at this point)
>>> prims.get_default_state()
(None, None)
>>>
>>> # set default state
>>> # - random positions for each prim
>>> # - same fixed orientation for all prim
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> prims.set_default_state(positions, orientations=[1.0, 0.0, 0.0, 0.0])
>>>
>>> # get default state (default state is set)
>>> prims.get_default_state()
(array(shape=(3, 3), dtype=float32), array(shape=(3, 4), dtype=float32))
>>>
>>> # reset prims to default state
>>> prims.reset_to_default_state()

reset_xform_op_properties() → None#

Reset the transformation operation attributes of the prims to a standard set.

Backends: usd.

USD Xform schema supports a wide range of transformation operation types. This method ensures that each wrapped prim has only the following transformations in the specified order. Any other transformation operations are removed, so they are not consumed.

xformOp:translate (double precision)
xformOp:orient (double precision)
xformOp:scale (double precision)

Note

This method preserves the poses of the prims in the world frame while reorganizing the transformation operations.

Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # reset transform operations of all prims
>>> prims.reset_xform_op_properties()
>>>
>>> # verify transform operations of the first wrapped prim
>>> prims.prims[0].GetPropertyNames()
[... 'xformOp:orient', 'xformOp:scale', 'xformOp:translate', 'xformOpOrder']

static resolve_paths( paths: str | list[str], raise_on_mixed_paths: bool = True, ) → tuple[list[str], list[str]]#

Resolve paths to prims in the stage to get existing and non-existing paths.

Backends: usd.

Parameters:

paths – Single path or list of paths to USD prims. Paths may contain regular expressions to match multiple prims.
raise_on_mixed_paths – Whether to raise an error if resulting paths are mixed or invalid.

Returns:

Two-elements tuple. 1) List of existing paths. 2) List of non-existing paths.

Raises:

ValueError – If resulting paths are mixed or invalid and raise_on_mixed_paths is True.

Set the default state (positions and orientations) of the prims.

Backends: usd.

Hint

Prims can be reset to their default state by calling the reset_to_default_state() method.

Parameters:

positions – Default positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Default orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

Set the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the local frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither translations nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> translations = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_local_poses(translations, orientations)

Set the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:

scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # set random positive scales for all prims
>>> scales = np.random.uniform(low=0.5, high=1.5, size=(3, 3))
>>> prims.set_local_scales(scales)

Set the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:

visibilities – Boolean flags to set the visibility state (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # make all prims invisible
>>> prims.set_visibilities([False])
>>>
>>> # make first and last prims invisible
>>> prims.set_visibilities([True])  # restore visibility from previous call
>>> prims.set_visibilities([False], indices=[0, 2])

Set the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_world_poses(positions, orientations)

property aux_output_level: str#

The auxiliary output level configured on the GenericModelOutput RenderVar.

Returns:: The configured level (e.g. "NONE", "BASIC", "EXTRA", "FULL").

property is_non_root_articulation_link: bool#

Indicate if the wrapped prims are a non-root link in an articulation tree.

Backends: usd.

Warning

Transformation of the poses of non-root links in an articulation tree are not supported.

Returns:: Whether the prims are a non-root link in an articulation tree.

property paths: list[str]#

Prim paths in the stage encapsulated by the wrapper.

Returns:: List of prim paths as strings.

Example:

>>> prims.paths
['/World/prim_0', '/World/prim_1', '/World/prim_2']

property prims: list[pxr.Usd.Prim]#

USD Prim objects encapsulated by the wrapper.

Returns:: List of USD Prim objects.

Example:

>>> prims.prims
[Usd.Prim(</World/prim_0>), Usd.Prim(</World/prim_1>), Usd.Prim(</World/prim_2>)]

property valid: bool#

Whether all prims in the wrapper are valid.

Returns:: True if all prim paths specified in the wrapper correspond to valid prims in stage, False otherwise.

Example:

>>> prims.valid
True

Bases: _SensorAuthoring

High level class for creating/wrapping USD Camera prims as RTX sensors.

Applies the OmniSensorAPI schema to the underlying UsdGeom.Camera prim, enabling tick-rate-controlled rendering. Optical parameters (focal length, clipping range, aperture, etc.) are accessible via the camera property.

Note

This class creates or wraps (one of both) USD Camera prims according to the following rules:

If the prim path exists, a wrapper is placed over the USD Camera prim.
If the prim path does not exist, a USD Camera prim is created at the path and a wrapper is placed over it.

Parameters:

path – Single path to existing or non-existing (one of both) USD Camera prim. Can include regular expression for matching a prim.
tick_rate – Sensor tick rate in Hz. When None (the default), the prim’s omni:sensor:tickRate attribute is left untouched, so any value already authored on the prim (e.g. from a USD asset) is preserved. For newly-created prims, the OmniSensorAPI schema default of 0 Hz applies (autotrigger mode).
schemas – Additional API schemas to apply to the prim (e.g. ["OmniLensDistortionOpenCvFisheyeAPI"]). Supports multi-instance schemas via "SchemaName:instanceName" syntax.
attributes – Attributes to set on the Camera prim (applied after schemas, so schema-specific attributes can be set in the same call).
positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims to a standard set. See reset_xform_op_properties() for more details.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one prim.

Example:

>>> from isaacsim.sensors.experimental.rtx import RtxCamera
>>>
>>> cam = RtxCamera("/World/cam", tick_rate=30.0)
>>> cam.camera.set_focal_lengths(24.0)
>>> cam.camera.set_clipping_ranges(0.1, 100.0)

Apply visual materials to the prims, and optionally, to their descendants.

Backends: usd.

Parameters:

materials – Visual materials to be applied to the prims (shape (N,)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
weaker_than_descendants – Boolean flags to indicate whether descendant materials should be overridden (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> from isaacsim.core.experimental.materials import OmniGlassMaterial
>>>
>>> # create a dark-red glass visual material
>>> material = OmniGlassMaterial("/World/material/glass")
>>> material.set_input_values("glass_ior", [1.25])
>>> material.set_input_values("depth", [0.001])
>>> material.set_input_values("thin_walled", [False])
>>> material.set_input_values("glass_color", [0.5, 0.0, 0.0])
>>>
>>> prims.apply_visual_materials(material)

Create an RtxCamera instance, optionally from a known config or USD file.

Parameters:

path – Single path to existing or non-existing (one of both) USD Camera prim.
tick_rate – Sensor tick rate in Hz. When None (the default), the asset’s omni:sensor:tickRate attribute is preserved. Pass an explicit value to override.
schemas – Additional API schemas to apply to the prim (e.g. ["OmniLensDistortionOpenCvFisheyeAPI"]). Supports multi-instance schemas via "SchemaName:instanceName" syntax.
attributes – Attributes to set on the Camera prim (applied after schemas, so schema-specific attributes can be set in the same call).
positions – Positions in the world frame (shape (N, 3)).
translations – Translations in the local frame (shape (N, 3)).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz).
scales – Scales to be applied to the prims (shape (N, 3)).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims.
config – Configuration name for the sensor (from SUPPORTED_CAMERA_CONFIGS).
usd_path – Path to a USD file containing the camera asset.
variant – Variant name for the camera configuration. Nested variants supported via dictionary; pairs applied in dict insertion order, so outer variant sets must come first.

Returns:

RtxCamera instance.

Raises:

ValueError – If both config and usd_path are provided.
ValueError – If the specified config is not found in SUPPORTED_CAMERA_CONFIGS.

Example:

>>> from isaacsim.sensors.experimental.rtx import RtxCamera
>>>
>>> cam = RtxCamera.create(path="/World/cam", tick_rate=30.0)
>>> cam.camera.set_focal_lengths(24.0)

static ensure_api(

prims: list[Usd.Prim],

api: type,

*args: Any,

**kwargs: Any,

) → list[type['UsdAPISchemaBase']]#

Ensure that all prims have the specified API schema applied.

Backends: usd.

If a prim doesn’t have the API schema, it will be applied. If it already has it, the existing API schema will be returned.

Parameters:

prims – List of USD Prims to ensure API schema on.
api – The API schema type to ensure.
*args – Additional positional arguments passed to API schema when applying it.
**kwargs – Additional keyword arguments passed to API schema when applying it.

Returns:

List of API schema objects, one for each input prim.

Example:

>>> import isaacsim.core.experimental.utils.prim as prim_utils
>>> from pxr import UsdPhysics
>>> from isaacsim.core.experimental.prims import Prim
>>>
>>> # given a USD stage with 3 prims at paths /World/prim_0, /World/prim_1, /World/prim_2,
>>> # ensure all prims have physics API schema
>>> usd_prims = [prim_utils.get_prim_at_path(f"/World/prim_{i}") for i in range(3)]
>>> physics_apis = Prim.ensure_api(usd_prims, UsdPhysics.RigidBodyAPI)

Get the applied visual materials.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: List of applied visual materials (shape (N,)). If a prim does not have a material, None is returned.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the applied visual material of the last wrapped prim
>>> prims.get_applied_visual_materials(indices=[2])[0]
<isaacsim.core.experimental.materials.impl.visual_materials.omni_glass.OmniGlassMaterial object at 0x...>

Get the default state (positions and orientations) of the prims.

Backends: usd.

Parameters:

indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Returns:

Two-elements tuple. 1) The default positions in the world frame (shape (N, 3)). 2) The default orientations in the world frame (shape (N, 4), quaternion wxyz). If the default state is not set using the set_default_state() method, None is returned.

Raises:

AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

get_local_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The translations in the local frame (shape (N, 3)). 2) The orientations in the local frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the local poses of all prims
>>> translations, orientations = prims.get_local_poses()
>>> translations.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the local pose of the first prim
>>> translations, orientations = prims.get_local_poses(indices=[0])
>>> translations.shape, orientations.shape
((1, 3), (1, 4))

get_local_scales( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Scales of the prims (shape (N, 3)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get local scales of all prims
>>> scales = prims.get_local_scales()
>>> scales.shape
(3, 3)

get_visibilities( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Boolean flags indicating the visibility state (shape (N, 1)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the visibility states of all prims
>>> visibilities = prims.get_visibilities()
>>> print(visibilities)
[[ True] [ True] [ True]]
>>>
>>> # get the visibility states of the first and last prims
>>> visibilities = prims.get_visibilities(indices=[0, 2])
>>> print(visibilities)
[[ True] [ True]]

get_world_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The positions in the world frame (shape (N, 3)). 2) The orientations in the world frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the world poses of all prims
>>> positions, orientations = prims.get_world_poses()
>>> positions.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the world pose of the first prim
>>> positions, orientations = prims.get_world_poses(indices=[0])
>>> positions.shape, orientations.shape
((1, 3), (1, 4))

reset_to_default_state( *, warn_on_non_default_state: bool = False, ) → None#

Reset the prims to the specified default state.

Backends: usd, usdrt, fabric.

This method applies the default state defined using the set_default_state() method.

Note

This method teleports prims to the specified default state (positions and orientations).

Warning

This method has no effect on non-root articulation links or when no default state is set. In this case, a warning message is logged if warn_on_non_default_state is True.

Parameters:: warn_on_non_default_state – Whether to log a warning message when no default state is set.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get default state (no default state set at this point)
>>> prims.get_default_state()
(None, None)
>>>
>>> # set default state
>>> # - random positions for each prim
>>> # - same fixed orientation for all prim
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> prims.set_default_state(positions, orientations=[1.0, 0.0, 0.0, 0.0])
>>>
>>> # get default state (default state is set)
>>> prims.get_default_state()
(array(shape=(3, 3), dtype=float32), array(shape=(3, 4), dtype=float32))
>>>
>>> # reset prims to default state
>>> prims.reset_to_default_state()

reset_xform_op_properties() → None#

Reset the transformation operation attributes of the prims to a standard set.

Backends: usd.

USD Xform schema supports a wide range of transformation operation types. This method ensures that each wrapped prim has only the following transformations in the specified order. Any other transformation operations are removed, so they are not consumed.

xformOp:translate (double precision)
xformOp:orient (double precision)
xformOp:scale (double precision)

Note

This method preserves the poses of the prims in the world frame while reorganizing the transformation operations.

Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # reset transform operations of all prims
>>> prims.reset_xform_op_properties()
>>>
>>> # verify transform operations of the first wrapped prim
>>> prims.prims[0].GetPropertyNames()
[... 'xformOp:orient', 'xformOp:scale', 'xformOp:translate', 'xformOpOrder']

static resolve_paths( paths: str | list[str], raise_on_mixed_paths: bool = True, ) → tuple[list[str], list[str]]#

Resolve paths to prims in the stage to get existing and non-existing paths.

Backends: usd.

Parameters:

paths – Single path or list of paths to USD prims. Paths may contain regular expressions to match multiple prims.
raise_on_mixed_paths – Whether to raise an error if resulting paths are mixed or invalid.

Returns:

Two-elements tuple. 1) List of existing paths. 2) List of non-existing paths.

Raises:

ValueError – If resulting paths are mixed or invalid and raise_on_mixed_paths is True.

Set the default state (positions and orientations) of the prims.

Backends: usd.

Hint

Prims can be reset to their default state by calling the reset_to_default_state() method.

Parameters:

positions – Default positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Default orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

Set the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the local frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither translations nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> translations = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_local_poses(translations, orientations)

Set the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:

scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # set random positive scales for all prims
>>> scales = np.random.uniform(low=0.5, high=1.5, size=(3, 3))
>>> prims.set_local_scales(scales)

Set the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:

visibilities – Boolean flags to set the visibility state (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # make all prims invisible
>>> prims.set_visibilities([False])
>>>
>>> # make first and last prims invisible
>>> prims.set_visibilities([True])  # restore visibility from previous call
>>> prims.set_visibilities([False], indices=[0, 2])

Set the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_world_poses(positions, orientations)

property aux_output_level: str#

The auxiliary output level configured on the GenericModelOutput RenderVar.

Returns:: The configured level (e.g. "NONE", "BASIC", "EXTRA", "FULL").

property camera: Camera#

Camera object for accessing optical parameters.

Returns a Camera wrapper over the same USD prim, providing access to focal length, clipping range, aperture, and other optical properties.

Returns:: Camera object wrapping the sensor prim.

Example:

>>> cam = RtxCamera("/World/cam")
>>> cam.camera.set_focal_lengths(50.0)
>>> cam.camera.get_focal_lengths()

property is_non_root_articulation_link: bool#

Indicate if the wrapped prims are a non-root link in an articulation tree.

Backends: usd.

Warning

Transformation of the poses of non-root links in an articulation tree are not supported.

Returns:: Whether the prims are a non-root link in an articulation tree.

property paths: list[str]#

Prim paths in the stage encapsulated by the wrapper.

Returns:: List of prim paths as strings.

Example:

>>> prims.paths
['/World/prim_0', '/World/prim_1', '/World/prim_2']

property prims: list[pxr.Usd.Prim]#

USD Prim objects encapsulated by the wrapper.

Returns:: List of USD Prim objects.

Example:

>>> prims.prims
[Usd.Prim(</World/prim_0>), Usd.Prim(</World/prim_1>), Usd.Prim(</World/prim_2>)]

property valid: bool#

Whether all prims in the wrapper are valid.

Returns:: True if all prim paths specified in the wrapper correspond to valid prims in stage, False otherwise.

Example:

>>> prims.valid
True

Bases: RtxCamera

Structured light camera sensor: an RtxCamera with cycling projectors.

Extends RtxCamera by creating a set of UsdLux.RectLight prims (one per projector pattern) under a shared parent Xform and cycling through them based on the current simulation time and a list of user-supplied rational timestamps.

Note

This class is an authoring class — it creates and manages USD prims. For data acquisition, wrap an instance in CameraSensor:

cam = StructuredLightCamera(...)
sensor = CameraSensor(cam, resolution=(720, 1280), annotators=["rgb"])

Timestamps

projector_timestamps is a list of (numerator, denominator) rational tuples, one per pattern. Each tuple represents the simulation time (in seconds) at which that pattern becomes active. The first entry must represent \(t = 0\) (typically (0, 1), though any (0, k) is accepted) and the list must be strictly increasing. Rational tuples avoid the floating-point precision issues that arise when timestamps span nanoseconds to milliseconds.

After the last timestamp, the cycle repeats with period projector_cycle_period. If not supplied at construction, the period is inferred as timestamps[-1] + (timestamps[1] - timestamps[0]) for \(N \\geq 2\) or Fraction(1, 30) for \(N = 1\). Calling set_projector_timestamps() preserves an explicitly-supplied period and re-infers an implicit one.

Warnings

If the observed simulation dt exceeds the minimum projector timestamp interval (patterns would be impossible to resolve), a warning is logged once.
If a single tick advances the active pattern by more than one index (mod N), a warning is logged once per tick where this occurs.

Parameters:

path – Prim path of the USD Camera prim to create or wrap.
projector_light_patterns – Paths to projector pattern images (e.g. PNG files). One UsdLux.RectLight prim is created per pattern.
projector_direction_texture – Path to the projector direction texture (typically an EXR file) that defines the per-pixel projection direction. Applied to every pattern’s RectLight prim.
projector_prim_path – Prim path of the projector parent Xform. Defaults to f"{path}/projectors".
projector_position – World-frame position of the projector Xform (shape (3,)). Defaults to the camera’s world position.
projector_orientation – World-frame quaternion (wxyz) of the projector Xform (shape (4,)). Defaults to the camera’s world orientation.
projector_timestamps – Activation times for each pattern as list[tuple[int, int]]. First entry must be (0, 1); the list must be strictly increasing. Defaults to [(i, 30) for i in range(N)].
projector_cycle_period – Explicit cycle period as (numerator, denominator). Defaults to an inferred value (see above).
projector_intensity – RectLight intensity. Defaults to 150000.0.
projector_width – RectLight width. Defaults to 1.0.
projector_height – RectLight height. Defaults to 1.0.
tick_rate – RTX camera tick rate in Hz. See RtxCamera.
schemas – Additional API schemas to apply. See RtxCamera.
attributes – Additional attributes to set on the Camera prim. See RtxCamera.
positions – Camera world-frame positions (shape (1, 3)). See RtxCamera.
translations – Camera local-frame translations (shape (1, 3)). See RtxCamera.
orientations – Camera world-frame orientations (shape (1, 4), wxyz). See RtxCamera.
scales – Camera scales (shape (1, 3)). See RtxCamera.
reset_xform_op_properties – Whether to reset the Camera’s xformOp stack. See RtxCamera.

Raises:

ValueError – If projector_light_patterns is empty.
ValueError – If projector_direction_texture is None.
ValueError – If projector_timestamps is invalid (wrong length, first entry not zero, not strictly increasing).
ValueError – If projector_cycle_period is not strictly greater than the last timestamp.

Example:

>>> from pathlib import Path
>>> from isaacsim.sensors.experimental.rtx import StructuredLightCamera
>>>
>>> patterns = [Path(f"patterns/image_{i:02d}.png") for i in range(10)]
>>> timestamps = [(i, 1000) for i in range(10)]  # 1 ms spacing
>>> cam = StructuredLightCamera(
...     "/World/camera",
...     projector_light_patterns=patterns,
...     projector_direction_texture=Path("direction_texture.exr"),
...     projector_timestamps=timestamps,
... )  

Apply visual materials to the prims, and optionally, to their descendants.

Backends: usd.

Parameters:

materials – Visual materials to be applied to the prims (shape (N,)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
weaker_than_descendants – Boolean flags to indicate whether descendant materials should be overridden (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> from isaacsim.core.experimental.materials import OmniGlassMaterial
>>>
>>> # create a dark-red glass visual material
>>> material = OmniGlassMaterial("/World/material/glass")
>>> material.set_input_values("glass_ior", [1.25])
>>> material.set_input_values("depth", [0.001])
>>> material.set_input_values("thin_walled", [False])
>>> material.set_input_values("glass_color", [0.5, 0.0, 0.0])
>>>
>>> prims.apply_visual_materials(material)

Create an RtxCamera instance, optionally from a known config or USD file.

Parameters:

path – Single path to existing or non-existing (one of both) USD Camera prim.
tick_rate – Sensor tick rate in Hz. When None (the default), the asset’s omni:sensor:tickRate attribute is preserved. Pass an explicit value to override.
schemas – Additional API schemas to apply to the prim (e.g. ["OmniLensDistortionOpenCvFisheyeAPI"]). Supports multi-instance schemas via "SchemaName:instanceName" syntax.
attributes – Attributes to set on the Camera prim (applied after schemas, so schema-specific attributes can be set in the same call).
positions – Positions in the world frame (shape (N, 3)).
translations – Translations in the local frame (shape (N, 3)).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz).
scales – Scales to be applied to the prims (shape (N, 3)).
reset_xform_op_properties – Whether to reset the transformation operation attributes of the prims.
config – Configuration name for the sensor (from SUPPORTED_CAMERA_CONFIGS).
usd_path – Path to a USD file containing the camera asset.
variant – Variant name for the camera configuration. Nested variants supported via dictionary; pairs applied in dict insertion order, so outer variant sets must come first.

Returns:

RtxCamera instance.

Raises:

ValueError – If both config and usd_path are provided.
ValueError – If the specified config is not found in SUPPORTED_CAMERA_CONFIGS.

Example:

>>> from isaacsim.sensors.experimental.rtx import RtxCamera
>>>
>>> cam = RtxCamera.create(path="/World/cam", tick_rate=30.0)
>>> cam.camera.set_focal_lengths(24.0)

destroy() → None#

Unregister the app-update observer and release references.

Safe to call multiple times. After destroy(), the instance will no longer update the active pattern on simulation-time changes.

static ensure_api(

prims: list[Usd.Prim],

api: type,

*args: Any,

**kwargs: Any,

) → list[type['UsdAPISchemaBase']]#

Ensure that all prims have the specified API schema applied.

Backends: usd.

If a prim doesn’t have the API schema, it will be applied. If it already has it, the existing API schema will be returned.

Parameters:

prims – List of USD Prims to ensure API schema on.
api – The API schema type to ensure.
*args – Additional positional arguments passed to API schema when applying it.
**kwargs – Additional keyword arguments passed to API schema when applying it.

Returns:

List of API schema objects, one for each input prim.

Example:

>>> import isaacsim.core.experimental.utils.prim as prim_utils
>>> from pxr import UsdPhysics
>>> from isaacsim.core.experimental.prims import Prim
>>>
>>> # given a USD stage with 3 prims at paths /World/prim_0, /World/prim_1, /World/prim_2,
>>> # ensure all prims have physics API schema
>>> usd_prims = [prim_utils.get_prim_at_path(f"/World/prim_{i}") for i in range(3)]
>>> physics_apis = Prim.ensure_api(usd_prims, UsdPhysics.RigidBodyAPI)

get_active_pattern_index() → int#: Return the index of the currently active projector pattern (0-based).

Get the applied visual materials.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: List of applied visual materials (shape (N,)). If a prim does not have a material, None is returned.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the applied visual material of the last wrapped prim
>>> prims.get_applied_visual_materials(indices=[2])[0]
<isaacsim.core.experimental.materials.impl.visual_materials.omni_glass.OmniGlassMaterial object at 0x...>

Get the default state (positions and orientations) of the prims.

Backends: usd.

Parameters:

indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Returns:

Two-elements tuple. 1) The default positions in the world frame (shape (N, 3)). 2) The default orientations in the world frame (shape (N, 4), quaternion wxyz). If the default state is not set using the set_default_state() method, None is returned.

Raises:

AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

get_local_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The translations in the local frame (shape (N, 3)). 2) The orientations in the local frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the local poses of all prims
>>> translations, orientations = prims.get_local_poses()
>>> translations.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the local pose of the first prim
>>> translations, orientations = prims.get_local_poses(indices=[0])
>>> translations.shape, orientations.shape
((1, 3), (1, 4))

get_local_scales( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Scales of the prims (shape (N, 3)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get local scales of all prims
>>> scales = prims.get_local_scales()
>>> scales.shape
(3, 3)

get_num_patterns() → int#: Return the number of projector patterns.

get_projector_cycle_period() → tuple[int, int]#: Return the projector cycle period as a rational tuple.

get_projector_direction_texture() → str | Path#: Return the projector direction texture identifier as supplied at construction.

get_projector_prim_path() → str#: Return the prim path of the projector parent Xform.

get_projector_timestamps() → list[tuple[int, int]]#: Return the projector activation timestamps as rational tuples.

get_rect_light_prims() → list[pxr.Usd.Prim]#: Return the list of UsdLux.RectLight prims, one per pattern.

get_visibilities( *, indices: int | list | np.ndarray | wp.array | None = None, ) → wp.array#

Get the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Boolean flags indicating the visibility state (shape (N, 1)).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the visibility states of all prims
>>> visibilities = prims.get_visibilities()
>>> print(visibilities)
[[ True] [ True] [ True]]
>>>
>>> # get the visibility states of the first and last prims
>>> visibilities = prims.get_visibilities(indices=[0, 2])
>>> print(visibilities)
[[ True] [ True]]

get_world_poses( *, indices: int | list | np.ndarray | wp.array | None = None, ) → tuple[wp.array, wp.array]#

Get the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Parameters:: indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.
Returns:: Two-elements tuple. 1) The positions in the world frame (shape (N, 3)). 2) The orientations in the world frame (shape (N, 4), quaternion wxyz).
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get the world poses of all prims
>>> positions, orientations = prims.get_world_poses()
>>> positions.shape, orientations.shape
((3, 3), (3, 4))
>>>
>>> # get the world pose of the first prim
>>> positions, orientations = prims.get_world_poses(indices=[0])
>>> positions.shape, orientations.shape
((1, 3), (1, 4))

post_reset() → None#

Reset projector state so the next tick starts fresh.

Clears the previous-tick simulation-time cache, resets the coarse-dt warning flag, and activates pattern 0.

reset_to_default_state( *, warn_on_non_default_state: bool = False, ) → None#

Reset the prims to the specified default state.

Backends: usd, usdrt, fabric.

This method applies the default state defined using the set_default_state() method.

Note

This method teleports prims to the specified default state (positions and orientations).

Warning

This method has no effect on non-root articulation links or when no default state is set. In this case, a warning message is logged if warn_on_non_default_state is True.

Parameters:: warn_on_non_default_state – Whether to log a warning message when no default state is set.
Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # get default state (no default state set at this point)
>>> prims.get_default_state()
(None, None)
>>>
>>> # set default state
>>> # - random positions for each prim
>>> # - same fixed orientation for all prim
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> prims.set_default_state(positions, orientations=[1.0, 0.0, 0.0, 0.0])
>>>
>>> # get default state (default state is set)
>>> prims.get_default_state()
(array(shape=(3, 3), dtype=float32), array(shape=(3, 4), dtype=float32))
>>>
>>> # reset prims to default state
>>> prims.reset_to_default_state()

reset_xform_op_properties() → None#

Reset the transformation operation attributes of the prims to a standard set.

Backends: usd.

USD Xform schema supports a wide range of transformation operation types. This method ensures that each wrapped prim has only the following transformations in the specified order. Any other transformation operations are removed, so they are not consumed.

xformOp:translate (double precision)
xformOp:orient (double precision)
xformOp:scale (double precision)

Note

This method preserves the poses of the prims in the world frame while reorganizing the transformation operations.

Raises:: AssertionError – Wrapped prims are not valid.

Example:

>>> # reset transform operations of all prims
>>> prims.reset_xform_op_properties()
>>>
>>> # verify transform operations of the first wrapped prim
>>> prims.prims[0].GetPropertyNames()
[... 'xformOp:orient', 'xformOp:scale', 'xformOp:translate', 'xformOpOrder']

static resolve_paths( paths: str | list[str], raise_on_mixed_paths: bool = True, ) → tuple[list[str], list[str]]#

Resolve paths to prims in the stage to get existing and non-existing paths.

Backends: usd.

Parameters:

paths – Single path or list of paths to USD prims. Paths may contain regular expressions to match multiple prims.
raise_on_mixed_paths – Whether to raise an error if resulting paths are mixed or invalid.

Returns:

Two-elements tuple. 1) List of existing paths. 2) List of non-existing paths.

Raises:

ValueError – If resulting paths are mixed or invalid and raise_on_mixed_paths is True.

set_active_pattern_manual( pattern_index: int, ) → None#

Manually activate the pattern at pattern_index.

Bypasses time-based cycling until the next app-update tick, at which point the pattern will be re-selected based on the current simulation time. Useful for offline tests and deterministic capture loops.

Parameters:: pattern_index – Index of the pattern to activate.
Raises:: IndexError – If pattern_index is out of range.

Set the default state (positions and orientations) of the prims.

Backends: usd.

Hint

Prims can be reset to their default state by calling the reset_to_default_state() method.

Parameters:

positions – Default positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Default orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.
AssertionError – If prims are non-root articulation links.

Set the poses (translations and orientations) in the local frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

translations – Translations in the local frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the local frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither translations nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> translations = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_local_poses(translations, orientations)

Set the local scales of the prims.

Backends: usd, usdrt, fabric.

Parameters:

scales – Scales to be applied to the prims (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # set random positive scales for all prims
>>> scales = np.random.uniform(low=0.5, high=1.5, size=(3, 3))
>>> prims.set_local_scales(scales)

set_projector_cycle_period( period: tuple[int, int] | None, ) → None#

Set the projector cycle period.

Parameters:: period – New cycle period as (numerator, denominator), or None to mark the period as implicit and re-infer from the current timestamps.
Raises:: ValueError – If period is not strictly greater than the last timestamp.

set_projector_timestamps( timestamps: list[tuple[int, int]], ) → None#

Replace the projector activation timestamps.

Behavior depends on how the cycle period was originally set:

If the cycle period was implicit (inferred from the original timestamps), it is re-inferred from the new timestamps.
If the cycle period was explicit (supplied at construction or via set_projector_cycle_period()), it is preserved. The call raises ValueError if the explicit cycle period is no longer strictly greater than the new last timestamp.

After a successful call, the coarse-dt warning is rearmed, the previous simulation-time cache is cleared, and pattern 0 is re-activated so the next tick starts in a consistent state.

Parameters:: timestamps – New activation times. See class docstring for the schema.
Raises:: ValueError – If timestamps is invalid or if an explicit cycle period becomes inconsistent with the new timestamps.

Set the visibility state (whether prims are visible or invisible during rendering) of the prims.

Backends: usd.

Parameters:

visibilities – Boolean flags to set the visibility state (shape (N, 1)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – Wrapped prims are not valid.

Example:

>>> # make all prims invisible
>>> prims.set_visibilities([False])
>>>
>>> # make first and last prims invisible
>>> prims.set_visibilities([True])  # restore visibility from previous call
>>> prims.set_visibilities([False], indices=[0, 2])

Set the poses (positions and orientations) in the world frame of the prims.

Backends: usd, usdrt, fabric.

Note

This method teleports prims to the specified poses.

Parameters:

positions – Positions in the world frame (shape (N, 3)). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
orientations – Orientations in the world frame (shape (N, 4), quaternion wxyz). If the input shape is smaller than expected, data will be broadcasted (following NumPy broadcast rules).
indices – Indices of prims to process (shape (N,)). If not defined, all wrapped prims are processed.

Raises:

AssertionError – If neither positions nor orientations are specified.
AssertionError – Wrapped prims are not valid.

Example:

>>> # set random poses for all prims
>>> positions = np.random.uniform(low=-1, high=1, size=(3, 3))
>>> orientations = np.random.randn(3, 4)
>>> orientations = orientations / np.linalg.norm(orientations, axis=-1, keepdims=True)  # normalize quaternions
>>> prims.set_world_poses(positions, orientations)

property aux_output_level: str#

The auxiliary output level configured on the GenericModelOutput RenderVar.

Returns:: The configured level (e.g. "NONE", "BASIC", "EXTRA", "FULL").

property camera: Camera#

Camera object for accessing optical parameters.

Returns a Camera wrapper over the same USD prim, providing access to focal length, clipping range, aperture, and other optical properties.

Returns:: Camera object wrapping the sensor prim.

Example:

>>> cam = RtxCamera("/World/cam")
>>> cam.camera.set_focal_lengths(50.0)
>>> cam.camera.get_focal_lengths()

property is_non_root_articulation_link: bool#

Indicate if the wrapped prims are a non-root link in an articulation tree.

Backends: usd.

Warning

Transformation of the poses of non-root links in an articulation tree are not supported.

Returns:: Whether the prims are a non-root link in an articulation tree.

property paths: list[str]#

Prim paths in the stage encapsulated by the wrapper.

Returns:: List of prim paths as strings.

Example:

>>> prims.paths
['/World/prim_0', '/World/prim_1', '/World/prim_2']

property prims: list[pxr.Usd.Prim]#

USD Prim objects encapsulated by the wrapper.

Returns:: List of USD Prim objects.

Example:

>>> prims.prims
[Usd.Prim(</World/prim_0>), Usd.Prim(</World/prim_1>), Usd.Prim(</World/prim_2>)]

property valid: bool#

Whether all prims in the wrapper are valid.

Returns:: True if all prim paths specified in the wrapper correspond to valid prims in stage, False otherwise.

Example:

>>> prims.valid
True

Sensors#

Bases: _SensorRuntime

Runtime class for operating a single RTX-based lidar sensor.

Wraps a Lidar authoring object, attaches Replicator annotators, and provides get_data() to retrieve sensor output at simulation time.

Parameters:

path – Lidar object or single path to an existing or non-existing USD OmniLidar prim. If a string path is provided, a Lidar instance is created internally.
annotators – Annotator/sensor types to configure.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one prim.
ValueError – If an unsupported annotator type is specified.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> from isaacsim.sensors.experimental.rtx import LidarSensor
>>>
>>> # given a USD stage with the OmniLidar prim: /World/prim_0
>>> # and a USD Cube prim: /World/cube
>>> sensor = LidarSensor(
...     "/World/prim_0",
...     annotators=["generic-model-output"],
... )  
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)

property lidar: Lidar#

Lidar object encapsulated by the sensor.

Returns:: Lidar object encapsulated by the sensor.

Example:

>>> sensor.lidar
<isaacsim.sensors.experimental.rtx.impl.lidar.Lidar object at 0x...>

Bases: _SensorRuntime

Runtime class for operating a single RTX-based radar sensor.

Wraps a Radar authoring object, attaches Replicator annotators, and provides get_data() to retrieve sensor output at simulation time.

Note

RTX Radar requires Motion BVH to be enabled. The setting /renderer/raytracingMotion/enabled must be set to True before creating the radar prim.

Parameters:

path – Radar object or single path to an existing or non-existing USD OmniRadar prim. If a string path is provided, a Radar instance is created internally.
annotators – Annotator/sensor types to configure.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one prim.
ValueError – If an unsupported annotator type is specified.
RuntimeError – If Motion BVH is not enabled when creating a new radar prim.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> from isaacsim.sensors.experimental.rtx import RadarSensor
>>>
>>> # given a USD stage with the OmniRadar prim: /World/prim_0
>>> # and a USD Cube prim: /World/cube
>>> sensor = RadarSensor(
...     "/World/prim_0",
...     annotators=["generic-model-output"],
... )  
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)

property radar: Radar#

Radar object encapsulated by the sensor.

Returns:: Radar object encapsulated by the sensor.

Example:

>>> sensor.radar
<isaacsim.sensors.experimental.rtx.impl.radar.Radar object at 0x...>

Bases: _SensorRuntime

Runtime class for operating a single RTX-based acoustic sensor.

Wraps an Acoustic authoring object, attaches Replicator annotators, and provides get_data() to retrieve sensor output at simulation time.

Parameters:

path – Acoustic object or single path to an existing or non-existing USD OmniAcoustic prim. If a string path is provided, an Acoustic instance is created internally.
annotators – Annotator/sensor types to configure.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one prim.
ValueError – If an unsupported annotator type is specified.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> from isaacsim.sensors.experimental.rtx import Acoustic, AcousticSensor
>>>
>>> # given a USD stage with the OmniAcoustic prim: /World/prim_0
>>> # and a USD Cube prim: /World/cube
>>> acoustic = Acoustic("/World/prim_0", aux_output_level="BASIC")
>>> sensor = AcousticSensor(
...     acoustic,
...     annotators=["generic-model-output"],
... )  
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)

property acoustic: Acoustic#

Acoustic object encapsulated by the sensor.

Returns:: Acoustic object encapsulated by the sensor.

Example:

>>> sensor.acoustic
<isaacsim.sensors.experimental.rtx.impl.acoustic.Acoustic object at 0x...>

class CameraSensor( path: str | RtxCamera, *, resolution: tuple[int, int], annotators: Literal['bounding_box_2d_loose', 'bounding_box_2d_tight', 'bounding_box_3d', 'distance_to_camera', 'distance_to_image_plane', 'instance_id_segmentation', 'instance_segmentation', 'motion_vectors', 'normals', 'pointcloud', 'rgb', 'rgba', 'semantic_segmentation'] | list[Literal['bounding_box_2d_loose', 'bounding_box_2d_tight', 'bounding_box_3d', 'distance_to_camera', 'distance_to_image_plane', 'instance_id_segmentation', 'instance_segmentation', 'motion_vectors', 'normals', 'pointcloud', 'rgb', 'rgba', 'semantic_segmentation']] | None = None, writers: str | list[str] | None = None, render_vars: list[str] | None = None, )#

Bases: _SensorRuntime

High level class for creating/wrapping and operating single camera sensor.

Parameters:

path – RtxCamera object or single path to existing or non-existing USD Camera prim. If a string path is provided, a RtxCamera instance is created internally.
resolution – Resolution of the sensor (following OpenCV/NumPy convention: (height, width)).
annotators – Annotator/sensor types to configure.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one camera prim.
ValueError – If an unsupported annotator type is specified.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> from isaacsim.sensors.experimental.rtx import CameraSensor
>>>
>>> # given a USD stage with the Camera prim: /World/prim_0
>>> resolution = (240, 320)  # following OpenCV/NumPy convention `(height, width)`
>>> camera_sensor = CameraSensor(
...     "/World/prim_0",
...     resolution=resolution,
...     annotators=["rgb", "distance_to_image_plane"],
... )  
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)

attach_annotators(annotators: str | list[str]) → None#

Attach annotators to the sensor.

Parameters:: annotators – Annotator/sensor types to attach.
Raises:: ValueError – If the specified annotator is not supported.

get_data( annotator: str, *, out: wp.array | None = None, ) → tuple[wp.array | None, dict[str, Any]]#

Fetch the specified annotator/sensor data for the camera.

Parameters:

annotator – Annotator/sensor type from which fetch the data.
out – Pre-allocated array to fill with the fetched data.

Returns:

Two-elements tuple. 1) Array containing the fetched data. If no data is available at the moment of calling the method, None is returned. 2) Dictionary containing additional information according to the requested annotator/sensor.

Raises:

ValueError – If the specified annotator is not supported.
ValueError – If the specified annotator is not configured.

property camera: Any#

Camera object for accessing optical parameters.

Returns:: Camera object wrapping the sensor prim.

property resolution: tuple[int, int]#

Resolution of the sensor.

Returns:: (height, width)).
Return type:: Resolution of sensor frames (following OpenCV/NumPy convention

class SingleViewDepthCameraSensor( path: str | RtxCamera, *, resolution: tuple[int, int], annotators: Literal['bounding_box_2d_loose', 'bounding_box_2d_tight', 'bounding_box_3d', 'distance_to_camera', 'distance_to_image_plane', 'instance_id_segmentation', 'instance_segmentation', 'motion_vectors', 'normals', 'pointcloud', 'semantic_segmentation', 'depth_sensor_distance', 'depth_sensor_imager', 'depth_sensor_point_cloud_color', 'depth_sensor_point_cloud_position'] | list[Literal['bounding_box_2d_loose', 'bounding_box_2d_tight', 'bounding_box_3d', 'distance_to_camera', 'distance_to_image_plane', 'instance_id_segmentation', 'instance_segmentation', 'motion_vectors', 'normals', 'pointcloud', 'semantic_segmentation', 'depth_sensor_distance', 'depth_sensor_imager', 'depth_sensor_point_cloud_color', 'depth_sensor_point_cloud_position']], )#

Bases: CameraSensor

High level class for creating/wrapping and operating single view depth camera sensor.

The sensor is modeled using a single camera view to simulate a stereo camera pair and compute disparity and depth. The sensor is implemented as a post-process operation in the renderer, where the OmniSensorDepthSensorSingleViewAPI schema is applied to the USD render product prim rather than to the camera prim.

Parameters:

path – Camera object or single path to existing or non-existing (one of both) USD Camera prim. Can include regular expression for matching a prim.
resolution – Resolution of the sensor (following OpenCV/NumPy convention: (height, width)).
annotators – Annotator/sensor types to configure.

Raises:

ValueError – If no prim is found matching the specified path.
ValueError – If the input argument refers to more than one camera prim.
ValueError – If an unsupported annotator type is specified.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> from isaacsim.sensors.experimental.rtx import SingleViewDepthCameraSensor
>>>
>>> # given a USD stage with the Camera prim: /World/prim_0
>>> resolution = (240, 320)  # following OpenCV/NumPy convention `(height, width)`
>>> camera_sensor = SingleViewDepthCameraSensor(
...     "/World/prim_0",
...     resolution=resolution,
...     annotators="depth_sensor_distance",
... )  
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)

static add_template_render_product(

parent_prim_path: str,

camera_prim_path: str,

**kwargs: Any,

) → pxr.Usd.Prim#

Add a template render product for a depth sensor to the USD stage.

Creates a RenderProduct prim with OmniSensorDepthSensorSingleViewAPI applied and a camera relationship pointing to the given camera prim. The render product is created as a child of parent_prim_path and is named <camera_name>_render_product.

This is used when building a depth camera USD asset for later use with SingleViewDepthCameraSensor. When the asset is loaded via RtxCamera.create(), SingleViewDepthCameraSensor automatically detects the embedded render product and copies its depth sensor attributes onto the dynamically created render product.

Parameters:

parent_prim_path – USD path to the parent prim under which the RenderProduct will be created (trailing slash is stripped automatically).
camera_prim_path – USD path to the Camera prim to associate with the RenderProduct.
**kwargs – Depth sensor attribute names and values to set on the RenderProduct (e.g. omni:rtx:post:depthSensor:baselineMM=42). A warning is logged for any key that does not correspond to an existing attribute on the prim.

Returns:

The created RenderProduct prim, or an invalid pxr.Usd.Prim if creation failed.

Example:

>>> SingleViewDepthCameraSensor.add_template_render_product(
...     parent_prim_path="/root/TemplateRenderProduct",
...     camera_prim_path="/root/Camera",
...     **{"omni:rtx:post:depthSensor:baselineMM": 42},
... )

get_enabled_outlier_removal() → bool#

Get the enabled state of the outlier removal filter of the depth sensor.

Filter out single pixel samples caused by antialiasing jitter and reprojection resolution.

Returns:: Boolean flag indicating if the outlier removal filter is enabled.

Example:

>>> camera_sensor.get_enabled_outlier_removal()
True

get_enabled_post_processing() → bool#

Get the enabled state of the post-process operation for depth sensing in the renderer of the prims.

Returns:: Boolean flag indicating if the depth sensor post-process is enabled.

Example:

>>> camera_sensor.get_enabled_post_processing()
True

get_sensor_baseline() → float#

Get the distance between the simulated depth camera sensor, in millimeters.

Larger positive/negative values will increase the unknown black/hole regions around objects where the depth camera sensor cannot see.

Returns:: The distance between the simulated depth camera sensor, in millimeters.

Example:

>>> camera_sensor.get_sensor_baseline()
55.0

get_sensor_disparity_confidence() → float#

Get the confidence threshold for the depth sensor.

Control how likely a depth sample is considered valid. Higher values make depth values vary wider across the quantization (noise mean) range.

Returns:: The confidence threshold for the depth sensor.

Example:

>>> camera_sensor.get_sensor_disparity_confidence()
 0.6999...

get_sensor_disparity_noise_downscale() → float#

Get the coarseness of the disparity noise, in pixels, of the depth sensor.

Higher values reduce the spatial resolution of the noise.

Returns:: The disparity noise downscale factor, in pixels.

Example:

>>> camera_sensor.get_sensor_disparity_noise_downscale()
1.0

get_sensor_distance_cutoffs() → tuple[float, float]#

Get the cutoff range (minimum and maximum distance) of the depth sensor.

Returns:: The minimum cutoff distance. The maximum cutoff distance.

Example:

>>> camera_sensor.get_sensor_distance_cutoffs()
(0.5, 10000000.0)

get_sensor_focal_length() → float#

Get the simulated focal length of the depth sensor, in pixels.

Combined with the sensor size, this sets the field of view for the disparity calculation. Since the actual FOV is controlled on the camera prim, this only adjusts the amount of left/right disparity. Lower focal length decreases disparity.

Returns:: The sensor focal length, in pixels.

Example:

>>> camera_sensor.get_sensor_focal_length()
897.0

get_sensor_maximum_disparity() → float#

Get the maximum number of disparity pixels for the depth sensor.

Higher values allow the sensor to resolve closer (more disparate) objects. Lower values reduce the depth sensing range.

Returns:: The maximum number of disparity pixels for the depth sensor.

Example:

>>> camera_sensor.get_sensor_maximum_disparity()
110.0

get_sensor_noise_parameters() → tuple[float, float]#

Get the quantization factor (mean and sigma) for the disparity noise, in pixels, of the depth sensor.

Returns:: Two-elements tuple. 1) The disparity noise mean value, in pixels. 2) The disparity noise sigma value, in pixels.

Example:

>>> camera_sensor.get_sensor_noise_parameters()
(0.25, 0.25)

get_sensor_output_mode() → int#

Get the output mode used to override the LDRColor buffer with a debug visualization of the depth sensor.

Supported modes:

0: Pass through LDRColor.
1: Repeated 1 meter grayscale gradient.
2: Grayscale gradient over min/max distance.
3: Rainbow gradient over min/max distance.
4: Input Depth values in grayscale.
5: Reprojected depth with confidence culling applied.
6: Confidence Map with Disparity.
7: Disparity values in grayscale.

Returns:: The output mode.

Example:

>>> camera_sensor.get_sensor_output_mode()
0

get_sensor_size() → float#

Get the size of the sensor, in pixels, of the depth sensor.

Combined with focal length, this affects the amount of disparity. Higher values decrease disparity.

Returns:: The sensor size, in pixels.

Example:

>>> camera_sensor.get_sensor_size()
1280.0

set_enabled_outlier_removal( enabled: bool, ) → None#

Enable or disable the outlier removal filter of the depth sensor.

Filter out single pixel samples caused by antialiasing jitter and reprojection resolution.

Parameters:: enabled – Boolean flag to enable/disable outlier removal.

Example:

>>> camera_sensor.set_enabled_outlier_removal(True)

set_enabled_post_processing( enabled: bool, ) → None#

Enable or disable the post-process operation for depth sensing in the renderer.

Parameters:: enabled – Boolean flag to enable/disable the depth sensor post-process.

Example:

>>> camera_sensor.set_enabled_post_processing(True)

set_sensor_baseline( baseline: float, ) → None#

Set the distance between the simulated depth camera sensor, in millimeters.

Larger positive/negative values will increase the unknown black/hole regions around objects where the camera sensor cannot see.

Parameters:: baseline – Sensor baseline in millimeters.

Example:

>>> camera_sensor.set_sensor_baseline(50.0)

set_sensor_disparity_confidence( confidence_threshold: float, ) → None#

Set the confidence threshold for the depth sensor.

Control how likely a depth sample is considered valid. Higher values make depth values vary wider across the quantization (noise mean) range.

Parameters:: confidence_threshold – Confidence threshold.

Example:

>>> camera_sensor.set_sensor_disparity_confidence(0.75)

set_sensor_disparity_noise_downscale( downscale: float, ) → None#

Set the coarseness of the disparity noise, in pixels, of the depth sensor.

Higher values reduce the spatial resolution of the noise.

Parameters:: downscale – Disparity noise downscale factor in pixels.

Example:

>>> camera_sensor.set_sensor_disparity_noise_downscale(1.5)

set_sensor_distance_cutoffs( minimum_distance: float = None, maximum_distance: float = None, ) → None#

Set the cutoff range (minimum and maximum distance) of the depth sensor.

Parameters:

minimum_distance – Minimum cutoff distance.
maximum_distance – Maximum cutoff distance.

Raises:

ValueError – If both minimum_distance and maximum_distance are not defined.

Example:

>>> camera_sensor.set_sensor_distance_cutoffs(minimum_distance=0.1, maximum_distance=1000000.0)

set_sensor_focal_length( focal_length: float, ) → None#

Set the simulated focal length of the depth sensor, in pixels.

Combined with the sensor size, this sets the field of view for the disparity calculation. Since the actual FOV is controlled on the camera prim, this only adjusts the amount of left/right disparity. Lower focal length decreases disparity.

Parameters:: focal_length – Sensor focal length in pixels.

Example:

>>> camera_sensor.set_sensor_focal_length(800.0)

set_sensor_maximum_disparity( maximum_disparity: float, ) → None#

Set the maximum number of disparity pixels for the depth sensor.

Higher values allow the sensor to resolve closer (more disparate) objects. Lower values reduce the depth sensing range.

Parameters:: maximum_disparity – Maximum disparity.

Example:

>>> camera_sensor.set_sensor_maximum_disparity(120.0)

set_sensor_noise_parameters( noise_mean: float = None, noise_sigma: float = None, ) → None#

Set the quantization factor (mean and sigma) for the disparity noise, in pixels, of the depth sensor.

Higher mean values reduce depth resolution. Higher sigma values make depth values vary wider across the quantization (noise mean) range.

Parameters:

noise_mean – Disparity noise mean value in pixels.
noise_sigma – Disparity noise sigma value in pixels.

Raises:

AssertionError – If neither noise_mean nor noise_sigma are specified.

Example:

>>> camera_sensor.set_sensor_noise_parameters(noise_mean=0.5, noise_sigma=0.1)

set_sensor_output_mode(mode: int) → None#

Set the output mode to override the LDRColor buffer with a debug visualization of the depth sensor.

Supported modes:

0: Pass through LDRColor.
1: Repeated 1 meter grayscale gradient.
2: Grayscale gradient over min/max distance.
3: Rainbow gradient over min/max distance.
4: Input Depth values in grayscale.
5: Reprojected depth with confidence culling applied.
6: Confidence Map with Disparity.
7: Disparity values in grayscale.

Parameters:: mode – Output mode.

Example:

>>> camera_sensor.set_sensor_output_mode(0)  # pass through LDRColor

set_sensor_size(size: float) → None#

Set the size of the sensor, in pixels, of the depth sensor.

Combined with focal length, this affects the amount of disparity. Higher values decrease disparity.

Parameters:: size – Sensor size, in pixels.

Example:

>>> camera_sensor.set_sensor_size(1920.0)

class TiledCameraSensor( paths: str | list[str] | Camera, *, resolution: tuple[int, int], annotators: Literal['distance_to_camera', 'distance_to_image_plane', 'instance_id_segmentation', 'instance_segmentation', 'motion_vectors', 'normals', 'rgb', 'rgba', 'semantic_segmentation'] | list[Literal['distance_to_camera', 'distance_to_image_plane', 'instance_id_segmentation', 'instance_segmentation', 'motion_vectors', 'normals', 'rgb', 'rgba', 'semantic_segmentation']], render_vars: list[str] | None = None, )#

Bases: object

High level class for creating/wrapping and operating tiled (batched) camera sensors.

Parameters:

paths – Camera object, single path or list of paths to existing or non-existing (one of both) USD Camera prims. Can include regular expressions for matching multiple prims.
resolution – Resolution of each individual sensor (following OpenCV/NumPy convention: (height, width)).
annotators – Annotator/sensor types to configure.

Raises:

ValueError – If no prims are found matching the specified paths.
ValueError – If an unsupported annotator type is specified.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> from isaacsim.sensors.experimental.rtx import TiledCameraSensor
>>>
>>> # given a USD stage with the Camera prims: /World/prim_0, /World/prim_1, and /World/prim_2
>>> resolution = (240, 320)  # following OpenCV/NumPy convention `(height, width)`
>>> tiled_camera_sensor = TiledCameraSensor(
...     "/World/prim_.*",
...     resolution=resolution,
...     annotators=["rgb", "distance_to_image_plane"],
... )  
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)

attach_annotators( annotators: str | list[str], ) → None#

Attach annotators to the sensor.

Parameters:: annotators – Annotator/sensor types to attach.
Raises:: ValueError – If the specified annotator is not supported.

Example:

>>> tiled_camera_sensor.annotators
['distance_to_image_plane', 'rgb']
>>> tiled_camera_sensor.attach_annotators("normals")
>>> tiled_camera_sensor.annotators
['distance_to_image_plane', 'normals', 'rgb']

detach_annotators( annotators: str | list[str], ) → None#

Detach annotators from the sensor.

Parameters:: annotators – Annotator/sensor types to detach. If the annotator is not attached, or it has already been detached, a warning is logged and the method does nothing.
Raises:: ValueError – If the specified annotator is not supported.

Example:

>>> tiled_camera_sensor.annotators
['distance_to_image_plane', 'normals', 'rgb']
>>> tiled_camera_sensor.detach_annotators(["distance_to_image_plane", "normals"])
>>> tiled_camera_sensor.annotators
['rgb']

get_data( annotator: str, *, tiled: bool = False, out: wp.array | None = None, ) → tuple[wp.array | None, dict[str, Any]]#

Fetch the specified annotator/sensor data for all cameras as a batch of frames or as a single tiled frame.

Parameters:

annotator – Annotator/sensor type from which fetch the data.
tiled – Whether to get annotator/sensor data as a single tiled frame.
out – Pre-allocated array to fill with the fetched data.

Returns:

Two-elements tuple. 1) Array containing the fetched data. If out is defined, such instance is returned filled with the data. If no data is available at the moment of calling the method, None is returned. 2) Dictionary containing additional information according to the requested annotator/sensor.

Raises:

ValueError – If the specified annotator is not supported.
ValueError – If the specified annotator is not configured when instantiating the object.

Example:

>>> data, info = tiled_camera_sensor.get_data("rgb")  
>>> data.shape  
(3, 240, 320, 3)
>>> info
{}

property annotators: list[str]#

Annotators.

Returns:: Sorted list of registered annotators.

Example:

>>> tiled_camera_sensor.annotators
['distance_to_image_plane', 'rgb']

property camera: Camera#

Camera object encapsulated by the sensor.

Returns:: Camera object encapsulated by the sensor.

Example:

>>> tiled_camera_sensor.camera
<isaacsim.core.experimental.objects.impl.camera.Camera object at 0x...>

property render_product: pxr.UsdRender.Product#

Render product.

Returns:: Render product of the tiled camera sensor.

Example:

>>> tiled_camera_sensor.render_product
UsdRender.Product(Usd.Prim(</Render/OmniverseKit/HydraTextures/tiled_camera_sensor_...>))

property resolution: tuple[int, int]#

Resolution of individual batched frames.

Returns:: (height, width)).
Return type:: Resolution of individual batched frames (following OpenCV/NumPy convention

Example:

>>> tiled_camera_sensor.resolution
(240, 320)

property tiled_resolution: tuple[int, int]#

Resolution of tiled frames.

Returns:: (height, width)).
Return type:: Resolution of tiled frames (following OpenCV/NumPy convention

Example:

>>> tiled_camera_sensor.tiled_resolution
(480, 640)

Utils#

parse_generic_model_output_data( data: warp.array, ) → GenericModelOutput#

Parse generic model output structure from annotator data.

Parameters:: data – generic-model-output data from an annotator.
Returns:: Generic model output structure.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> import isaacsim.core.experimental.utils.stage as stage_utils
>>> from isaacsim.sensors.experimental.rtx import LidarSensor, parse_generic_model_output_data
>>>
>>> stage_utils.define_prim("/World/sphere", "Sphere") 
>>> sensor = LidarSensor("/World/lidar", annotators=["generic-model-output"]) 
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)
>>>
>>> data, _ = sensor.get_data("generic-model-output") 
>>> parse_generic_model_output_data(data) 
<generic_model_output.GenericModelOutput object at 0x...>

parse_stable_id_map_data(data: warp.array) → dict#

Parse Stable ID Map data from annotator data.

Parameters:: data – stable-id-map annotator data.
Returns:: Dictionary mapping stable IDs to their prim paths.

Warning

Some object IDs returned by the LiDAR may not have an entry in this map. The renderer emits each 128-bit stable ID as a per-instance base ID combined with an “upper index” in the high 32 bits — the submesh index for mesh geometry and the per-triangle primitive index for procedural geometry. The map registers per- instance entries and (when subsetCount > 1) per-GeomSubset entries, but it does not register per-primitive entries, so hits on procedural geometry, on submeshes that weren’t expanded, or on renderer-internal instances without a USD prim path will produce IDs with no map entry, and a direct map[id] lookup will raise KeyError. Use map.get(id, "<unknown>") to handle missing IDs gracefully — see the bundled resolve_lidar_object_ids.py example for the recommended pattern.

Example:

>>> import isaacsim.core.experimental.utils.app as app_utils
>>> import isaacsim.core.experimental.utils.stage as stage_utils
>>> from isaacsim.sensors.experimental.rtx import LidarSensor, parse_stable_id_map_data
>>>
>>> stage_utils.define_prim("/World/sphere", "Sphere") 
>>> sensor = LidarSensor("/World/lidar", annotators=["stable-id-map"]) 
>>>
>>> # play the simulation so the sensor can fetch data
>>> app_utils.play(commit=True)
>>>
>>> data, _ = sensor.get_data("stable-id-map") 
>>> parse_stable_id_map_data(data) 
{0: '/World/sphere'}

Lidar configuration registry#

SUPPORTED_LIDAR_CONFIGS#: Mapping from known Isaac Sim lidar asset paths to optional variant name sets.

SUPPORTED_LIDAR_VARIANT_SET_NAME#: Variant set name expected on supported lidar prims.