Isaac Sim Benchmarks#

Note

A full set of KPIs across different hardware configurations will be published in the official 5.0 release. Please check back for updates.

This page contains key performance indicators (KPIs) for Isaac Sim, captured across different reference hardware and measured using the isaacsim.benchmark.services extension. It also contains a guide on how to collect the same KPIs on your hardware, to compare to our published performance specs.

GPU-Independent KPIs#

These KPIs measure Isaac Sim performance independent of the GPU on which Isaac Sim is running.

Note

Official 5.0 release will contain updated numbers for below KPIs.

GPU-Independent KPIs#

Name

Definition

Units

Value

Binary package size (Windows)

Size of Windows binary package

GB

Binary package size (Ubuntu)

Size of Ubuntu binary package

GB

Docker container size

Size of Docker container before extraction on NGC

GB

pip package size

Size of pip package as downloaded

GB

Startup time (async)

Time from launching Isaac Sim executable to app ready appearing in logs

seconds

Startup time (non-async)

Time from initializing SimulationApp in standalone Python to app ready appearing in logs

seconds

GPU-Dependent KPIs#

These KPIs measure Isaac Sim performance on reference hardware, including frame rate for benchmark scenes and render rate for specific sensor combinations. KPIs are reported as the average KPI value across 600 frames.

The KPIs are generated from the scripts in the below section.

Note

A KPI table with different hardware configurations will be published here in the official 5.0 release.

Measuring KPIs on Local Hardware#

Isaac Sim KPIs can be measured using the Python scripts provided in standalone_examples/benchmarks. The KPIs provided above are computed using the scripts as follows. Commands are providing in bash syntax (for Ubuntu); for Windows, replace .sh with .bat and \ for multiline commands to `.

  • Startup time (async): Measured as Runtime for phase: startup from the logs after running

    ./isaac-sim.sh --no-window --/app/quitAfter=200 --/app/file/ignoreUnsavedOnExit=1 \
  --enable isaacsim.benchmark.service

  • Startup time (non-async): Measured as Runtime for phase: startup from the logs after running

    ./python.sh standalone_examples/api/isaacsim.simulation_app/hello_world.py \
  --enable isaacsim.benchmark.services

  • Full Warehouse Sample Scene load time & FPS: Load time is measured as Runtime for phase: loading and FPS is measured as Mean FPS for phase: benchmark.

    ./python.sh standalone_examples/benchmarks/benchmark_scene_loading.py \
  --env-url /Isaac/Environments/Simple_Warehouse/full_warehouse.usd

  • Physics Steps per second: Physics steps per second is measured as 1000.0/Mean Physics Frametime for phase: benchmark.

    ./python.sh standalone_examples/benchmarks/benchmark_robots_o3dyn.py \
  --num-robots 2

  • Isaac ROS Sample Scene FPS: FPS is measured as Mean FPS for phase: benchmark.

    ./python.sh standalone_examples/benchmarks/benchmark_scene_loading.py \
  --env-url /Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd

  • ROS 2 Render & Publishing Speed: First, measure FPS as Mean FPS for phase: benchmark. Then, compute this metric as [FPS] * 8 * 1920 * 1200 / 1e6, accounting for rendering 4 Hawk stereo camera pairs.

    ./python.sh standalone_examples/benchmarks/benchmark_robots_nova_carter_ros2.py \
  --num-robots 1 --enable-3d-lidar 1 --enable-2d-lidar 2 --enable-hawks 4 --viewport-updates

  • SDG images per second (simple): First, measure FPS as Mean FPS for phase: benchmark. Then compute this metrics as [FPS] * 2 * 1280 * 720 / 1e6, accounting for rendering 2 720p cameras.

    ./python.sh standalone_examples/benchmarks/benchmark_sdg.py \
  --num-cameras 2 \
  --resolution 1280 720 \
  --asset-count 100 \
  --annotators rgb distance_to_image_plane  \
  --skip-write

  • SDG images per second (complex): First, measure FPS as Mean FPS for phase: benchmark. Then compute this metrics as [FPS] * 2 * 1280 * 720 / 1e6, accounting for rendering 2 720p cameras.

    ./python.sh standalone_examples/benchmarks/benchmark_sdg.py \
  --num-cameras 2 \
  --resolution 1280 720 \
  --asset-count 100 \
  --annotators all  \
  --skip-write