Camera Calibration

Camera Calibration Tool is a component of the isaacsim.sensors.rtx.placement extension, which generates camera calibration data for deployed cameras in the scene.

Open the Calibration Tool Panel.

• The Calibration Tool UI will automatically be opened on the right side of the screen. It is accessible by Tools > Action and Event Data Generation > Camera Calibration.

Input Fields

Place Info: A string that describes the location of the scene, including city, building, and room. It will be converted and stored in calibration.json. Review the output example for more details.

• Input Format: city=[city name]/building=[building name]/room=[room name]

• Example:

  • Input: city=Santa Clara/building=NVIDIA Voyager/room=Visitor Lobby

  • Output: in calibration.json:

  {
      "place": [
          {
          "name": "city",
          "value": "Santa Clara"
          },
          {
              "name": "building",
              "value": "NVIDIA Voyager"
          },
          {
              "name": "room".
              "value": "Visitor Lobby"
          }
      ],
  }
  

Scene Root Prim Path: The path to the root prim of the scene. This is used to approximate the top view camera’s position. The top view camera will look at the scene root prim’s center.

Floor & Ceiling Height: The floor and ceiling height of the scene.

Note

• The ceiling height is used to adjust the clipping range of the top-view camera, making it easier for users to create accurate top views.

• By default, the ceiling height is set to -1, which means the top view camera’s clipping range will use be its default value.

• By customizing the ceiling height, users can clip out prims or objects above the specified value when creating a top-view camera.

• Alternatively, users can create a top view using the Section Tool extension.

  Create Top View Camera With Section Tool Extension

Top View Camera: This camera will be used to render the top_view images.

• Create: After Scene Root Prim Path is set, clicking this button will automatically generate a topview camera that looks at the scene root’s center. The top view camera will be generated under /World/Top_Camera.

• Path: After clicking Create, the top view camera’s path will be shown here. You can also use this field to select existing top view cameras in the stage.

Note

• The Top View Camera must be vertical to the ground and it must cover the position of all the calibration dots under World/Calibration_Dots and cameras under the World/Cameras.

• The Top View Camera should have a rotation of [0,0,0] with the projection type set to “orthographic”.

Raycast Density: The density of the raycast. The higher this value, the more detailed the FOV contour will be. A density value of N indicates that N * N rays will be cast and they are uniformly distributed for each camera.

• Default value: 100

Minimum FOV Polygon Edge Length (meter): The minimum length of edges in the polygon’s contour. Edges shorter than this length are ignored and the vertices are connected to the next point that meets this criteria. The unit is meter.

• Default value: 0 (no simplification in drawing the contour)

Minimum Area of FOV Polygon Hole to Ignore: When generating data, holes in the FOV polygon that are smaller than this threshold value are ignored. Holes are the areas that are not included in the FOV polygon.

• Default value: 0 (don’t ignore any hole in FOV polygons)

Create Camera View Images: Whether to include camera view images in the output folder.

Create FOV Polygon Images: Whether to render top view image with FOV polygon in debug data folder.

Show FOV Polygon: Whether to show FOV polygons from the currently selected camera.

Output Folder Path: The path to the output folder. Click on the folder icon to select the output folder path.

Buttons and Functions

Note

Before starting to generate the calibration file, the following prerequisites must be met:

• Top View Camera Path field is set up with a valid camera prim path to a valid top view camera.

• The output folder path value must be valid.

• The Place Info must have the correct format and input.

• The cameras must be under /World/Cameras.

• Create Dot Prims: Generate calibration dot prims for each camera. Calibration dots will be randomly generated and they are used to sample the polygons contour that each camera can view.

• Generate Calibration File: Create calibration.json that stores all the camera calibration data.

  Note

  You must run Create Dot Prims before generating the calibration file.

• Generate Top View Image: Generate the Top View image and store the image in the output folder. An imageMetadata.json file will be generated to store the image metadata.

  Note

  • If the Create FOV Polygon Images is checked, the FOV polygon is visualized on the topview layout. The FOV images are generated in a debug data folder within the output folder.

  • You must have the current camera viewport set to your top view camera.

Walkthrough

Here is a walkthrough for using the Camera Calibration tool. We are using the Isaac Sim Full Warehouse for demonstration.

Note

• Stage unit must be in meters.

• A valid NavMesh is required.

Step 1 - Enable the Extension

1. Enable isaacsim.sensors.rtx.placement extension

2. Activate camera calibration tool panel

Step 2 - Create Cameras

Add two cameras under /World/Cameras as sample cameras. Ideally, the cameras should be able to view the walkable area of the scene.

Tip

To add cameras to the stage, follow the Isaac Sim Camera tutorial.

Alternatively, you can use IRA to spawn cameras.

Step 3 - Create Top View Camera [with Section Tool extension]

This step is to generate a top view camera with the help of Section Tool extension (omni.kit.window.section). Section Tool uses a horizontal plane to slice the stage, and it can help remove the ceiling from the view, which may block the camera’s view of the floorplan.

1. Adjust the perspective camera’s view so that you can overview the whole environment. This is for adjusting the section tool height in the next step.

   

2. Search “Section” from Window > Extension UI and enable the omni.kit.window.section extension. The Section Tool UI will be enabled and docked on the right side of the screen. It can be re-enabled from Tools > Section.

3. To create an optimal top view we need to exclude objects like the ceiling. We can do so by manipulate the transform of the Section Tool plane /SectionTools/Section_Tool_Object.

   • You can use the move tool to drag the plane up and down to slice the scene.

   

   • Alternatively, you can also directly set the transform of the plane from the Property window on the bottom right of the screen. Set the Transform for the plane to the same as the below iamge for optimal result.

   

   Note

   You can hide the Section Tool plane by unchecking Options > Display Section Manipulator in its UI.

4. Set the Scene Root Prim Path. The Top View Camera generated by this tool will look at this scene root. In this case, set it to /Root.

5. Click Create and the top view camera will be generated and its path will be shown in the text field.

   Note

   • Because the ceiling of the warehouse is already clipped using the Section Tool extension, there’s no need to set the Floor & Ceiling Height. Leave it at the default value.

   

6. Switch the viewport to the new top view camera to verify that it covers the floorplan.

   

Tip

To switch the viewport to the top-view camera, click the Camera icon, then click Cameras > Calibration_Top_Camera camera.

Step 4 - Set Up the Calibration Tool Attributes

This step is to enter the information needed for camera calibration.

1. Enter the place information in Place Info. In this case, it’s city=Santa Clara/building=Isaac Sim Warehouse/room=Warehouse.

2. Set Raycast Density, Minimum FOV Polygon Edge Length, and Minimum Area of FOV Polygon Hole to Ignore. See the Input Field for more details. In this case, we are using the default values.

3. Check the Create Camera View Images, Create FOV Polygon Images, and Show FOV Polygon boxes if these additional data is neeeded.

4. Set Output Folder Path by either entering the path or clicking the folder picker icon.

Step 5 - Generate Calibration Dots

Generate Calibration Dots for each Camera by clicking on the Create Dot Prims button.

Note

• Calibration dot prims are generated under /World/Calibration_Dots/[Camera Name]/, where [Camera Name] is the name of the camera.

• For each Camera prim under /World/Cameras, six calibration dots are generated. The dot prims are used to calculate the projection matrix for each camera.

• You can switch your viewport to any camera’s view to check whether all the calibration dots are visible.

Step 6 - Generate Calibration File

1. Generate the calibration file by clicking on the Generate Calibration File button. This generates a calibration.json file to Output Folder Path.

2. After the calibration.json file is generated. You can visualize the FOV in the stage by selecting the target camera.

   

Note

Your result may look different because it depends on the camera parameters. In our case, the translate of the camera is (-13.02311, 7.20828, 5.0), the orient is (-55.253, -56.035, -150.088), and the camera focal length is 20.94.

Step 7 - Generate Top View Image

To visualize the generated FOV polygon top view image, generate the image by clicking on Generate Top View Image button.

The Top View Camera’s view will be rendered and output to the [Output Folder Path]. An imageMetadata.json file is also generated to store image metadata.

Note

If Create FOV Polygon Images is checked, for each camera there will be an image with a white-shaded FOV polygon from the top view. The FOV polygon images will be generated under [Output Folder Path]/Debug/fieldOfViewPolygon.