Surface Gripper Extension

About

The Surface Gripper Extension Extension is used to create a suction-cup gripper behavior for an end-effector. It works by creating a Joint between two bodies when the parent body is close enough to the child at the gripper point of contact.

This extension is enabled by default. If it is ever disabled, it can be re-enabled from the Extension Manager by searching for isaacsim.robot.surface_gripper.

API Documentation

See the API Documentation for usage information.

Tutorials & Examples

Activate the Robotics Examples content browser from Windows > Examples > Robotics Examples. Navigate to Manipulation, select the Surface Gripper Example, and click the load button in the information window on the right side of the Robotics Examples content browser. You may need to adjust the GUI to see the load button.

Surface Gripper Example

To run the Example:

1. Press the Load button. The scene should begin playing.

2. Click the Gripper State button to CLOSE the gripper, attaching the cone to the cube. The cone will turn RED on successful attachment.

3. Adjust the Gripper Force and Gripper Speed sliders to change the force intensity at the Cone and press the respective APPLY`. Excessive forces will break the gripper attachment, and the cone will go back to GREEN when the surface gripper is open.

Surface Gripper Properties

Property	Description
Parent Path	The rigid body that will contain the gripper
Offset	local Offset transform from the origin of the parent. The Gripper will attempt to close on the X direction of the offset pose.
Grip Threshold	Distance from the gripper point that will accept closing contact
Force Limit	Force that the gripper can hold before it will break the constraint
Torque Limit	Torque that the gripper can withstand before it will break the constraint
Bend Angle	Angle that the surface gripper can bend when a load is applied
Stiffness	Bend Stiffness
Damping	Bend Damping
Retry Close	When flag is on, the gripper will attempt to close until it is sucessful in grabbing an object, otherwise it will attempt once and stop.

Omniverse OmniGraph Node

The Surface Gripper extension provides a UI implementation through Omniverse OmniGraph. To use it, select the rigid body prim to attach the surface gripper, and click on Tools > Robotics > Omnigraph Controllers > Surface Gripper. This will create the surface gripper action graph.

1. Use the SurfaceGripperOffset prim to define the gripper placement

2. Change the Surface Gripper properties in the SurfaceGripperNode on the properties panel.

Note

The body selected must have the Rigid Body API applied to it, otherwise it will result on an error and the gripper won’t be created on simulation.

Tutorial: Creating a Surface Gripper Using OmniGraph

Let’s begin by Creating a new, empty stage, and add a ground plane to it with on File > New, then Create > Physics > Ground Plane.

Next, we will create two cubes to serve as our rigid bodies: Create > Mesh > Cube. Move one of the cubes above the other, Select both, and on the Properties pane click the + Button, and add Physics > Rigid Body with Colliders Preset.

Now, select the box on top, and go to Tools > Robotics > Omnigraph Controllers > Surface Gripper. This will generate an Omniverse OmniGraph that you can use to control the surface Gripper. Inside the Omniverse OmniGraph, you will find three Prims: impulse_monitor, SurfaceGripperNode, and SurfaceGripperOffset.

The impulse_monitor is a trigger event that gets called whenever there’s an update to the surface gripper condition (e.g the grip is broken). SurfaceGripperNode is the Node that is used to configure the gripper, using the parameters above. Finally, SurfaceGripperOffset is the placement of the surface gripper on the rigid body it is attached to. The gripper will always attempt to close in the offset’s X direction. In our example we want it to be facing down, so we rotate the offset by 90 degrees on the Y axis.

Note

To view the appropriate offset direction, change the gizmo display to show local coordinates by either clicking in the coordinates button , or pressing W. In local Coordinates mode, it will have an orange core

Now, the gripper needs to be placed outside of the body’s Collision box, otherwise it won’t be able to detect the connecting bodies, and instead it will attempt to connect to itself. If that happens, a warning message will appear, showing by how much you must offset the current position so you clear out of the collision. While it will still work if you don’t do so, it will incur in additional compute time to find the collision border of the current object and move the grip point out of it at every closing attempt.

To control the gripper, you need to connect an upstream node to the Open and Close Triggers. The simplest way would be to connect an Impulse event - the same type of node that gets automatically added to the onStep. Open the SurfaceGripperActionGraph that was added inside the cube by selecting it, then right click and “Open Graph”. To add the node, search on the left bar for Impulse, or select the impulse from On Step and duplicate it with Ctrl-D, and connect it to the Close, and a second one to the Open triggers.

To actuate With the simulation running, when you select the Impulse event, on the properties panel, select the State EnableImpulse and check it. It will blink and return to unselected. At this point the surface gripper will be closed on the other cube. If you have not moved the gripper position to the surface of the cube, you will receive the following message:

[Warning] [isaacsim.robot.surface_gripper.python] Surface Gripper is inside the parent Rigid body collider. please move it forward in the X offset direction by 0.500997 to avoid wasted computation. Move the offset by the given distance and it will be on the surface of the object.

Note

Due to a known issue, when reusing the UI to Enable the Impulse, you’ll have to click on it three times to check, uncheck and check again without leaving the selection. This is not necessary if interacting with the node in code.

To interact with the Omnigraph through code, first get the prim at the node path, then set the enable impulse to 1, and call request_compute.

from pxr import Usd, UsdGeom
import omni.usd
import omni.graph as og
stage = omni.usd.get_context().get_stage()


ogn = og.core.get_node_by_path("/World/Cube_01/SurfaceGripperActionGraph/impulse_monitor_01") #Get the Impulse node connected to Close

attr = ogn.get_attribute("state:enableImpulse") #Get the Attribute to enable the impulse
attr.set(1) #Set it to "True"
ogn.request_compute() # Trigger the node execution that will send an impulse to the surface Gripper.

Creating a Surface Gripper fully on code

This section describes how to implement a surface gripper completely from code. In this mode, no USD attribute is created to represent the surface gripper, and it exists only in the physics simulation. The code below was extracted from the Surface Gripper example code.

Surface Gripper Properties

Surface Gripper properties are defined within the _create_scenario function. It is during this step that you define what will be the surface gripper parent rigid body, and what is the offset to the parent’s origin. The offset is in the parents frame of reference. Once the properties are defined, you create a Surface Gripper object, and initialize it with the defined properties. All control for the gripper then is done through the gripper object. Let’s take a closer look at the different parameters of the Gripper Properties:

async def _create_scenario(self, task):
    # [...]

    # Create the Gripper Properties
    self.sgp = Surface_Gripper_Properties()
    self.sgp.d6JointPath = ""
    self.sgp.parentPath = "/GripperCone"                # Set the Cone as the parent object
    self.sgp.offset = dc.Transform()                    # Offset the transform to the base of the Cone
    self.sgp.offset.p.x = 0
    self.sgp.offset.p.z = -0.1001                       # Offset the transform to the base of the Cone
    self.sgp.offset.r = [0.7171, 0, 0.7171, 0]          # Rotate to point gripper in Z direction
    self.sgp.gripThreshold = 0.02                       # Set attachment to within 2cm of the base of the Cone
    self.sgp.forceLimit = 1.0e2                         # Set other limits of the gripper ...
    self.sgp.torqueLimit = 1.0e3
    self.sgp.bendAngle = np.pi / 4
    self.sgp.stiffness = 1.0e4
    self.sgp.damping = 1.0e3
    self.sgp.retryClose = False                         # If set to True, surface gripper will keep trying to close until it picks up an object




    self.surface_gripper = Surface_Gripper(self._dc)    # Create the Gripper
    self.surface_gripper.initialize(self.sgp)           # Initialize the Gripper with these Properties

    # [...]

Update Gripper State

The Surface Gripper is updated on every simulation step in the _on_simulation_step function. This is mandatory to capture the event of the surface gripper breaking contact, so you can trigger the appropriate response on your code.

In the example we use it to update the gripper status, and then change the cone color to RED if it’s closed or GREEN if it’s open.

def _on_simulation_step(self, step):

    # [...]

    self.surface_gripper.update()                       # On every sim step, update the gripper status
    if self.surface_gripper.is_closed():                # Assign color to Cone based on current state
        self.coneGeom.GetDisplayColorAttr().Set([self.color_closed])
    else:
        self.coneGeom.GetDisplayColorAttr().Set([self.color_open])

    # [...]

Controlling the Gripper

The Gripper State is controlled by the on_toggle_gripper_button_clicked function connected to the extension’s UI. It toggles the Gripper State between OPEN and CLOSE.

def _on_toggle_gripper_button_clicked(self, button):    # When receiving a button press
    if self._editor.is_playing():
        if self.surface_gripper.is_closed():            # Toggle Open or Close
            self.surface_gripper.open()
        else:
            self.surface_gripper.close()