Isaac Sim walkthrough builds a robot pick-and-place task from SimReady assets

Isaac Sim walkthrough builds a robot pick-and-place task from SimReady assets

The walkthrough shows how to compose a beverage can pick-and-place task in Isaac Sim from assets in SimReady Central, an open asset library on Hugging Face. The parts are a UR17G arm, a Robotiq 2F-140 gripper and a beverage can. An AI agent with Isaac Sim skills assembles the robot and generates the controls. The task is to grasp the can, lift it, move it to a target and release it.

The post defines its terms first. For robotics, SimReady means an OpenUSD asset has been prepared with the properties a defined simulation use case needs, such as collisions, mass, joints and joint drives. SimReady Foundation sets this out in three levels: a profile (a group of features for a use case, such as a robot body), features (capabilities such as controllable joints) and requirements (checks on the settings behind each feature, such as a joint's drive settings). The asset packages ship with the models and their simulation settings. You add the scene and task code: the arm-gripper assembly, the work surface, starting positions, and the controls that move the arm and open and close the gripper.

The post warns that a robot can look correct while stationary yet move or touch objects wrongly once the simulation runs. Before the task, it suggests three checks, which you can ask the agent to run or do by hand: joint settings (limits and drive settings), movement (a small arm move, and opening and closing the gripper), and contact (the can rests on the work surface and does not fall through it).

The software list is Isaac Sim 6.2, the version used for the example; an agent that can use the Isaac Sim skills; the Hugging Face CLI, only if you pick the command-line download; and the USD Search endpoint. In the example the agent is GPT-6 Sol. The authors say another agent and model combination may work if it can load the required skills and execute Python against Isaac Sim, and that generated code and results can vary.

Step 1 is downloading the three assets from SimReady Central, either with Save to folder or with the supplied Hugging Face command. Each asset folder must stay intact so Isaac Sim finds the supporting files. The validation reports are named: the arm's uses the Robot-Body 4.0.0 profile, the gripper's uses Robot-Gripper 3.0.0, and both record passing PhysX driven-joint checks, which inspect joint-drive settings. The post says this profile validation is a shared baseline for reuse, and the pick-and-place run then tests how arm, gripper, can and controller work together.

Step 2 is assembly. You start Isaac Sim from its install directory with the remote Python server enabled (./isaac-sim.sh --enable isaacsim.code_editor.python_server), and start the agent from the same directory so it can reach the skills. After confirming all three assets load without missing-file errors, you ask the agent to inspect the joint limits and drive settings. The prompt then asks it to assemble the arm and gripper using the Robot Assembly tutorial in Isaac Sim and to validate the assembly. The agent uses the isaac-sim-remote and usd-articulation skills for the assembly and the physics validation, so the arm and gripper can be controlled as a single articulation. Manual assembly with the Robot Assembler tutorial is the alternative.

Step 3 is the run. The prompt asks the agent to run pick and place with the assembled robot and the soda can, to use a contact-based grasp, and to validate the accuracy and placement error. The skills listed are isaac-sim-remote, physics-simulation and an IK manipulation skill. The sequence is approach, grasp, lift, hold, transfer and release. The post's Figure 6 video shows the arm moving the can from a yellow pickup marker to a green target, at 4× speed. If a run differs, you note where the sequence fails, ask the agent to inspect the scene and generate controls, and reset the can and robot to the same start state before retrying.

The post lists its simulation assumptions. The setup is fixed: the can starts at a known position, the robot is anchored, the workspace is clear and flat, no vision was used and collision checks were limited. The can is simplified: deformation and liquid motion are not modeled, so the example does not test crushing. The grasp is scripted: a horizontal grasp uses a 2 N·m drive-effort cap chosen through simulation tests, not calibrated to fingertip forces or recommended by the manufacturer, and contact and friction hold the can while the arm follows fixed waypoints without replanning. The results apply only to this scripted setup and do not establish robustness to uncertain object positions, different payloads or real hardware.

The post closes with pointers: browse SimReady Central for robots and objects, read the SimReady Foundation profiles alongside each asset's validation reports, follow the guide for making your own robotics assets from CAD, and use the submission guide to contribute. It links two community projects as inspiration, Michael Wagner's voice-controlled VR demo and Maziyar Panahi's Microduck simulation, and asks readers to share their scenes, assets and Isaac Sim version in the comments.

Key facts

  • The example composes a UR17G arm, a Robotiq 2F-140 gripper and a beverage can from SimReady Central in Isaac Sim 6.2, then runs a grasp, lift, transfer and release task.
  • An AI agent (GPT-6 Sol in the example) uses the isaac-sim-remote and usd-articulation skills to assemble and validate the robot, and generates the pick-and-place controls.
  • The arm's validation report uses the Robot-Body 4.0.0 profile and the gripper's uses Robot-Gripper 3.0.0; both record passing PhysX driven-joint checks.
  • The grasp is scripted: a 2 N·m drive-effort cap chosen through simulation tests, not calibrated to fingertip forces or recommended by the manufacturer, with fixed waypoints and no replanning.
  • The authors state the results do not establish robustness to uncertain object positions, different payloads or real hardware.

Why it matters

The post shows a concrete path from reusable, pre-validated robot assets to a working simulated manipulation task, with an AI agent doing the assembly and control scripting. Its central point is the split of responsibilities: profile validation gives a shared baseline for reusing an asset, while the pick-and-place run tests how arm, gripper, can and controller behave together in the assembled scene. It is a practical tutorial rather than a new release.

Who it affects

Developers building robot simulations in Isaac Sim who want to reuse assets from SimReady Central instead of preparing their own. Also people who want to try an AI agent with Isaac Sim skills for scene assembly and control generation. The post also points to a guide for creating your own robotics assets from CAD and a submission guide for contributing them.

How to use it

Install Isaac Sim 6.2 and set up an agent that can use the bundled Isaac Sim skills. Download the UR17G, Robotiq 2F-140 and beverage can packages from SimReady Central, keeping each folder intact. Start Isaac Sim with the remote Python server enabled, start the agent from the Isaac Sim directory, and give it the local USD paths. Ask it to assemble and validate the robot, then to run a contact-based pick and place with a placement-error check. Before adapting the task, inspect the generated code and check the behavior. Reset the can and robot to the same start state between retries so runs are comparable. The post gives no price or licence terms.

How solid is it

This is a first-party walkthrough with sample prompts, named validation profiles and a video of the run. The post reports no success rate, placement-error figure or timing. The agent-generated code and results can vary, and the post says so. Only the example's agent and model (GPT-6 Sol) are named; another combination may work if it can load the required skills and run Python against Isaac Sim.

Risks and caveats

The setup is fixed and simplified: a known can position, an anchored robot, a clear flat workspace, no vision and limited collision checks. The can has no deformation or liquid motion, so crushing is not tested. The 2 N·m drive-effort cap is not calibrated to fingertip forces or recommended by the manufacturer, and the arm follows fixed waypoints without replanning. The authors say the results do not establish robustness to uncertain object positions, different payloads or real hardware, and they advise inspecting the generated code before adapting the task.

“These results apply to this scripted setup. They do not establish robustness to uncertain object positions, different payloads, or real hardware.”

— From the walkthrough's simulation assumptions