v1.2.0

👀 Overview

We leverage the new release of Isaac Sim, 4.2.0, and bring RTX-based tiled rendering, support for multi-agent environments, and introduce many bug fixes and improvements.

Additionally, we have published an example for generating rewards using an LLM based on Eureka, available here https://github.com/isaac-sim/IsaacLabEureka.

Full Changelog: v1.1.0...v1.2.0

🔆 Highlighted Features

shadowhand_cube_tiled_rendering.mp4

shadowhand_multi_agent_hand_over.mp4

deformable_teddy_pick_up.mp4

✨ New Features

• Adds RTX-based tiled rendering. This improves the overall rendering speed and quality.
• Adds the direct workflow perceptive Shadowhand Cube Repose environment Isaac-Repose-Cube-Shadow-Vision-Direct-v0 by @kellyguo11.
• Adds support for multi-agent environments with the Direct workflow, with support for MAPPO and IPPO in SKRL by @Toni-SM
• Adds the direct workflow multi-agent environments Isaac-Cart-Double-Pendulum-Direct-v0 and Isaac-Shadow-Hand-Over-Direct-v0 by @Toni-SM
• Adds throughput benchmarking scripts for the different learning workflows by @kellyguo11 in #759
• Adds results for the benchmarks in the documentation here for different types of hardware by @kellyguo11
• Adds the direct workflow Allegro hand environment by @kellyguo11 in #709
• Adds video recording to the play scripts in RL workflows by @j3soon in #763
• Adds comparison tables for the supported RL libraries here by @kellyguo11
• Add APIs for deformable asset by @masoudmoghani in #630
• Adds support for MJCF converter by @qqqwan in #957
• Adds a function to define camera configs through intrinsic matrix by @pascal-roth in #617
• Adds configurable modifiers to observation manager by @jtigue-bdai in #830
• Adds the Hydra configuration system for RL training by @Dhoeller19 in #700

🔧 Improvements

• Uses PhysX accelerations for rigid body acceleration data by @Mayankm96 in #760
• Adds documentation on the frames for asset data by @Mayankm96 in #742
• Renames Unitree configs in locomotion tasks to match properly by @Mayankm96 in #714
• Adds option to set the height of the border in the TerrainGenerator by @pascal-roth in #744
• Adds a cli arg to run_all_tests.py for testing a selected extension by @jsmith-bdai in #753
• Decouples rigid object and articulation asset classes by @Mayankm96 in #644
• Adds performance optimizations for domain randomization by @kellyguo11 in #494
• Allows having hybrid dimensional terms inside an observation group by @Mayankm96 in #772
• Adds a flag to preserve joint order inside JointActionCfg action term by @xav-nal in #787
• Adds the ability to resume training from a checkpoint with rl_games by @sizsJEon in #797
• Adds windows configuration to VS code tasks by @johnBuffer in #963
• Adapts A and D button bindings in the keyboard device by @zoctipus in #910
• Uses torch.einsum for quat_rotate and quat_rotate_inverse operations by @dxyy1 in #900
• Expands on articulation test for multiple instances and devices by @jsmith-bdai in #872
• Adds setting of environment seed at initialization by @Mayankm96 in #940
• Disables default viewport when headless but cameras are enabled by @kellyguo11 in #851
• Simplifies the return type for parse_env_cfg method by @Mayankm96 in #965
• Simplifies the if-elses inside the event manager apply method by @Mayankm96 in #948

🐛 Bug Fixes

• Fixes rendering frame delays. Rendered images now faithfully represent the latest state of the physics scene. We added the flag
  rerender_on_reset in the environment configs to toggle an additional render step when a reset happens. When activated, the images/observation always represent the latest state of the environment, but this also reduces performance.
• Fixes wrap_to_pi function in math utilities by @Mayankm96 in #771
• Fixes setting of pose when spawning a mesh by @masoudmoghani in #692
• Fixes caching of the terrain using the terrain generator by @Mayankm96 in #757
• Fixes running train scripts when rsl_rl is not installed by @Dhoeller19 in #784, #789
• Adds flag to recompute inertia when randomizing the mass of a rigid body by @Mayankm96 in #989
• Fixes support for classmethod when defining a configclass by @Mayankm96 in #901
• Fixes Sb3VecEnvWrapper to clear buffer on reset by @EricJin2002 in #974
• Fixes venv and conda pip installation on windows by @kellyguo11 in #970
• Sets native livestream extensions to Isaac Sim 4.1-4.0 defaults by @jtigue-bdai in #954
• Defaults the gym video recorder fps to match episode decimation by @ozhanozen in #894
• Fixes the event manager's apply method by @kellyguo11 in #936
• Updates camera docs with world units and introduces new test for intrinsics by @pascal-roth in #886
• Adds the ability to resume training from a checkpoint with rl_games by @sizsJEon in #797

💔 Breaking Changes

• Simplifies device setting in SimulationCfg and AppLauncher by @Dhoeller19 in #696
• Fixes conflict in teleop-device command line argument in scripts by @Dhoeller19 in #791
• Converts container.sh into Python utilities by @hhansen-bdai in f565c33
• Drops support for TiledCamera for Isaac Sim 4.1

✈️ Migration Guide

Setting the simulation device into the simulation context

Details

Previously, changing the simulation device to CPU required users to set other simulation parameters (such as disabling GPU physics and GPU pipelines). This made setting up the device appear complex. We now simplify the checks for device directly inside the simulation context, so users only need to specify the device through the configuration object.

Before:

sim_utils.SimulationCfg(device="cpu", use_gpu_pipeline=False, dt=0.01, physx=sim_utils.PhysxCfg(use_gpu=False))

Now:

sim_utils.SimulationCfg(device="cpu", dt=0.01, physx=sim_utils.PhysxCfg())

Setting the simulation device from CLI

Details

Previously, users could specify the device through the command line argument --device_id. However, this made it ambiguous when users wanted to set the device to CPU. Thus, instead of the device ID, users need to specify the device explicitly through the argument --device. The valid options for the device name are:

• "cpu": runs simulation on CPU
• "cuda": runs simulation on GPU with device ID at default index
• "cuda:N": runs simulation on GPU with device ID at N. For instance, "cuda:0" will use device at index "0".

Due to the above change, the command line interaction with some of the scripts has changed.

Before:

./isaaclab.sh -p source/standalone/workflows/sb3/train.py --task Isaac-Cartpole-v0 --headless --cpu

Now:

./isaaclab.sh -p source/standalone/workflows/sb3/train.py --task Isaac-Cartpole-v0 --headless --device cpu

Renaming of teleoperation device CLI in standalone scripts

Details

Since --device is now an argument provided by the AppLauncher, it conflicted with the command-line argument used for specifying the teleoperation-device in some of the standalone scripts. Thus, to fix this conflict, the teleoperation-device now needs to be specified through --teleop_device argument.

Before:

./isaaclab.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --device keyboard

Now:

./isaaclab.sh -p source/standalone/environments/teleoperation/teleop_se3_agent.py --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --teleop_device k...

v1.1.0

👀 Overview

With the release of Isaac Sim 4.0 and 4.1, support for Isaac Sim 2023.1.1 has been discontinued. We strongly encourage all users to upgrade to Isaac Sim 4.1 to take advantage of the latest features and improvements. For detailed information on this upgrade, please refer to the release notes available here.

Besides the above, the Isaac Lab release brings new features and improvements, as detailed below. We thank all our contributors for their continued support.

Full Changelog: v1.0.0...v1.1.0

✨ New Features

• Adds distributed multi-GPU learning support for skrl by @Toni-SM in #574
• Updates skrl integration to support training/evaluation using JAX by @Toni-SM in #592
• Adds lidar pattern for raycaster sensor by @pascal-roth in #616
• Adds support for PBS job scheduler-based clusters by @shafeef901 in #605
• Adds APIs for spawning deformable meshes by @Mayankm96 in #613

🔧 Improvements

• Changes documentation color to the green theme by @Mayankm96 in #585
• Fixes sphinx tabs to make them work in dark theme by @Mayankm96 in #584
• Fixes VSCode settings to work with pip installation of Isaac Sim by @Mayankm96 in #628
• Fixes isaaclab scripts to deal with Isaac Sim pip installation by @Mayankm96 in #631
• Optimizes interactive scene for homogeneous cloning by @kellyguo11 in #636
• Improves docker X11 forwarding documentation by @j3soon in #685

🐛 Bug Fixes

• Reads gravity direction from simulation inside RigidObjectData by @Mayankm96 in #582
• Fixes reference count in asset instances due to circular references by @Mayankm96 in #580
• Fixes issue with asset deinitialization due to torch > 2.1 by @Mayankm96 in #640
• Fixes the rendering logic regression in environments by @Dhoeller19 in #614
• Fixes the check for action-space inside Stable-Baselines3 wrapper by @Mayankm96 in #610
• Fixes warning message in Articulation config processing by @locoxsoco in #699
• Fixes action term in the reach environment by @masoudmoghani in #710
• Fixes training UR10 reach with RL_GAMES and SKRL by @sudhirpratapyadav in #678
• Adds event manager call to simple manage-based env by @Mayankm96 in #666

💔 Breaking Changes

• Drops official support for Isaac Sim 2023.1.1
• Removes the use of body view inside the asset classes by @Mayankm96 in #643
• Renames SimulationCfg.substeps to SimulationCfg.render_interval by @Dhoeller19 in #515

✈️ Migration Guide

Renaming of SimulationCfg.substeps

Previously, the users set both omni.isaac.lab.sim.SimulationCfg.dt and omni.isaac.lab.sim.SimulationCfg.substeps, which marked the physics insulation time-step and sub-steps, respectively. It was unclear whether sub-steps meant the number of integration steps inside the physics time-step dt or the number of physics steps inside a rendering step.

Since in the code base, the attribute was used as the latter, it has been renamed to render_interval for clarity.

Removal of Deprecated Attributes

As notified in previous releases, we removed the classes and attributes marked as deprecated. These are as follows:

• The mdp.add_body_mass method in the events. Please use the mdp.randomize_rigid_body_mass instead.
• The classes managers.RandomizationManager and managers.RandomizationTermCfg. Please use the managers.EventManager and managers.EventTermCfg classes instead.
• The following properties in omni.isaac.lab.sensors.FrameTransformerData:
  • target_rot_source --> target_quat_w
  • target_rot_w --> target_quat_source
  • source_rot_w --> source_quat_w
• The attribute body_physx_view from the omni.isaac.lab.assets.Articulation and omni.isaac.lab.assets.RigidObject classes. These caused confusion when used with the articulation view since the body names did not follow the same ordering.

🤗 New Contributors

• @Brayden-Zhang made their first contribution in #600
• @fan-ziqi made their first contribution in #648
• @sudhirpratapyadav made their first contribution in #678
• @locoxsoco made their first contribution in #699
• @masoudmoghani made their first contribution in #710
• @j3soon made their first contribution in #685

v1.0.0

👀 Overview

Welcome to the first official release of Isaac Lab!

Building upon the foundation of the Orbit framework, we have integrated the RL environment designing workflow from OmniIsaacGymEnvs. This allows users to choose a suitable task-design approach for their applications.

While we maintain backward compatibility with Isaac Sim 2023.1.1, we highly recommend using Isaac Lab with Isaac Sim 4.0.0 version for the latest features and improvements.

Full Changelog: v0.3.1...v1.0.0

✨ New Features

• Integrated CI/CD pipeline, which is triggered on pull requests and publishes the results publicly
• Extended support for Windows OS platforms
• Added tiled rendered based Camera sensor implementation. This provides optimized RGB-D rendering throughputs of up to 10k frames per second.
• Added support for multi-GPU and multi-node training for the RL-Games library
• Integrated APIs for environment designing (direct workflow) without relying on managers
• Added implementation of delayed PD actuator model
• Added various new learning environments :
  • Cartpole balancing using images
  • Shadow hand cube reorientation
  • Boston Dynamics Spot locomotion
  • Unitree H1 and G1 locomotion
  • ANYmal-C navigation
  • Quadcopter target reaching

🔧 Improvements

• Reduced start-up time for scripts (inherited from Isaac Sim 4.0 improvements)
• Added lazy buffer implementation for rigid object and articulation data. Instead of updating all the quantities at every step call, the lazy buffers are updated only when the user queries them
• Added SKRL support to more environments

💔 Breaking Change

For users coming from Orbit, this release brings certain breaking changes. Please check the migration guide for more information.

✈️ Migration Guide

Please find detailed migration guides as follows:

• From Orbit to IsaacLab
• From OmniIsaacGymEnvs to IsaacLab

🤗 New Contributors

• @abizovnuralem made their first contribution in #452
• @eltociear made their first contribution in #460
• @zoctipus made their first contribution in #486
• @JunghwanRo made their first contribution in #497

🌟 Acknowledgements

We wholeheartedly thank @Mayankm96, @kellyguo11 and the Boston Dynamics AI Institute for their significant contributions to the framework.

v0.3.1

👀 Overview

This is an intermediate release with minor patch fixes and improvements.

Note: This is the final release of the framework under the name "Orbit". The framework will soon be renamed to "Isaac Lab" and be moved to a new GitHub organization. More details will be provided shortly.

Full Changelog: v0.3.0...v0.3.1

✨ New Features

• Allows setting USD variants when loading prim from USD file by @lorenwel
• Adds fix_root_link attribute to ArticulationRootPropertiesCfg by @Mayankm96

🔧 Improvements

• Removes duplicated cassie configuration in core extension by @MuhongGuo
• Fixes type-hinting for articulation properties in from_files_cfg.py by @MuhongGuo
• Adds X11 setup checks for the Docker container by @hhansen-bdai

🐛 Bug Fixes

• Fixes configclass shared references to keep compound objects independent across subclass instances by @hhansen-bdai
• Fixes loading of ContactSensor when using it in an extension by @fyu-bdai
• Fixes RSL-RL ONNX exporter for empirical normalization by @Nemantor

🤗 New Contributors

• @MuhongGuo made their first contribution in #383
• @lorenwel made their first contribution in #402

v0.3.0

👀 Overview

This release includes various fixes and improvements to the framework. It additionally includes many new features, as listed below.

Full Changelog: v0.2.0...v0.3.0

✨ New Features

• Adds terrain-aware patch sampling into the terrain generator by @nikitardn
• Adds a viewer camera controller to the base environment by @farbod-farshidian
• Adds animation recording for environments by @Mayankm96
• Adds Franka cabinet opening environment from IsaacGymEnvs by @renezurbruegg
• Adds Allegro hand cube manipulation environment from IsaacGymEnvs by @Mayankm96, @arbhardwaj98
• Adds Allegro and Shadow Hand asset configurations by @Mayankm96
• Adds Kinova (Jaco2, Gen3) asset configurations by @Mayankm96
• Adds approximate torque calculation for implicit actuator by @nikitardn
• Adds loading of custom kit experience files using AppLauncher by @Mayankm96

🔧 Improvements

• Adds a runner script to execute all tests in the source directory by @jsmith-bdai
• Allows loading of initialized configs in load_cfg_from_registry by @nikitardn
• Checks default joint states are configured within limits by @Dhoeller19
• Expands on unit tests for rigid object asset and terrain importer classes by @jsmith-bdai
• Adds unit tests for contact sensor class by @fyu-bdai
• Adds signal interrupt handle to AppLauncher by @Mayankm96
• Adds ROS2 Humble to Dockerfile by @hhansen-bdai
• Improved usage instructions for cluster deployment by @pascal-roth
• Removes unnecessary future imports for Python 3.10 by @Mayankm96

🐛 Bug Fixes

• Fixes running environments with a single instance by @Dhoeller19
• Fixes source frame indexing in FrameTransfomer sensor by @jsmith-bdai
• Fixes handling of time-out signal in RSL-RL and RL-Games wrapper by @Mayankm96
• Fixes unwanted squeeze in Articulation class for 1-joint assets by @Mayankm96
• Fixes shape argument ordering in hf_terrains.random_uniform_terrain by @nikitardn
• Fixes joint and body sub-indexing for observations and rewards by @Dhoeller19
• Fixes camera sensor for Isaac Sim 2023.1 update by @hhansen-bdai
• Fixes imitation learning workflow for lift environment by @jsmith-bdai
• Fixes apply actions method in the NonHolonomicAction action term class by @KyleM73
• Fixes the tensor shape for the contact sensor's force matrix data by @abmoRobotics
• Fixes missing max raycast distance in RayCaster sensor by @renezurbruegg
• Fixes rendering of RTX sensors within the environment stepping by @Dhoeller19

💔 Breaking Changes

• Drops support for Isaac Sim 2022.2.2 and earlier by @hhansen-bdai
• Removes compat submodule from orbit by @Mayankm96
• Removes omni.isaac.contrib_tasks in favor of separate project template by @nburger-bdai
• Changes link names ordering in articulation to follow PhysX by @Mayankm96
• Renames RandomizationManager to EventManager by @pascal-roth
• Renames some terms in omni.isaac.orbit.envs.mdp to avoid confusion by @Mayankm96

✈️ Migration Guide

Renaming of Randomization Manager to Event Manager

While the randomization manager referred to all possible " randomizations " in the environment, many users felt that the triggering
of the terms was not apparent. Additionally, for non-RL use-cases, randomization seemed like a misnomer. Hence, we renamed the class to be called event manager.

Replace the following import and usage:

from omni.isaac.orbit.managers import RandomizationTermCfg as RandTerm

@configclass
class RandomizationCfg:
    """Configuration for randomization."""

    reset_base = RandTerm(
        func=mdp.reset_root_state_uniform,
        mode="reset",
        params={"pose_range": {}, "velocity_range": {}},
    )

@configclass
class MyEnvCfg:

     randomization: RandomizationCfg = RandomizationCfg()

with the following:

from omni.isaac.orbit.managers import EventTermCfg as EventTerm

@configclass
class EventCfg:
    """Configuration for events."""

    reset_base = EventTerm(
        func=mdp.reset_root_state_uniform,
        mode="reset",
        params={"pose_range": {}, "velocity_range": {}},
    )

@configclass
class MyEnvCfg:

     events: EventCfg = EventCfg()

Renaming of MDP terms

We observed that several of the MDP terms had close to overlapping names, such as the reward term joint_pos_limits and termination term joint_pos_limit. To avoid errors, we decided to make the term names clearer.

Please note the following changes in MDP terms:

• Observation: joint_pos_norm -> joint_pos_limit_normalized
• Event: add_body_mass -> randomize_rigid_body_mass
• Termination: base_height -> root_height_below_minimum
• Termination: joint_pos_limit -> joint_pos_out_of_limit
• Termination: joint_pos_manual_limit -> joint_pos_out_of_manual_limit
• Termination: joint_vel_limit -> joint_vel_out_of_limit
• Termination: joint_vel_manual_limit -> joint_vel_out_of_manual_limit
• Termination: joint_torque_limit -> joint_effort_out_of_limit

Removal of omni.isaac.contrib_tasks

Initially, we intended to have the contrib_tasksmodule for developers to send their environment implementations. However, many users preferred a more isolated development environment for their projects. Hence, we removed the contrib_tasks and now provide a separate template repository for building projects that use Orbit.

However, we encourage the community to provide their environment implementations directly to the orbit_tasks extension. This allows us to ensure the environments are maintained during future releases of the framework.

For more details, please check: https://github.com/isaac-orbit/orbit.ext_template

🤗 New Contributors

• @fyu-bdai made their first contribution in 06ada15
• @arbhardwaj98 made their first contribution in 1a42eb9
• @phenomen21 made their first contribution in #248
• @Ltesfaye made their first contribution in #320
• @KyleM73 made their first contribution in #317
• @VladimirFokow made their first contribution in #321
• @shafeef901 made their first contribution in #300
• @RumblingTurtle made their first contribution in #341

Release 0.2.0

👀 Overview

This release refactors Orbit APIs to make them more modular and performance-efficient. It also introduces various new functionalities, such as batched sensors and managers handling randomization, termination, action spaces, and curriculum.

Full Changelog: v0.1.0...v0.2.0

✨ New Features

• Procedural terrain generation using height-fields and trimesh library
• Sensors for contact sensing, frame transformations, and GPU-based ray-casting
• Scene manager to handle a collection of assets and sensors
• Managers to handle various aspects of environment designing - action space, observation space, randomization, termination, rewards, and curriculum
• Spawning support to load meshes in different formats (OBJ, FBX, STL) and URDF files into simulation directly
• Support for docker and cluster-based deployments
• New environments for flat and rough terrain locomotion from legged_gym

🔧 Improvements

• Merged the previous type-specific static markers into a unified VisualizationMarkers class
• Switched to using PhysX interfaces directly instead of Isaac Sim interface classes for performance
• Migrated the gym.Env class from the OpenAI Gym 0.21 version to the Gymnasium v0.29 version

💔 Breaking Changes

This release is incompatible with v0.1.0 due to heavy refactoring and changes. We recommend users revisit the tutorials to familiarize themselves with the new APIs. We apologize for any inconvenience caused but believe the enhancements in this release justify the transition.

🤗 New Contributors

• @kouroshD
• @Toni-SM
• @pascal-roth
• @nikitardn
• @Dhoeller19
• @jsmith-bdai
• @AutonomousHansen
• @farbod-farshidian

Release 0.1.0

👀 Overview

The initial release of the ORBIT framework. It includes the core APIs built around Isaac Sim for robot learning. In addition, provides an initial set of environments for learning.

The main set of features includes:

• Python configuration handling system using the decorator @configclass
• Different explicit actuator models for processing commands
• Actuator groups to handle complex articulations
• Support for different robots:
  • legged robots: ANYmal-C, ANYmal-B, Unitree A1
  • fixed-arm manipulators: Franka Emika arm, UR5
  • wheel-based mobile manipulators: Clearpath Ridgeback with Franka Emika arm or UR5
  • legged mobile manipulators
• Batched implementation of motion generators
  • differential inverse kinematics
• Initial set of environments for learning
  • classic: cart-pole, humanoid, ant
  • locomotion: legged locomotion on flat terrain
  • manipulation: table-top object lifting, end-effector tracking
• Demonstrations of different learning workflows
  • Imitation learning: robomimic
  • Reinforcement learning: stable-baselines3, rsl-rl, rl-games