-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Adds dexterous lift and reorientation manipulation environments #3378
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a1bf1d9
to
0b5a9b9
Compare
Mayankm96
reviewed
Sep 8, 2025
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/config/__init__.py
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/__init__.py
Outdated
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/curriculums.py
Outdated
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/curriculums.py
Outdated
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/observations.py
Outdated
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/observations.py
Outdated
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
...e/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/pose_commands_cfg.py
Outdated
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
...e/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/pose_commands_cfg.py
Outdated
Show resolved
Hide resolved
Mayankm96
reviewed
Sep 8, 2025
...e/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/pose_commands_cfg.py
Outdated
Show resolved
Hide resolved
b65190b
to
9d1d6af
Compare
kellyguo11
reviewed
Sep 8, 2025
...sks/manager_based/manipulation/dexsuite/config/kuka_allegro/dexsuite_kuka_allegro_env_cfg.py
Outdated
Show resolved
Hide resolved
...sks/manager_based/manipulation/dexsuite/config/kuka_allegro/dexsuite_kuka_allegro_env_cfg.py
Outdated
Show resolved
Hide resolved
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/mdp/utils.py
Outdated
Show resolved
Hide resolved
096cf34
to
ae5b346
Compare
d10a9ee
to
db99d22
Compare
6 tasks
db99d22
to
c5bcba4
Compare
6aead8b
to
7d00448
Compare
kellyguo11
pushed a commit
that referenced
this pull request
Sep 9, 2025
# Description This PR provides remake and extension to orginal environment kuka-allegro-reorientation implemented in paper: DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training (https://arxiv.org/abs/2305.12127) [Aleksei Petrenko](https://arxiv.org/search/cs?searchtype=author&query=Petrenko,+A), [Arthur Allshire](https://arxiv.org/search/cs?searchtype=author&query=Allshire,+A), [Gavriel State](https://arxiv.org/search/cs?searchtype=author&query=State,+G), [Ankur Handa](https://arxiv.org/search/cs?searchtype=author&query=Handa,+A), [Viktor Makoviychuk](https://arxiv.org/search/cs?searchtype=author&query=Makoviychuk,+V) and another environment kuka-allegro-lift implemented in paper: Visuomotor Policies to Grasp Anything with Dexterous Hands (https://dextrah-rgb.github.io/) [Ritvik Singh](https://www.ritvik-singh.com/), [Arthur Allshire](https://allshire.org/), [Ankur Handa](https://ankurhanda.github.io/), [Nathan Ratliff](https://www.nathanratliff.com/), [Karl Van Wyk](https://scholar.google.com/citations?user=TCYAoF8AAAAJ&hl=en) Though this is a remake, this remake ends up differs quite a lot in environment details for reasons like: 1. Simplify reward structure, 2. Unify environment implemtation, 3. Standarize mdp, 4. Utilizes manager-based API That in my opinion, makes environment study and extension more accessible, and analyzable. For example you can train lift policy first then continuing the checkpoint in reorientation environment, since they share the observation space. : )) It is a best to consider this a very careful re-interpretation rather than exact execution to migrate them to IsaacLab Here is the training curve if you just train with `./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Dexsuite-Kuka-Allegro-Lift-v0 --num_envs 8192 --headless` `./isaaclab.sh -p -m torch.distributed.run --nnodes=1 --nproc_per_node=4 scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Dexsuite-Kuka-Allegro-Reorient-v0 --num_envs 40960 --headless --distributed` lift training ~ 4 hours reorientation training ~ 2 days Note that it requires a order of magnitude more data and time for reorientation to converge compare to lift under almost identical setup training curve(screen captured from Wandb) - reward, Cyan: reorient, Purple: Lift <img width="1487" height="780" alt="Screenshot from 2025-09-07 22-58-13" src="https://github.com/user-attachments/assets/bfa911de-4fee-4c0d-b39c-e9c33fae28f4" /> video results lift   reorient   Memo: I really enjoy working on this remake, and hopefully for whoever plan to play and extend on this remake find it helpful and similarily joyful as I did. I will be very excited to see what you got : )) Octi CAUTION: Do Not Merge until the asset is uploaded to S3 bucket! Fixes # (issue) <!-- As you go through the list, delete the ones that are not applicable. --> - New feature (non-breaking change which adds functionality) ## Screenshots Please attach before and after screenshots of the change if applicable. <!-- Example: | Before | After | | ------ | ----- | | _gif/png before_ | _gif/png after_ | To upload images to a PR -- simply drag and drop an image while in edit mode and it should upload the image directly. You can then paste that source into the above before/after sections. --> ## Checklist - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there <!-- As you go through the checklist above, you can mark something as done by putting an x character in it For example, - [x] I have done this task - [ ] I have not done this task -->
Just want to comment and say, awesome work Octi!! |
ooctipus
added a commit
to ooctipus/IsaacLab
that referenced
this pull request
Sep 20, 2025
…c-sim#3378) # Description This PR provides remake and extension to orginal environment kuka-allegro-reorientation implemented in paper: DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training (https://arxiv.org/abs/2305.12127) [Aleksei Petrenko](https://arxiv.org/search/cs?searchtype=author&query=Petrenko,+A), [Arthur Allshire](https://arxiv.org/search/cs?searchtype=author&query=Allshire,+A), [Gavriel State](https://arxiv.org/search/cs?searchtype=author&query=State,+G), [Ankur Handa](https://arxiv.org/search/cs?searchtype=author&query=Handa,+A), [Viktor Makoviychuk](https://arxiv.org/search/cs?searchtype=author&query=Makoviychuk,+V) and another environment kuka-allegro-lift implemented in paper: Visuomotor Policies to Grasp Anything with Dexterous Hands (https://dextrah-rgb.github.io/) [Ritvik Singh](https://www.ritvik-singh.com/), [Arthur Allshire](https://allshire.org/), [Ankur Handa](https://ankurhanda.github.io/), [Nathan Ratliff](https://www.nathanratliff.com/), [Karl Van Wyk](https://scholar.google.com/citations?user=TCYAoF8AAAAJ&hl=en) Though this is a remake, this remake ends up differs quite a lot in environment details for reasons like: 1. Simplify reward structure, 2. Unify environment implemtation, 3. Standarize mdp, 4. Utilizes manager-based API That in my opinion, makes environment study and extension more accessible, and analyzable. For example you can train lift policy first then continuing the checkpoint in reorientation environment, since they share the observation space. : )) It is a best to consider this a very careful re-interpretation rather than exact execution to migrate them to IsaacLab Here is the training curve if you just train with `./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Dexsuite-Kuka-Allegro-Lift-v0 --num_envs 8192 --headless` `./isaaclab.sh -p -m torch.distributed.run --nnodes=1 --nproc_per_node=4 scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Dexsuite-Kuka-Allegro-Reorient-v0 --num_envs 40960 --headless --distributed` lift training ~ 4 hours reorientation training ~ 2 days Note that it requires a order of magnitude more data and time for reorientation to converge compare to lift under almost identical setup training curve(screen captured from Wandb) - reward, Cyan: reorient, Purple: Lift <img width="1487" height="780" alt="Screenshot from 2025-09-07 22-58-13" src="https://github.com/user-attachments/assets/bfa911de-4fee-4c0d-b39c-e9c33fae28f4" /> video results lift   reorient   Memo: I really enjoy working on this remake, and hopefully for whoever plan to play and extend on this remake find it helpful and similarily joyful as I did. I will be very excited to see what you got : )) Octi CAUTION: Do Not Merge until the asset is uploaded to S3 bucket! Fixes # (issue) <!-- As you go through the list, delete the ones that are not applicable. --> - New feature (non-breaking change which adds functionality) ## Screenshots Please attach before and after screenshots of the change if applicable. <!-- Example: | Before | After | | ------ | ----- | | _gif/png before_ | _gif/png after_ | To upload images to a PR -- simply drag and drop an image while in edit mode and it should upload the image directly. You can then paste that source into the above before/after sections. --> ## Checklist - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there <!-- As you go through the checklist above, you can mark something as done by putting an x character in it For example, - [x] I have done this task - [ ] I have not done this task -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR provides remake and extension to orginal environment kuka-allegro-reorientation implemented in paper:
DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training
(https://arxiv.org/abs/2305.12127)
Aleksei Petrenko, Arthur Allshire, Gavriel State, Ankur Handa, Viktor Makoviychuk
and another environment kuka-allegro-lift implemented in paper:
Visuomotor Policies to Grasp Anything with Dexterous Hands
(https://dextrah-rgb.github.io/)
Ritvik Singh, Arthur Allshire, Ankur Handa, Nathan Ratliff, Karl Van Wyk
Though this is a remake, this remake ends up differs quite a lot in environment details for reasons like:
That in my opinion, makes environment study and extension more accessible, and analyzable. For example you can train lift policy first then continuing the checkpoint in reorientation environment, since they share the observation space. : ))
It is a best to consider this a very careful re-interpretation rather than exact execution to migrate them to IsaacLab
Here is the training curve if you just train with
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Dexsuite-Kuka-Allegro-Lift-v0 --num_envs 8192 --headless
./isaaclab.sh -p -m torch.distributed.run --nnodes=1 --nproc_per_node=4 scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Dexsuite-Kuka-Allegro-Reorient-v0 --num_envs 40960 --headless --distributed
lift training ~ 4 hours
reorientation training ~ 2 days
Note that it requires a order of magnitude more data and time for reorientation to converge compare to lift under almost identical setup
training curve(screen captured from Wandb) - reward,

Cyan: reorient, Purple: Lift
video results


lift
reorient


Memo:
I really enjoy working on this remake, and hopefully for whoever plan to play and extend on this remake find it helpful and similarily joyful as I did. I will be very excited to see what you got : ))
Octi
CAUTION:
Do Not Merge until the asset is uploaded to S3 bucket!
Fixes # (issue)
Screenshots
Please attach before and after screenshots of the change if applicable.
Checklist
pre-commit
checks with./isaaclab.sh --format
config/extension.toml
fileCONTRIBUTORS.md
or my name already exists there