Skip to content

Commit 87b8106

Browse files
authored
Updates documentation for manus teleop (#3605)
# Description Since the ManusSDK is not redistributed in Isaac Lab, we have implemented a input device plugin Isaac Sim extension which allows users to download and build the plugin against the ManusSDK, and use it in Isaac Lab teleportation. This change aims to provide documentation of the procedure. ## Type of change - Documentation ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there
1 parent 8302227 commit 87b8106

File tree

1 file changed

+44
-5
lines changed

1 file changed

+44
-5
lines changed

docs/source/how-to/cloudxr_teleoperation.rst

Lines changed: 44 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -416,6 +416,28 @@ Requires Isaac Sim 5.1 or later.
416416

417417
Run the teleoperation example with Manus + Vive tracking:
418418

419+
.. dropdown:: Installation instructions
420+
:open:
421+
422+
Vive tracker integration is provided through the libsurvive library. Install the required udev rules by copying
423+
`81-vive.rules <https://github.com/collabora/libsurvive/blob/32cf62c52744fdc32003ef8169e8b81f6f31526b/useful_files/81-vive.rules>`_
424+
to ``/etc/udev/rules.d/`` and restarting the udev service.
425+
426+
.. code-block:: bash
427+
428+
sudo udevadm control --reload-rules && sudo udevadm trigger
429+
430+
The Manus integration is provided through the Isaac Sim teleoperation input plugin framework.
431+
Install the plugin by following the build and installation steps in `isaac-teleop-device-plugins <https://github.com/isaac-sim/isaac-teleop-device-plugins>`_.
432+
433+
In the same terminal from which you will launch Isaac Lab, set:
434+
435+
.. code-block:: bash
436+
437+
export ISAACSIM_HANDTRACKER_LIB=<path to isaac-teleop-device-plugins>/build-manus-default/lib/libIsaacSimManusHandTracking.so
438+
439+
Once the plugin is installed, run the teleoperation example:
440+
419441
.. code-block:: bash
420442
421443
./isaaclab.sh -p scripts/environments/teleoperation/teleop_se3_agent.py \
@@ -424,20 +446,37 @@ Run the teleoperation example with Manus + Vive tracking:
424446
--xr \
425447
--enable_pinocchio
426448
427-
Begin the session with your palms facing up.
428-
This is necessary for calibrating Vive tracker poses using Apple Vision Pro wrist poses from a few initial frames,
429-
as the Vive trackers attached to the back of the hands occlude the optical hand tracking.
449+
The recommended workflow, is to start Isaac Lab, click **Start AR**, and then put on the Manus gloves, vive trackers, and
450+
headset. Once you are ready to begin the session, use voice commands to launch the Isaac XR teleop sample client and
451+
connect to Isaac Lab.
452+
453+
Isaac Lab automatically calibrates the Vive trackers using wrist pose data from the Apple Vision Pro during the initial
454+
frames of the session. If calibration fails, for example, if the red dots do not accurately follow the teleoperator's
455+
hands, restart Isaac Lab and begin with your hands in a palm-up position to improve calibration reliability.
430456

431457
For optimal performance, position the lighthouse above the hands, tilted slightly downward.
432-
One lighthouse is sufficient if both hands are visible.
433458
Ensure the lighthouse remains stable; a stand is recommended to prevent wobbling.
434459

460+
Ensure that while the task is being teleoperated, the hands remain stable and visible to the lighthouse at all times.
461+
See: `Installing the Base Stations <https://www.vive.com/us/support/vive/category_howto/installing-the-base-stations.html>`_
462+
and `Tips for Setting Up the Base Stations <https://www.vive.com/us/support/vive/category_howto/tips-for-setting-up-the-base-stations.html>`_
463+
464+
.. note::
465+
466+
On first launch of the Manus Vive device, the Vive lighthouses may take a few seconds to calibrate. Keep the Vive trackers
467+
stable and visible to the lighthouse during this time. If the light houses are moved or if tracking fails or is unstable,
468+
calibration can be forced by deleting the calibration file at: ``$XDG_RUNTIME_DIR/libsurvive/config.json``. If XDG_RUNTIME_DIR
469+
is not set, the default directory is ``~/.config/libsurvive``.
470+
471+
For more information consult the libsurvive documentation: `libsurvive <https://github.com/collabora/libsurvive>`_.
472+
435473
.. note::
436474

437475
To avoid resource contention and crashes, ensure Manus and Vive devices are connected to different USB controllers/buses.
438476
Use ``lsusb -t`` to identify different buses and connect devices accordingly.
439477

440-
Vive trackers are automatically calculated to map to the left and right wrist joints.
478+
Vive trackers are automatically calculated to map to the left and right wrist joints obtained from a stable
479+
OpenXR hand tracking wrist pose.
441480
This auto-mapping calculation supports up to 2 Vive trackers;
442481
if more than 2 Vive trackers are detected, it uses the first two trackers detected for calibration, which may not be correct.
443482

0 commit comments

Comments
 (0)