[Question] how to design/devide subtask for long-horizon task when using mimicgen

[wghou]

Question

Hi,

I'm working on sim data generation with dexmimigen, based on the source code in IsaacLab. It work well for single-arm manipulation tasks. But for dual-arm manipulation task, I got confused with how to design subtasks, especially for long-horizon task.

Is there any recipe, tutrials or demos for how to devide subtasks?

For example, in ExhaustPipeGR1T2MimicEnvCfg, NutPourGR1T2MimicEnvCfg and PickPlaceGR1T2MimicEnvCfg, how to design these subtask? Is there any exmples or introduction for how to used SubTaskConstraintConfig?

Thanks!

[RandomOakForest]

Thank you for posting this. I'll move your questions to our Discussions section for the team to follow up.

[peterd-NV]

Hi @wghou, thanks for reaching out. In general there is a lot of freedom for users to define subtasks any way they want but we recommend the following guidance:

1. Each subtask is defined with respect to an object reference that determines how the robot's arm trajectory should be transformed. Therefore, you should define a new stubtask each time the arm's motion changes to depend on a different object. For example, if a robot is picking up a cup and placing it into a bin, there should be two subtasks. The first subtask is w.r.t. the cup as the robot must pick it up, the second subtask is w.r.t to the bin as the arm trajectory now depends on the location of the bin in order to place the cup in it.
2. Minimize the number of subtasks defined. If the reference object does not change, then the is no need to define a new subtask. While defining additional subtasks is possible, having more than is necessary can lead to poor data generation results as additional subtask interpolation/stitching is introduced.

SubTaskConstraintConfig may be used to synchronize and order subtasks between the left and right arms for dual-arm manipulation tasks. We currently do not have an example that requires the need for synchronization in our provided GR1 Humanoid environments. However, you can experiment with them by adding them to your Mimic environment config's task_constraint_configs parameter. Each SubTaskConstraintConfig specifies the temporal relationship between two subtasks in the Mimic environment config's subtask_configs.

For example, to create a SubTaskConstraintConfig for a Humanoid env which enforces that the second subtask of the left arm always ends at the same time as the second subtask of the right arm, you would create the following constraint:

task_constraint = SubTaskConstraintConfig()
task_constraint.eef_subtask_constraint_tuple = (("left", 2), ("right", 2))
task_constraint.constraint_type = SubTaskConstraintType.COORDINATION

[wghou]

Thanks very much for your reply! I will take more experiments and explore the better way to define subtask. Thanks!