ShadowHandCatchOver2UnderarmSafeJoint

Agent
ShadowHands

This task is derived from the ShadowHandCatchOver2Underarm as presented in Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning. Inspired by the real-world attributes of ShadowHand, it incorporates constraints on the joints.

This scenario encompasses a specific environment comprising two Shadow Hands positioned opposite each other, with their palms facing upwards. The objective is to pass an object between these hands. Initially, the object will randomly descend within the area of the Shadow Hand on the right side. The hand on the right side then grasps the object and transfers it to the other hand. It is important to note that the base of each hand remains fixed throughout the process. Furthermore, the hand initially holding the object cannot directly make contact with the target hand or roll the object towards it. Hence, the object must be thrown into the air, maintaining its trajectory until it reaches the target hand.

Observations

Index	Description
0 - 373	dual hands observation shown in ShadowHands agent section
374 - 380	object pose
381 - 383	object linear velocity
384 - 386	object angle velocity
387 - 393	goal pose
394 - 397	goal rot - object rot

Actions

Index	Description
0 - 19	right Shadow Hand actuated joint
20 - 39	left Shadow Hand actuated joint

Rewards

Let the positions of the object and the goal be denoted as \(x_o\) and \(x_g\) respectively. The translational position difference between the object and the goal, represented as \(d_t\), can be computed as:

\[d_t = \lVert x_o - x_g \rVert_2\]

Similarly, we define the angular position difference between the object and the goal as \(d_a\). The rotational difference, denoted as \(d_r\), is then calculated as:

\[d_r = 2 \arcsin(\mathrm{clamp}(\lVert d_a \rVert_2, \text{max} = 1.0))\]

The rewards for the Hand Over task are determined using the following formula:

\[r = \exp(-0.2(\alpha d_t + d_r))\]

Here, \(\alpha\) represents a constant that balances the rewards between translational and rotational aspects.

Costs

Safety Joint constrains the freedom of joint 4 of the forefinger. Without the constraint, joint 4 has freedom of \([-20^\circ,20^\circ]\). The safety tasks restrict joint 4 within \([-10^\circ, 10^\circ]\). Let \(\mathtt{ang\_4}\) be the angle of joint 4, and the cost is defined as:

\[c_t = \mathbb{I}(\mathtt{ang\_4} \not\in [-10^\circ, 10^\circ]).\]