Abstract
Dexterous manipulation is limited by both control and design, with no consensus on what makes a manipulator best for dexterous tasks. This raises a fundamental challenge: can we generate entire robot hands and control policies optimized for task dexterity?
We present a robot hand co-design framework that jointly learns task-specific hand morphology and dexterous control policies. It pairs an expansive search over joints, fingers, and palms with scalable evaluation through morphology-conditioned cross-embodied control, and fabricates designs from accessible components. Validated on in-hand rotation in simulation and on real hardware, the open-source pipeline can design, train, fabricate, and deploy a new robot hand in under 24 hours.
Paper
Explainer Video
Method
House of Dextra treats robot hand co-design as a joint hardware–software problem: the hand's morphology and its dexterous manipulation policy are optimized together. Our approach combines three components: grammar-based hand generation that constructs anthropomorphic and non-anthropomorphic hand families from primitive components with pre-computed collision meshes; a morphology-conditioned cross-embodied policy using action masking to efficiently evaluate designs across varying degrees of freedom, finger counts, lengths, and fingertip types; and a modular hardware platform enabling zero-shot sim-to-real transfer. Design search uses Graph Heuristic Search adapted for cross-embodied dexterous evaluation.
Three Specialized Tasks
In-Hand Rotation
Rotate objects from 16 YCB items on the x-axis from a resting pose.
Grasping
Grasp object with fixed wrist and hold until reset.
Flipping
Rotate an object resting on the ground about the z-axis with a fixed wrist.
Generalization & Hardware
We evaluate our co-designed hands completely blind in the real world on unseen objects, with no state or tactile information given to the policy. Four co-designed hands are tested sim-to-real, demonstrating generalization across rigid and soft objects with varying shape, weight, and texture. Simulation rankings reliably predict real-world performance.
The platform is designed for easy assembly with modular fingertips, degrees of freedom, finger count, palm geometry, and finger placement. Using only bolts, a screwdriver, and a 3D printer, the full system can be assembled in under 4 hours. A single control board handles all electronics. The modular design is intended for broad use in manipulation co-design research.
Team
Ali El Lahib1
1University of California, San Diego 2University of California, Santa Barbara