Our demo videos showcase MMHMR's ability to reconstruct highly accurate and realistic 3D hand meshes from single RGB images, overcoming challenges like complex articulations, self-occlusions, and depth ambiguities.
MMHMR Training
MMHMR Training Phase . MMHMR consists of two key components: (1) VQ-MANO, which encodes 3D hand poses into a sequence of discrete tokens within a latent space, and (2) a Context-Guided Masked Transformer that models the probabilistic distributions of these tokens, conditioned on the input image, 2D pose cues, and a partially masked token sequence.
State-of-the-Art Comparison
Effectiveness of Proposed MMHMR Components
Impact of most Confident Tokens
Impact of pose token confidence on MMHMR's reconstruction accuracy and diversity in occluded regions. The figure compares 3D hand meshes using the 1st, 7th, 10th, 100th, 250th, and 1000th most confident tokens. Higher-confidence tokens yield accurate reconstructions, while lower-confidence ones introduce variations, demonstrating how MMHMR captures uncertainty and generates diverse reconstructions, especially in occluded regions.
HINT Benchmark: Challenging Poses in Wild
Complex and Occluded Scenarios
Visualizing In-Wild Results
Visualizing Reference Keypoints and their Offsets
Mesh-Guided Control for Hand Image Generation
Text-to-Mesh Generation
