dorsal/arxiv
View SchemaDexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulation
| Authors | Yutong Liang, Shiyi Xu, Yulong Zhang, Bowen Zhan, He Zhang, Libin Liu |
|---|---|
| Categories | |
| ArXiv ID | 2601.05844vv1 |
| URL | https://arxiv.org/abs/2601.05844 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Capturing fine-grained hand-object interactions is challenging due to severe self-occlusion from closely spaced fingers and the subtlety of in-hand manipulation motions. Existing optical motion capture systems rely on expensive camera setups and extensive manual post-processing, while low-cost vision-based methods often suffer from reduced accuracy and reliability under occlusion. To address these challenges, we present DexterCap, a low-cost optical capture system for dexterous in-hand manipulation. DexterCap uses dense, character-coded marker patches to achieve robust tracking under severe self-occlusion, together with an automated reconstruction pipeline that requires minimal manual effort. With DexterCap, we introduce DexterHand, a dataset of fine-grained hand-object interactions covering diverse manipulation behaviors and objects, from simple primitives to complex articulated objects such as a Rubik's Cube. We release the dataset and code to support future research on dexterous hand-object interaction.
{
"annotation_id": "ffa26028-6e7c-4d86-be08-ea44ee6f4647",
"date_created": "2026-02-17T05:53:04.065000Z",
"date_modified": "2026-02-17T05:53:04.065000Z",
"file_hash": "2ae3c6251edcd283e4d39ab2cec8b420ac49354b6fed836553ffc1f887521a2d",
"private": false,
"record": {
"abstract": "Capturing fine-grained hand-object interactions is challenging due to severe self-occlusion from closely spaced fingers and the subtlety of in-hand manipulation motions. Existing optical motion capture systems rely on expensive camera setups and extensive manual post-processing, while low-cost vision-based methods often suffer from reduced accuracy and reliability under occlusion. To address these challenges, we present DexterCap, a low-cost optical capture system for dexterous in-hand manipulation. DexterCap uses dense, character-coded marker patches to achieve robust tracking under severe self-occlusion, together with an automated reconstruction pipeline that requires minimal manual effort. With DexterCap, we introduce DexterHand, a dataset of fine-grained hand-object interactions covering diverse manipulation behaviors and objects, from simple primitives to complex articulated objects such as a Rubik\u0027s Cube. We release the dataset and code to support future research on dexterous hand-object interaction.",
"arxiv_id": "2601.05844",
"authors": [
"Yutong Liang",
"Shiyi Xu",
"Yulong Zhang",
"Bowen Zhan",
"He Zhang",
"Libin Liu"
],
"categories": [
"cs.GR",
"cs.AI",
"cs.RO"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulation",
"url": "https://arxiv.org/abs/2601.05844",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "8daf793f-ce56-4b02-9528-e987cffc66be",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}