The IPN Hand Dataset
“A new benchmark video dataset with sufficient size, variation, and real-world elements able to train and evaluate deep neural networks for continuous Hand Gesture Recognition (HGR)”
The IPN Hand dataset contains more than 4,000 gesture instances and 800,000 frames from 50 subjects. We design 13 static and dynamic gestures for interaction with touchless screens. Compared to other publicly available hand gesture datasets, IPN Hand includes the largest number of continuous gestures per video, and the largest speed of intra-class variation.
The data collection was designed considering real-world issues of continuous HGR, including continuous gestures performed without transitional states, natural movements as non-gesture segments, scenes including clutter backgrounds, extreme illumination conditions, as well as static and dynamic environments. Example of continuous gestures without transitional states:
An introduction video of the dataset can be found here. More details in our ICPR2020 paper.
Details
The subjects from the dataset were asked to record gestures using their own PC keeping the defined resolution and frame rate. Thus, only RGB videos were captured, and the distance between the camera and each subject varies. All videos were recorded in the resolution of 640x480 at 30 fps.
Each subject continuously performed 21 gestures with three random breaks in a single video. We defined 13 gestures to control the pointer and actions focused on the interaction with touchless screens.
Description and statics of each gesture are shown in the next table. Duration is measured in the number of frames (30 frames = 1 s).
| id | Label | Gesture | Instances | Mean duration (std) | |
|---|---|---|---|---|---|
| 1 | D0X | Non-gesture | 1431 | 147 (133) | |
| 2 | B0A | Pointing with one finger | 1010 | 219 (67) | |
| 3 | B0B | Pointing with two fingers | 1007 | 224 (69) | |
| 4 | G01 | Click with one finger | 200 | 56 (29) | |
| 5 | G02 | Click with two fingers | 200 | 60 (43) | |
| 6 | G03 | Throw up | 200 | 62 (25) | |
| 7 | G04 | Throw down | 201 | 65 (28) | |
| 8 | G05 | Throw left | 200 | 66 (27) | |
| 9 | G06 | Throw right | 200 | 64 (28) | |
| 10 | G07 | Open twice | 200 | 76 (31) | |
| 11 | G08 | Double click with one finger | 200 | 68 (28) | |
| 12 | G09 | Double click with two fingers | 200 | 70 (30) | |
| 13 | G10 | Zoom in | 200 | 65 (29) | |
| 14 | G11 | Zoom out | 200 | 64 (28) | |
| All non-gestures: | 1431 | 147 (133) | |||
| All gestures: | 4218 | 140 (94) | |||
| Total: | 5649 | 142 (105) |
Video examples of all classes (.GIF) here
Baseline results
Baseline results for isolated and continuous hand gesture recognition of the IPN Hand dataset can be found here.
Downloadable files
Apart from the RGB frames, real-time optical flow and hand segmentation results are also available. The methods used to calculate them are described in our ICPR2020 paper. Examples of the data included in IPN Hand:
Description of each downloadable file:
- MP4 videos
`Original MP4 videos at 640x480, 30 fps` [4.6GB] The 200 videos are compressed into five independent .tgz files, each with 40 videos. - Video frames
`Resized RGB frames to 320x240` [8.37GB] All frames from the 200 videos are compressed into five independent .tgz files. - Optical flow frames
`Color-coded optical flow frames at 320x240` [11.4GB] All frames are compressed into 10 independent .tgz files. - Hand segmentation frames
`Binary masks at 320x240` [982MB] All frames are compressed in a single file of 1GB size. - Annotations
`Metadata and annotations of each video` [364KB] Six .txt, and one .csv files describing train/test labels and metadata, respectively.
Citation
If you find useful the IPN Hand dataset for your research, please cite the paper:
@inproceedings{bega2020IPNhand,
title={IPN Hand: A Video Dataset and Benchmark for Real-Time Continuous Hand Gesture Recognition},
author={Benitez-Garcia, Gibran and Olivares-Mercado, Jesus and Sanchez-Perez, Gabriel and Yanai, Keiji},
booktitle={25th International Conference on Pattern Recognition, {ICPR 2020}, Milan, Italy, Jan 10--15, 2021},
pages={4340--4347},
year={2021},
organization={IEEE}
}
License
The data and annotations in the IPN Hand dataset are licensed under a Creative Commons Attribution 4.0 License.