Skip to content

Latest commit

 

History

History
106 lines (76 loc) · 4.51 KB

README.md

File metadata and controls

106 lines (76 loc) · 4.51 KB
Raw

4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras

Note: As a python variable name cannot start with a number, we refer to this method as FourDAG in the following text and code.

Introduction

We provide the config files for FourDAG: 4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras.

Official Implementation

@inproceedings{Zhang20204DAG,
  title={4D Association Graph for Realtime Multi-Person Motion Capture Using Multiple Video Cameras},
  author={Yuxiang Zhang and Liang An and Tao Yu and Xiu Li and Kun Li and Yebin Liu},
  journal={IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2020},
  pages={1321-1330}
}

Prepare limb information and datasets

  • Prepare limb information:
sh scripts/download_weight.sh

You could find perception models in weight file.

  • Prepare the datasets:

You could download Shelf, Campus or FourDAG datasets, and convert original dataset to our unified meta-data. Considering that it takes long to run a converter, we have done it for you. Please download compressed zip file for converted meta-data from here, and place meta-data under ROOT/xrmocap_data/DATASET.

The final file structure would be like:

xrmocap
├── xrmocap
├── docs
├── tools
├── configs
├── weight
|   └── limb_info.json
└── xrmocap_data
    ├── CampusSeq1
    ├── Shelf
    |   ├── Camera0
    |   ├── ...
    |   ├── Camera4
    |   └── xrmocap_meta_testset
    └── FourDAG
        ├── seq2
        ├── seq4
        ├── seq5
        ├── xrmocap_meta_seq2
        ├── xrmocap_meta_seq4
        └── xrmocap_meta_seq5

You can download just one dataset of Shelf, Campus and FourDAG.

Results

We evaluate FourDAG on 3 benchmarks, report the Percentage of Correct Parts (PCP) on Shelf/Campus/FourDAG datasets.

You can find the recommended configs in configs/foudage/*/eval_keypoints3d.py.

Campus

The 2D keypoints and pafs data we use is generated by openpose, and you can download it from here.

Config Actor 0 Actor 1 Actor 2 Average PCK@100mm MPJPE PA-MPJPE Download
eval_keypoints3d.py 64.58 91.90 87.99 81.49 65.07 287.81 168.48 log

Shelf

The 2D keypoints and pafs data we use is generated by fasterrcnn, and you can download it from here.

Config Actor 0 Actor 1 Actor 2 Average PCK@100mm MPJPE PA-MPJPE Download
eval_keypoints3d.py 99.72 97.00 92.55 96.43 97.24 51.31 43.54 log

FourDAG

The 2D keypoints and pafs data we use is generated by mmpose, and you can download it from here.

  • seq2
Config Actor 0 Actor 1 Average PCK@200mm MPJPE PA-MPJPE Download
eval_keypoints3d.py 91.38 85.75 88.57 95.26 105.56 81.67 log
  • seq4
Config Actor 0 Actor 1 Actor 2 Average PCK@200mm MPJPE PA-MPJPE Download
eval_keypoints3d.py 91.22 86.97 92.62 90.27 96.26 97.71 78.94 log