Skip to content

Latest commit

 

History

History
107 lines (68 loc) · 3.39 KB

README.md

File metadata and controls

107 lines (68 loc) · 3.39 KB

Towards single camera human 3D-kinematics

This repository is currently under construction

Installation

  1. Requirement

     Python 3.8.0 
 PyTorch 1.11.0
 OpenSim 4.3+

  2. Python package

    Clone this repo and run the following:

     conda env create -f environment_setup.yml

    Activate the environment using

     conda activate d3ke

  3. OpenSim 4.3

    1. Download and Install OpenSim

    2. (On Windows)Install python API

      • In installation_folder/OpenSim 4.x/sdk/Python, run

          python setup_win_python38.py

  python -m pip install .

    3. (On other operating systems) Follow the instructions to setup the opensim scripting environment here

    4. Copy all *.obj files from resources/opensim/geometry to <installation_folder>/OpenSim 4.x/Geometry

    Note: Scripts requiring to import OpenSim are only verified on Windows.

Dataset and SMPL+H models

  1. BMLmovi
    • Register to get access to the downloads section.
    • Download .avi videos of PG1 and PG2 cameras from the F round (F_PGX_Subject_X_L.avi).
    • Download Camera Parameters.tar.
    • Download v3d files (F_Subjects_1_45.tar).
  2. AMASS
    • Download SMPL+H body data of BMLmovi.
  3. SMPL+H Models
    • Register to get access to the downloads section.
    • Download the extended SMPL+H model (used in AMASS project).
  4. DMPLs
    • Register to get access to the downloads section.
    • Download DMPLs for AMASS.
  5. PASCAL Visual Object Classes (ONLY NECESSARY FOR TRAINING)
    • Download the training/validation data

Unpacking resources

  1. Unpack the downloaded SMPL and DMPL archives into ms_model_estimation/resources

  2. Unpack the downloaded AMASS data into the top-level folder resources/amass

  3. Unpack the F_Subjects_1_45 folder and unpack content of all subfolders into resources/V3D/F

OpenSim GT Generation

Run the generate_opensim_gt script:

python generate_opensim_gt.py

This process might take several hours!

Once the dataset is generated the scaled OpenSim model and motion files can be found in resources/opensim/BMLmovi/BMLmovi

Dataset Preparation

After the ground truth has been generated, the dataset needs to be prepared.

Run the prepare_dataset script and provide the location where the BMLMovi videos are stored:

python prepare_dataset.py --BMLMoviDir path/to/bmlmovi/videos

NOTE: for generating data for training you should also provide the path to the Pascal VOC dataset

python prepare_dataset.py --BMLMoviDir path/to/bmlmovi/videos --PascalDir path/to/pascal_voc/data

This process might again take several hours!

Evaluation

Download models

Run inference

Run the run_inference script :

python run_inference.py

This will use D3KE to run predictions on the subset of BMLMovi used for testing.

Training