initial commit

LICENSE

@@ -0,0 +1,113 @@
+License
+
+Software Copyright License for non-commercial scientific research purposes
+Please read carefully the following terms and conditions and any accompanying documentation before you download
+and/or use the SPEC model, data and software, (the "Model & Software"), including 3D meshes, software, and scripts.
+By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use
+of this github repository), you acknowledge that you have read these terms and conditions, understand them, and
+agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use
+the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights
+under this License
+
+Ownership / Licensees
+The Model & Software and the associated materials has been developed at the
+
+Max Planck Institute for Intelligent Systems (hereinafter "MPI").
+
+Any copyright or patent right is owned by and proprietary material of the
+
+Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (hereinafter “MPG”; MPI and MPG hereinafter
+collectively “Max-Planck”)
+
+hereinafter the “Licensor”.
+
+This software includes the SMPL Body Model. By downloading this software, you are agreeing to be bound by the terms of the SMPL Model License
+
+        https://smpl.is.tue.mpg.de/modellicense
+
+which is necessary to create SMPL body models. 
+
+SMPL bodies that are generated with SPEC can be distributed freely under the SMPL Body License
+
+        https://smpl.is.tue.mpg.de/bodylicense
+
+License Grant
+Licensor grants you (Licensee) personally a single-user, non-exclusive, non-transferable, free of charge right:
+
+To install the Model & Software on computers owned, leased or otherwise controlled by you and/or your organization;
+To use the Model & Software for the sole purpose of performing non-commercial scientific research, non-commercial
+education, or non-commercial artistic projects;
+Any other use, in particular any use for commercial purposes, is prohibited. This includes, without limitation,
+incorporation in a commercial product, use in a commercial service, or production of other artifacts for
+commercial purposes. The Model & Software may not be reproduced, modified and/or made available in any form to
+any third party without Max-Planck’s prior written permission.
+
+The Model & Software may not be used for pornographic purposes or to generate pornographic material whether
+commercial or not. This license also prohibits the use of the Model & Software to train methods/algorithms/neural
+networks/etc. for commercial use of any kind. By downloading the Model & Software,
+you agree not to reverse engineer it.
+
+No Distribution
+The Model & Software and the license herein granted shall not be copied, shared, distributed, re-sold, offered
+for re-sale, transferred or sub-licensed in whole or in part except that you may make one copy for archive
+purposes only.
+
+Disclaimer of Representations and Warranties
+You expressly acknowledge and agree that the Model & Software results from basic research, is provided “AS IS”,
+may contain errors, and that any use of the Model & Software is at your sole risk. LICENSOR MAKES NO REPRESENTATIONS
+OR WARRANTIES OF ANY KIND CONCERNING THE MODEL & SOFTWARE, NEITHER EXPRESS NOR IMPLIED, AND THE ABSENCE OF ANY
+LEGAL OR ACTUAL DEFECTS, WHETHER DISCOVERABLE OR NOT. Specifically, and not to limit the foregoing, licensor
+makes no representations or warranties (i) regarding the merchantability or fitness for a particular purpose of
+the Model & Software, (ii) that the use of the Model & Software will not infringe any patents, copyrights or other
+intellectual property rights of a third party, and (iii) that the use of the Model & Software will not cause any
+damage of any kind to you or a third party.
+
+Limitation of Liability
+Because this Model & Software License Agreement qualifies as a donation, according to Section 521 of the German
+Civil Code (Bürgerliches Gesetzbuch – BGB) Licensor as a donor is liable for intent and gross negligence only.
+If the Licensor fraudulently conceals a legal or material defect, they are obliged to compensate the Licensee
+for the resulting damage.
+
+Licensor shall be liable for loss of data only up to the amount of typical recovery costs which would have
+arisen had proper and regular data backup measures been taken. For the avoidance of doubt Licensor shall be
+liable in accordance with the German Product Liability Act in the event of product liability. The foregoing
+applies also to Licensor’s legal representatives or assistants in performance. Any further liability shall be excluded.
+Patent claims generated through the usage of the Model & Software cannot be directed towards the copyright holders.
+The Model & Software is provided in the state of development the licensor defines. If modified or extended by
+Licensee, the Licensor makes no claims about the fitness of the Model & Software and is not responsible
+for any problems such modifications cause.
+
+No Maintenance Services
+You understand and agree that Licensor is under no obligation to provide either maintenance services,
+update services, notices of latent defects, or corrections of defects with regard to the Model & Software.
+Licensor nevertheless reserves the right to update, modify, or discontinue the Model & Software at any time.
+
+Defects of the Model & Software must be notified in writing to the Licensor with a comprehensible description
+of the error symptoms. The notification of the defect should enable the reproduction of the error.
+The Licensee is encouraged to communicate any use, results, modification or publication.
+
+Publications using the Model & Software
+You acknowledge that the Model & Software is a valuable scientific resource and agree to appropriately reference
+the following paper in any publication making use of the Model & Software.
+
+Subjects' Consent
+All subjects gave informed written consent to share their data for research purposes. You agree to delete data or
+change their use, in case a subject changes or withdraws their consent.
+
+Citation:
+
+@inproceedings{SPEC:ICCV:2021,
+  title = {{SPEC}: Seeing People in the Wild with an Estimated Camera},
+  author = {Kocabas, Muhammed and Huang, Chun-Hao P. and Tesch, Joachim and M\"uller, Lea and Hilliges, Otmar and Black, Michael J.},
+  booktitle = {Proc. International Conference on Computer Vision (ICCV)},
+  pages = {11035--11045},
+  month = oct,
+  year = {2021},
+  doi = {},
+  month_numeric = {10}
+}
+
+Commercial licensing opportunities
+For commercial uses of the Model & Software, please send email to [email protected]
+
+This Agreement shall be governed by the laws of the Federal Republic of Germany except for the UN Sales Convention.

README.md

@@ -0,0 +1,195 @@
+# SPEC: Seeing People in the Wild with an Estimated Camera [ICCV 2021]
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)]()
+[![report](https://img.shields.io/badge/Project-Page-blue)](https://spec.is.tue.mpg.de/)
+[![report](https://img.shields.io/badge/ArXiv-Paper-red)](https://arxiv.org/abs/2110.00620)
+
+> [**SPEC: Seeing People in the Wild with an Estimated Camera**](https://arxiv.org/abs/2110.00620),            
+> [Muhammed Kocabas](https://ps.is.tuebingen.mpg.de/person/mkocabas), 
+> [Chun-Hao Paul Huang](https://ps.is.tuebingen.mpg.de/person/chuang2), 
+> [Joachim Tesch](https://ps.is.tuebingen.mpg.de/person/jtesch), 
+> [Lea Müller](https://ps.is.tuebingen.mpg.de/person/lmueller2), 
+> [Otmar Hilliges](https://ait.ethz.ch/people/hilliges/),
+[Michael J. Black](https://ps.is.tuebingen.mpg.de/person/black),        
+> *International Conference on Computer Vision (ICCV), 2021* 
+
+<p float="center">
+  <img src="docs/assets/spec_gif.gif" width="100%" />
+</p>
+
+## Features
+
+SPEC is a camera-aware human body pose and shape estimation method. 
+It both predicts the camera parameters and SMPL body model for a given image.
+[CamCalib](camcalib) predicts the camera parameters. 
+[SPEC](spec) uses these parameters to predict SMPL body model parameters.
+
+This implementation:
+- has the demo code for SPEC and CamCalib implemented in PyTorch.
+- achieves SOTA results in SPEC-SYN and SPEC-MTP datasets.
+- shows how to perform evaluation on SPEC-SYN and SPEC-MTP datasets.
+
+## Updates
+
+- 13/10/2021: Demo and evaluation code is released.
+
+## Getting Started
+
+SPEC has been implemented and tested on Ubuntu 18.04 with python >= 3.7.
+If you don't have a suitable device, try running our Colab demo.
+
+Clone the repo:
+
+```shell
+git clone https://github.com/mkocabas/SPEC.git
+```
+
+Install the requirements using virtualenv or conda:
+
+```shell
+# pip
+source scripts/install_pip.sh
+
+# conda
+source scripts/install_conda.sh
+```
+
+## Running the Demo
+
+### SPEC
+
+First, you need to download the required data 
+(i.e our trained model and SMPL model parameters). It is approximately 1GB. 
+To do this you can just run:
+
+```shell
+source scripts/prepare_data.sh
+```
+
+Then, running the demo is as simple as:
+
+```shell
+python scripts/spec_demo.py \
+  --image_folder data/sample_images \
+  --output_folder logs/spec/sample_images
+```
+Sample demo output:
+
+<p float="left">
+  <img src="docs/assets/COCO_val2014_000000386912_000000.jpg" width="70%" />
+</p>
+
+Here the green line is the horizon obtained using estimated camera parameters.
+On the right, the ground plane is visualized to show how accurate the global translation is.
+
+### CamCalib
+
+If you are only interested in estimating the camera parameters of an image, 
+run the CamCalib demo:
+
+```shell
+python scripts/camcalib_demo.py \
+  --img_folder <input image folder> \
+  --out_folder <output folder> \
+  --show # visualize the raw network predictions
+```
+
+This script outputs a pickle file which contains the predicted camera parameters for each input image along with an output
+image which visualizes the camera parameters as a horizon line. Pickle file contains:
+```
+'vfov' : vertical field of view in radians
+'f_pix': focal length in pixels
+'pitch': pitch in radians
+'roll' : roll in radians
+```
+
+## Google Colab
+
+## Training
+
+Training instructions will follow soon.
+
+## Datasets
+
+Pano360, SPEC-MTP, and SPEC-SYN are new datasets introduced in our paper.
+You can download them from the [Downloads](https://spec.is.tue.mpg.de/download.php) 
+section of our [project page](https://spec.is.tue.mpg.de).
+
+For Pano360 dataset, we have released the Flickr image ids which can be used to download images
+using FlickrAPI. 
+We have provided a [download script](scripts/download_flickr.py) in this repo.
+Some of the images will be missing due to users deleting their photos. 
+In this case, you can also use `scrape_and_download` function provided 
+in the script to find and download more photos.
+
+After downloading the SPEC-SYN, SPEC-MTP, Pano360, and [3DPW](https://virtualhumans.mpi-inf.mpg.de/3DPW/) 
+datasets, the `data` folder should look like:
+
+```shell
+data/
+├── body_models
+│   └── smpl
+├── camcalib
+│   └── checkpoints
+├── dataset_extras
+├── dataset_folders
+│   ├── 3dpw
+│   ├── pano360
+│   ├── spec-mtp
+│   └── spec-syn
+├── sample_images
+└── spec
+    └── checkpoints
+```
+
+## Evaluation
+You can evaluate SPEC on SPEC-SYN, SPEC-MTP, and 3DPW datasets by running:
+```shell
+python scripts/spec_eval.py \
+  --cfg data/spec/checkpoints/spec_config.yaml \
+  --opts DATASET.VAL_DS spec-syn_spec-mtp_3dpw-test-cam
+```
+
+Running this script should give results reported in this table:
+
+| | W-MPJPE | PA-MPJPE | W-PVE|
+|--- | --- | --- | ---|
+|SPEC-MTP | 124.3 | 71.8 | 147.1|
+|SPEC-SYN | 74.9 | 54.5 | 90.5|
+|3DPW | 106.7 | 53.3 | 124.7|
+
+## Citation
+
+```bibtex
+@inproceedings{SPEC:ICCV:2021,
+  title = {{SPEC}: Seeing People in the Wild with an Estimated Camera},
+  author = {Kocabas, Muhammed and Huang, Chun-Hao P. and Tesch, Joachim and M\"uller, Lea and Hilliges, Otmar and Black, Michael J.},
+  booktitle = {Proc. International Conference on Computer Vision (ICCV)},
+  pages = {11035--11045},
+  month = oct,
+  year = {2021},
+  doi = {},
+  month_numeric = {10}
+}
+```
+## License
+
+This code is available for **non-commercial scientific research purposes** as defined in the [LICENSE file](LICENSE). By downloading and using this code you agree to the terms in the [LICENSE](LICENSE). Third-party datasets and software are subject to their respective licenses.
+
+## References
+
+We indicate if a function or script is borrowed externally inside each file.
+Here are some great resources we benefit:
+
+- Pano360 dataset preprocessing is borrowed from [Single View Metrology in the Wild](https://github.com/Jerrypiglet/ScaleNet).
+- Most of the utility functions depends on our another paper [PARE](github.com/mkocabas/PARE).
+
+Consider citing these works if you use them in your project.
+
+
+
+## Contact
+
+For questions, please contact [email protected]
+
+For commercial licensing (and all related questions for business applications), please contact [email protected].