Skip to content

[ECCV 2024] Official Implementation of the paper "HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects"

License

Notifications You must be signed in to change notification settings

Aricling/HIMO_dataset

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects

This repository contains the content of the following paper:

HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects
Xintao Lv 1,* , Liang Xu 1,2* , Yichao Yan 1, Xin Jin 2, Congsheng Xu 1, Shuwen Wu1, Yifan Liu1, Lincheng Li3, Mengxiao Bi3, Wenjun Zeng2, Xiaokang Yang 1
1 Shanghai Jiao Tong University , 2 Eastern Institute of Technology, Ningbo, 3 NetEase Fuxi AI Lab

Dataset Download

Please fill out this form to request authorization to download HIMO for research purposes. After downloading the dataset, unzip the data in ./data and you'll get the following structure:

./data
|-- joints
|   |-- S01T001.npy
|   |-- ...
|-- smplx
|   |-- S01T001.npz
|   |-- ...
|-- object_pose
|   |-- S01T001.npy
|   |-- ...
|-- text
|   |-- S01T001.txt
|   |-- ...
|-- segments
|   |-- S01T001.json
|   |-- ...
|-- object_mesh
|   |-- Apple.obj
|   |-- ...

Data Visualization

We use the AIT-Viewer to visualize the dataset. You can follow the instructions below to visualize it.

pip install -r visualize/requirements.txt

You also need to download the SMPL-X models and place them in ./body_models, which should look like:

./body_models
|-- smplx
    ├── SMPLX_FEMALE.npz
    ├── SMPLX_FEMALE.pkl
    ├── SMPLX_MALE.npz
    ├── SMPLX_MALE.pkl
    ├── SMPLX_NEUTRAL.npz
    ├── SMPLX_NEUTRAL.pkl
    └── SMPLX_NEUTRAL_2020.npz

Then you can run the following command to visualize the dataset.

# Visualize the skeleton
python visualize/skel_viewer.py
# Visualize the SMPLX
python visualize/smplx_viewer.py
# Visualize the segment data
python visualize/segment_viewer.py

Training

To train the model in 2-object setting, run

python -m src.train.train_net_2o --exp_name net_2o --num_epochs 1000

To train the model in 3-object setting, run

python -m src.train.train_net_3o --exp_name net_3o --num_epochs 1000

To evaluate the model, you need to train your own evaluator or use the checkpoint we provide here (put them under ./save). Then run

python -m src.eval.eval_himo_2o

or

python -m src.eval.eval_himo_3o

Visualization

About

[ECCV 2024] Official Implementation of the paper "HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%