Code and Models for "Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection" in ICCV 2017.

By Mohammadreza Zolfaghari, Gabriel L. Oliveira, Nima Sedaghat, Thomas Brox

Update

• 2018.4.30: Scripts for creating body-part mask to train body-part segmentation network.
• 2018.2.26: The pretrained models and scripts for creating human pose maps are released.

Contents

0. Citation
1. Requirements
2. Installation
3. Usage
4. Models
5. Results
6. [Project page](#Project page)

Citation

If you find ChainedNet useful in your research, please consider to cite:

    @InProceedings{ZOSB17a,
    author       = "Mohammadreza Zolfaghari and
                Gabriel L. Oliveira and
                Nima Sedaghat and
                Thomas Brox",
    title        = "Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection",
    booktitle    = "IEEE International Conference on Computer Vision (ICCV)",
    month        = " ",
    year         = "2017",
    url          = "http://lmb.informatik.uni-freiburg.de/Publications/2017/ZOSB17a"
    }

Requirements

1. Requirements for Python
2. Requirements for Matlab
3. Requirements for Caffe and pycaffe (see: Caffe installation instructions)

Installation

1. git clone ... TODO.

2. Build Caffe and pycaffe

   cd $caffe_FAST_ROOT/
   # Now follow the Caffe installation instructions here:
   # http://caffe.berkeleyvision.org/installation.html
   make all -j8 && make pycaffe && make matcaffe

Usage

After successfully completing the installation, you are ready to run all the following experiments.

Part 0: Network Inputs

• RGB, use extract_frames_frmRate.sh in the scripts folder to extract frames.

• Optical Flow: TVL1, Brox

• Pose

       Inputs            
  

Part 1: Body Part Segmentation

Please follow steps explained in Body Part Segmentation

Image+mask	BodyPart mask

Part 2: Training the Chained Multi-stream network

Note: TODO

Part 3: Results

Note: TODO

Recognition	Detection

Project page

https://lmb.informatik.uni-freiburg.de/projects/action_chain/

Contact

Mohammadreza Zolfaghari

Questions can also be left as issues in the repository. We will be happy to answer them.