AllocNet: Learning Time Allocation for Trajectory Optimization

About

AllocNet is a lightweight learning-based trajectory optimization framework.

Authors: Yuwei Wu, Xiatao Sun, Igor Spasojevic, and Vijay Kumar from the Kumar Lab.

Video Links: Youtube

Related Paper: Y. Wu, X. Sun, I. Spasojevic and V. Kumar, "Deep Learning for Optimization of Trajectories for Quadrotors," in IEEE Robotics and Automation Letters, vol. 9, no. 3, pp. 2479-2486, March 2024 arxiv Preprint

If this repo helps your research, please cite our paper at:

@ARTICLE{10412114,
  author={Wu, Yuwei and Sun, Xiatao and Spasojevic, Igor and Kumar, Vijay},
  journal={IEEE Robotics and Automation Letters}, 
  title={Deep Learning for Optimization of Trajectories for Quadrotors}, 
  year={2024},
  volume={9},
  number={3},
  pages={2479-2486}}

Acknowledgements

• Dataset: The raw point cloud dataset from M3ED
• Front-end Path Planning: We use OMPL planning library
• Planning Modules and Visualization: We use the module in GCOPTER

Run our pre-trained Model in Simulation

The repo has been tested on 20.04 with ros-desktop-full installation.

1. Prerequisites

1.1 ROS and OMPL

Follow the guidance to install ROS and install OMPL:

sudo apt install libompl-dev

1.2 libtorch

Download the libtorch and put it into the planner folder: GPU version, or CPU version

1.3 QP solver

We use osqp to solve quadratic programming, install by:

git clone -b release-0.6.3 https://github.com/osqp/osqp.git
cd osqp
git submodule init
git submodule update
mkdir build & cd build
cmake ..
sudo make install

cd ../..
git clone https://github.com/robotology/osqp-eigen.git
cd osqp-eigen
mkdir build & cd build
cmake ..
sudo make install

2. Build on ROS

git clone git@github.com:yuwei-wu/AllocNet.git && cd AllocNet/src
wstool init && wstool merge utils.rosinstall && wstool update
catkin build

The default mode is set to the GPU version. To switch to the CPU, navigate to line 29 in the 'learning_planning.hpp' file and replace 'device(torch::kGPU)' with 'device(torch::kCPU)'. After making this change, recompile the code for the updates to take effect.

You can also check: - Installing C++ Distributions of PyTorch

3. Run

source devel/setup.bash
roslaunch planner learning_planning.launch

click 2D Nav Goal to trigger planning:

Train new models

0. Folder Structure

network/
│
├── config/                - Configuration files for training and testing.
│
│
├── utils/                 - Utility functions and classes.
│   └── learning/          - Contains network classes and layers
│
└── train_minsnap_<...>.py - Scripts for training
└── test_minsnap_<...>.py  - Scripts for testing
└── ts_conversion_<...>.py - Scripts for converting to TorchScript

1. Pre-requisites

• Ubuntu 20.04 / Windows 11
  • If using WSL2 with simulation running in windows, please add export WSL_HOST_IP=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}') to your .bashrc file to allow communication between Windows and the subsystem.
• Python 3.8
• CUDA 11.7

2. Setup

2.1 Install Dependencies

• Install Ubuntu packages
  • sudo apt-get install python3-dev python3-venv
• Create a virtual environment
  • python3 -m venv venv
• Activate the virtual environment
  • source venv/bin/activate
• Install the requirements
  • pip install wheel
  • pip install numpy==1.24.2
  • pip install -r requirements.txt

2.2 Setup Iris

Follow the instructions to install, and you may need to change the CMakeLists.txt in iris-distro/CMakeLists.txt

iris: https://github.com/rdeits/iris-distro

For AMD CPU, if you encounter a core dump, please refer to instructions in this link:

https://github.com/rdeits/iris-distro/issues/81

pip install -U kaleido

3. Run

• For training, please run python train_minsnap_<model_configuration>.py
• For testing, please run python test_minsnap_<model_configuration>.py
• For converting the learned model to TorchScript, please run python ts_conversion_<model_configuration>.py

Maintaince

For any technical issues, please contact Yuwei Wu (yuweiwu@seas.upenn.edu, yuweiwu20001@outlook.com).