Skip to content

federated learning autonomous driving in CARLA simulation

Notifications You must be signed in to change notification settings

SIAT-INVS/CarlaFLCAV

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CarlaFLCAV

carla_flcav

CarlaFLCAV is an open-source FLCAV simulation platform based on CARLA simulator that supports:

  • Multi-modal dataset generation: Including point-cloud, image, radar data with associated calibration, synchronization, and annotation

  • Training and inference: Examples for CAV perception, including object detection, traffic sign detection, and weather classification

  • Various FL frameworks: FedAvg, device selection, noisy aggregation, parameter selection, distillation, and personalization

  • Optimization based modules: Network resource and road sensor pose optimization.

Demo

fedsecond.mp4

Federated SECOND for 3D point cloud object detection

fedyolo.mp4

Federated YOLOV5 for 2D image object detection

fedlstm.mp4

Federated LSTM for BEV trajectory prediction

fedfusion.mp4

Cooperative perceptioin with road sensors for federated distillation

Test Environment

  • Ubuntu 20.04
  • Python 3.8
  • CARLA 0.9.13
  • CUDA 11.3 (Nvidia Driver 470)
  • Pytorch 1.10.0

Citation

CarlaFLCAV can reproduce results in the following papers:

@article{CarlaFLCAV,
  title={Federated deep learning meets autonomous vehicle perception: Design and verification},
  author={Shuai Wang and Chengyang Li and Derrick Wing Kwan Ng and Yonina C. Eldar and H. Vincent Poor and Qi Hao and Chengzhong Xu},
  journal={IEEE Network},
  year={2023},
  volume={37},
  number={3},
  pages={16--25}
}

@article{CarlaFLOTA,
  title={Edge federated learning via unit-modulus over-the-air computation},
  author={Shuai Wang and Yuncong Hong and Rui Wang and Qi Hao and Yik-Chung Wu and Derrick Wing Kwan Ng},
  journal={IEEE Transactions on Communications},
  year={2022},
  volume={70},
  number={5},
  pages={3141--3156}
}

CarlaFLCAV Arxiv version: http://arxiv.org/abs/2206.01748

CarlaFLOTA Arxiv version: https://arxiv.org/abs/2101.12051

Acknowledgement

Authors

Shuai Wang

Chengyang Li