Skip to content

Official repository for Efficient Baselines for Motion Prediction in Autonomous Driving. Presented at CVPR and ICRA Workshops 2022, and ITSC conference 2022.

License

Notifications You must be signed in to change notification settings

MPCheng-ZW/mapfe4mp

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Efficient Baselines for Motion Prediction in Autonomous Driving

arXiv visitors

Carlos Gómez-Huélamo, Marcos V. Conde

RobeSafe group, University of Alcalá, UAH

Computer Vision Lab, CAIDAS, University of Würzburg


News 🚀🚀

  • [01/2023] Work on Transactions on Intelligent Transportation Systems (T-ITS, Q1-JCR)
  • [10/2022] GNN attention-based method will be released. We will extend the baselines :)
  • [09/2022] Ongoing repo update. The most recent work was presented at IEEE ITSC 2022

This is the official repository and PyTorch implementations of different works presented at:

Our papers:

Motion Prediction (MP) of multiple surroundings agents is a crucial task in arbitrarily complex environments, from simple social robots to self-driving vehicles. Current techniques tackle this problem using end-to-end pipelines, where the input data is usually a rendered top-view of the physical information and the past trajectories of the most relevant agents; leveraging this information is a must to obtain optimal performance. In that sense, a reliable Autonomous Driving (AD) system must produce reasonable predictions on time, however, despite many approaches use simple ConvNets and LSTMs, models might be too complex for real-time applications when using both sources of information (map and past trajectories). Moreover, the performance of such models highly depends on the amount of training data, which can be expensive (particularly the annotated High-Definition (HD) maps).

In this work, we propose several baselines for the well-known Argoverse Motion Forecasting Benchmark. We aim to develop compact models using state-of-the-art techniques for MP, including attention mechanisms and GNNs. Our lightweight models use standard social information and interpretable map information such as points from the driveable area and plausible centerlines, in opposition to black-box CNN-based methods for map processing, to generate plausible multimodal trajectories, achieving state-of-the-art accuracy.

Transformer-based Architecture Diagram (click me)

GAN-based Architecture Diagram (click me)


Get Started and Usage

First of all, we highly encourage the use of the VSCode editor, regarding the debugging capability and its quite interesting extensions.

Installation via Docker

Coming soon ...

Installation via Conda

Tested in Ubuntu 16.04, Ubuntu 18.04.

CUDA version >= 11.2

If available, check requirements.txt

conda create --name mapfe4mp_env python=3.8 \
conda install -n mapfe4mp ipykernel --update-deps --force-reinstall
python3 -m pip install --upgrade pip \
python3 -m pip install --upgrade Pillow \
pip install \
    prodict \
    torch \
    pyyaml \
    torchvision \
    tensorboard \
    glob2 \
    matplotlib \
    sklearn \
    gitpython \
    thop \
    fvcore \
    torchstat \
    torchsummary \
    ipykernel \
    sns

In order to install torch-geometric and associated modules for the GNN computations, please check the following documentation:

https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html

E.g.

pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.13.0+cu116.html

N.B. If you find any error with the packages, try to install again the specific package separately.

Download argoverse-api (1.0) in another folder (out of this directory):

git clone https://github.com/argoai/argoverse-api.git

Go to the argoverse-api folder and install the requirements of setup.py.

N.B. Comment those packages that have specific versions, and just write the name of the package in order to have the latest version. E.g. numpy==1.19.9 -> numpy. N.B. You must have the conda environment activated in order to have argoverse as a Python package of your environment

pip install -e .

Don't worry if there are some errors (such as "ERROR: Could not build wheels for opencv-python which use PEP 517 and cannot be installed directly"), just run pip install -e . again.

Finally, copy the map information of the Argoverse Motion-Forecasting

Argoverse Motion-Forecasting dataset v1.1

Check data/README.md.

How to train

Assuming your model is in model/models/your_model.py/, you must have the name of this script in the train.py script. Additionally, it is advisable to have a config file and trainer per model, since they must be quite different. For example:

python train.py --trainer mapfe4mp --device_gpu 3

Notes

Recommended VSCode extensions:

  • Jupyter
  • Python
  • Python for VSCode

If you cannot open a Jupyter Notebook correctly, just reinstall the Jupyter extension in your VSCode editor.

Please, be aware that after installing a new extension, you should reload your VSCode window.

Quantitative results



Please check our papers for further details.

To check your quantitative results using the Tensorboard tool, execute (assuming you have cloned this repo in your $HOME folder):

cd ~/mapfe4mp/save/argoverse/your_model/your_split_percent && tensorboard --logdir . --port=your_port

E.g. cd ~/mapfe4mp/save/argoverse/mapfe4mp/100.0_percent && python -m tensorboard.main --logdir . --port=6100

Qualitative results

If you would like to access all the qualitative samples for the Argoverse 1.0 validation set, please contact us.

We are preparing a tutorial notebook for generating these visualizations:

Citation and Acknowledgement

Please cite this work if you use our code or ideas.

Work done with Miguel Ortiz, Santiago Montiel, Rafael Barea, Luis M. Bergasa - RobeSafe group

@article{gomez2022exploring,
  title={Exploring Map-based Features for Efficient Attention-based Vehicle Motion Prediction},
  author={G{\'o}mez-Hu{\'e}lamo, Carlos and Conde, Marcos V and Ortiz, Miguel},
  journal={arXiv preprint arXiv:2205.13071},
  year={2022}
}

@article{gomez2022gan,
  title={Exploring Attention GAN for Vehicle Motion Prediction},
  author={G{\'o}mez-Hu{\'e}lamo, Carlos and Conde, Marcos V. and Ortiz, Miguel and Montiel, Santiago and Barea, Rafael and Bergasa, Luis M.},
  journal={arXiv preprint arXiv:2209.12674},
  year={2022}
}

Contact

Please add in the email subject "mapfe4mp" or "exploring map features paper"

Carlos Gómez-Huélamo: [email protected]
Marcos Conde: [email protected]

About

Official repository for Efficient Baselines for Motion Prediction in Autonomous Driving. Presented at CVPR and ICRA Workshops 2022, and ITSC conference 2022.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 76.3%
  • Python 23.7%