Skip to content

Latest commit

 

History

History
32 lines (24 loc) · 2.85 KB

README.md

File metadata and controls

32 lines (24 loc) · 2.85 KB
Raw

ChatGPT-AirSim Interface

Demonstration video

These scripts allow users to interact with and command a drone inside a Microsoft AirSim simulation using ChatGPT. ChatGPT is given access to a high-level function library containing some primitives for navigation, environmental information etc. which in turn maps to several internal AirSim functions. This mapping is done through an AirSim wrapper, which can be seen in airsim_wrapper.py. An example of a carefully crafted prompt (which also contains the high level function details) can be seen in prompts/airsim_basic.txt. The core logic of the interface is in chatgpt_airsim.py.

For details on how to use AirSim and its original API, please visit the AirSim documentation.

Getting started (Windows only - for now)

  • As a prerequisite, you need to create an OpenAI account that allows access to ChatGPT through the OpenAI API. You can do so by visiting https://platform.openai.com and signing up for an account.
  • Set up the conda environment
conda env create -f environment.yml
  • Activate the environment and install the AirSim client.
conda activate chatgpt
pip install airsim
  • Set up an API key by visiting https://platform.openai.com/account/api-keys. Copy the API key and paste it in the OPENAI_API_KEY field of config.json.
  • Download the AirSim zip file from Releases, and unzip the package.
  • Copy settings.json to C:\Users\<username>\Documents\AirSim\.

Running the interface

  • Run the AirSim simulation as .\run.bat from the sim folder.
  • Once simulation is up and running, run python chatgpt_airsim.py from this repo.

The interface shows an AirSim> prompt when it is ready to take user questions/commands. The commands are passed to ChatGPT and once the response is received (please note that the response could take some time to show up depending on ChatGPT response time), the code blocks within the response are automatically extracted and run in AirSim.

We emphasize that this is meant to be a sample environment and interface aimed at getting started with ChatGPT and robotics applications inside a simulation sandbox. ChatGPT generated code could result in unexpected behavior or crashes, especially as the interface is written in a way that it executes the commands directly. That said, we do encourage users to test out the capabilities of the model, and to contribute new prompting techniques, additional wrapper functions, or any other enhancements to this interface.

We encourage users to keep evaluating new methods of accessing the ChatGPT service such as the Azure OpenAI service as appropriate.