Welcome to the LLM Challenge/Contest @ Algorithms Summit 2023. This repository represents a starter kit for participants to get started with the challenge. This challenge explores one aspect of LLM workflows, namely prompt engineering 🛠️ 🔩 💻.
Please go through the following documents to get started with the challenge.
- Setting Python Environment/Dependencies
- Running Notebooks and Python Scripts
- Challenge Dataset and Submission Format
- Challenge Submission and Evaluation
We have created some videos to go along the above documents and understand the task of the challenge better. These videos can be found here. We recommend watching Code Setup and OpenAI API Key Setup before LLM Challenge Overview. The slides that go with the last video can be found here.
- Do not take embeddings at face value. Train an ML model that improves embedding-based retrieval.
- Vector store is not the only way to have an embedding-based retrieval. Check Exemplar SVM for an example of how instance-based ranking of text snippets can be done.
- A chain of LLM calls (e.g., LangChain's stuff, map-reduce, map-rank, map-refine).
- Other ideas discussed in the
LLM Challenge Overviewvideo linked above and listed in the accompanying slides.
Feel free to share your experience on the TEAMS group. Or simply raise an issue for this repository.
As we go through the nuts and bolts of Prompt Engineering, let's remind ourselves with ADI GenAI policy and understand what can and can not be done with these LLMs for ADI. As long as the information that we share with GenAI tools is publicly available, there should not be an problem with ADI's policy. More details can be found here.
We would like to thank Nick Moran, Wenjie Lu, Steve Wacks, Sefa Demirtas, Tao Yu, Matt Crivello, Andrew Fitzell, Marc Light, Dave Boland, and Chris Cianciolo.
Happy prompt-engineering 😎