Skip to content

Commit

Permalink
docs: update llm readme (#188)
Browse files Browse the repository at this point in the history
* docs: update llm readme

* markdown
  • Loading branch information
cabreraalex authored Oct 20, 2023
1 parent 8f13ddc commit 67f5316
Showing 1 changed file with 22 additions and 10 deletions.
32 changes: 22 additions & 10 deletions examples/open_llm_leaderboard/README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,35 @@
# Open LLM Leaderboard

This example downloads data from the [Open LLM Leaderboard][1] and ingests it
into a Zeno Project.
There are four tasks in the leaderboard, for each task, there is one notebook
to upload your data.

You can configure which models to upload data for.
Use Zeno to visualize the data and model outputs of the [Open LLM Leaderboard][1]!
There is a notebook for uploading the raw task data and model results for each
of the four tasks in the leaderboard.
The notebooks re-use the outputs from the leaderboard so you don't have to
run any inference to explore the results.

You can pick which model results to upload by passing in the org/model strings
into the notebooks from
the
[Leaderboard Details](https://huggingface.co/datasets/open-llm-leaderboard/details/tree/main)
dataset.

> Explore our
> [example report](https://hub.zenoml.com/report/a13x/What%20does%20the%20OpenLLM%20Leaderboard%20measure%3F)
> to get an idea of what the resulting data will look like.
## Setup

To run this example, you'll need to install the requirements.
To run this example, first install the requirements:

```bash
pip install -r requirements.txt
```

You also need to add an environment variable named `ZENO_API_KEY` that contains
your API key to be able to upload data to Zeno.
You'll then need to get an API key from Zeno Hub.
Create an account at [https://hub.zenoml.com](https://hub.zenoml.com) and navigate
to [your account page](https://hub.zenoml.com/account) to get an API key.
Add this key as an environment variable, `ZENO_API_KEY`.

Then, simply run any of the notebooks.
You can now run the notebooks to create a Zeno Project for any of the
four benchmark datasets.

[1]: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

0 comments on commit 67f5316

Please sign in to comment.