Document supported models, devices and screenshot of chrome traces (#13)

README.md

@@ -88,9 +88,27 @@ The simulator supports a plethora of parameters for the simulation description w
 
 The metrics will be logged to wandb directly and a copy will be stored in the `simulator_output` directory along with the chrome trace. A description of all the logged metrics can be found [here](vidur/metrics/README.md).
 
-## Adding a new model
+## Supported Models
 
-Instructions on adding a new model can be found [here](vidur/profiling/README.md).
+| Model / Device | A100 80GB DGX | H100 DGX | 4xA100 80GB Pairwise NVLink Node | 8xA40 Pairwise NVLink Node |
+| --- | --- | --- | --- | --- |
+| `meta-llama/Llama-2-7b-hf` | ✅ | ✅ | ✅ | ✅ |
+| `codellama/CodeLlama-34b-Instruct-hf"` | ✅ | ✅ | ✅ | ✅ |
+| `meta-llama/Llama-2-70b-hf` | ✅ | ✅ | ✅ | ✅ |
+| `internlm/internlm-20b` | ✅ | ✅ | ✅ | ✅ |
+| `Qwen/Qwen-72B` | ✅ | ✅ | ✅ | ✅ |
+
+* Pipeline parallelism is supported for all models. The PP dimension should divide the number of layers in the model.
+* In DGX nodes, there are 8 GPUs, fully connected via NVLink. So TP1, TP2, TP4 and TP8 are supported.
+* In 4x pairwise NVLink nodes, there are 4 GPUs, so TP1, TP2 and TP4 are supported. TP4 here is less performant than TP4 in DGX nodes because (GPU1, GPU2) are connected via NVLink and (GPU3, GPU4) are connected via NVLink. but between these layers, the interconnect is slower.
+* You can use any combination of TP and PP. For example, you can run LLaMA2-70B on TP2-PP2 on a 4xA100 80GB Pairwise NVLink Node.
+* Instructions on adding a new model to existing or new SKUs can be found [here](vidur/profiling/README.md).
+
+## Chrome Trace
+
+Vidur exports chrome traces of each simulation. The trace can be found in the `simulator_output` directory. The trace can be opened by navigating to `chrome://tracing/` or `edge://tracing/` and loading the trace.
+
+![Chrome Trace](./assets/chrome_trace.png)
 
 ## Formatting Code