New metrics for benchmark

[mathmax12]

🚀 The feature, motivation and pitch

Very nice work, Llama team. Llama is the most popular open-source LLM project and has been adopted for different platforms. It will be great to see more metrics, e.g., tokens/s, sample/s, and % TFLOps, collected during training for benchmarking purposes.

Alternatives

No response

Additional context

No response

[init27]

Thanks for the feedback!
Could you please share what metrics you'd love to see (if any) apart from the ones shared in our evals?

[mathmax12]

Here are some thoughts that come to mind:
Average TFLOP/s per GPU
Average Tokens/s per GPU
Average Samples/s
GPU Utilization (% TFLOP/s per GPU)