What's ggml-large-v3-q5_0.bin? #2316
Unanswered
playgithub
asked this question in
Q&A
Replies: 1 comment 2 replies
-
|
The one with the Q is a quantized model |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
What's a quantized model? |
Beta Was this translation helpful? Give feedback.
-
|
It's a model, where the data is "compressed" by removing redundant data. Since the data is removed, it is using less VRAM/RAM, takes less space on your device and works faster, because it's smaller. In the case of large3-v3, it is 3 times smaller. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
ggml-large-v3-q5_0.bin is about 1.08GB,
ggml-large-v3.bin is about 3.1GB.
What's the difference?
Beta Was this translation helpful? Give feedback.
All reactions