Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrating contributions from NomicAI for integation to LLama.cpp and GPT4ALL #352

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

axsaucedo
Copy link
Member

@cebtenzzre creating an initial draft PR for integration of contributinos from NomicAI for integation to LLama.cpp and GPT4ALL as part of nomic-ai/gpt4all#1852.

This PR can serve as an opportunity to dive further into some of the requirements, as well as alignment to the newer updates that have been added since, including the extension of the cmake build system.

Changes included in this PR:

  • TODO

cebtenzzre and others added 8 commits May 8, 2024 11:22
libkompute.a, ShaderOpMult.hpp, and ShaderLogisticRegression.hpp are not
needed at runtime by GPT4All, so don't install them.

Signed-off-by: Jared Van Bortel <[email protected]>
Signed-off-by: crydsch <[email protected]>
(cherry picked from commit 2a9f82a)
Signed-off-by: Jared Van Bortel <[email protected]>
This allows Vulkan instances to be reused, since a program may wish to
switch devices, but the NVIDIA driver eventually fails to create Vulkan
instances if you call VkFreeInstance/VkDestroyInstance too many times.

Signed-off-by: Jared Van Bortel <[email protected]>
Sequences tend to reference tensors, so one cleanup pass is not enough.
clear() is most useful if it frees all resources on the device,
especially because we can only call VkCreateDevice/VkDestroyDevice so
many times before the NVIDIA driver can no longer create devices.

Signed-off-by: Jared Van Bortel <[email protected]>
*Technically*, our mat*mat shaders require some level of shader float16
support. Leaving this requirement disabled allows this code to work on
more GPUs without issue, but the validation layers report an error in
this case.

With this change, GPUs that actually claim to support shaderFloat16
should no longer report a validation error.

Signed-off-by: Jared Van Bortel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants