Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add hugging face text to image integration #1162

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

d-lobo
Copy link

@d-lobo d-lobo commented Aug 4, 2024

Hi,

I was experimenting with Spring AI and some models deployed on hugging face, realizing that a lot of stuff is missing for huggingface explaining the reason for this PR.
The changes provide a baseline implementation for querying text-to-image models deployed on huggingface.
I tried to align the code as much as possible on the existing implementation for chat models, however in order to support different image types on the api, I was required to add a custom template for swagger codegen, since the default templates generate a priorization for application/json with regards to the accept header (which in turn allows the generated api to retrieve base64 encoded strings only, leading to the fact, that only PNG images could be generated). This seems unfavorable for an text-to-image api. Thus the modified templates adds the possibility to specify the desired image type on requesting the model.

  • adds huggingface text-to-image integration
  • provided modified template for swagger codegen (only used for text-to-image inference api)

@markpollack
Copy link
Member

thanks you very much, i did the original quick pass around hugging face and appreciate this improvement.

@d-lobo
Copy link
Author

d-lobo commented Sep 20, 2024

thank you for your response. I wasn't sure if this PR would be considered useful at this moment. However I think I have to make smaller adjustments, as some aspects of the codebase have changed in the meantime, due to other merged PRs, thus marking it as draft. Will take some time, since I'm on vacation right now :D

@d-lobo d-lobo marked this pull request as draft September 20, 2024 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants