Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Bard, Anthropic, Cohere, Azure, OpenAI Models - using litellm #503

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

ishaan-jaff
Copy link

I'm the maintainer of litellm https://github.com/BerriAI/litellm - a simple & light package to call Bard, Anthropic, Cohere, Azure, OpenAI LLM Models

This PR adds support for models from all the above mentioned providers, in this pr I made a new adapter called litellm.py

Here's a sample of how it's used:

from litellm import completion

## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion("command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

@sre-ci-robot sre-ci-robot requested review from cxie and SimFG August 3, 2023 20:11
@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ishaan-jaff
To complete the pull request process, please assign simfg after the PR has been reviewed.
You can assign the PR to them by writing /assign @simfg in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot
Copy link
Collaborator

Welcome @ishaan-jaff! It looks like this is your first PR to zilliztech/GPTCache 🎉

@ishaan-jaff
Copy link
Author

ishaan-jaff commented Aug 3, 2023

@cxie @SimFG @xiaofan-luan saw adding new LLM models was on the roadmap, made a PR to integrate with liteLLM which adds new LLM models. if this looks good would love to add support to GPTCache

@SimFG
Copy link
Collaborator

SimFG commented Aug 4, 2023

@ishaan-jaff thanks for your pr, please make the dev branch as the target

@SimFG SimFG changed the base branch from main to dev August 4, 2023 02:03
@krrishdholakia
Copy link

what would next steps on this PR be? @ishaan-jaff @SimFG

@SimFG
Copy link
Collaborator

SimFG commented Aug 7, 2023

@ishaan-jaff please add some test cases for your pull request

@SimFG SimFG self-assigned this Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants