-
Notifications
You must be signed in to change notification settings - Fork 283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable running multiple reverse proxys with different targets. #54
Comments
Hi @michaelfeil , from sparrow.api import create_app
from .openai import Openai
app = create_app(title="openai_forward", version="1.0")
def add_route(obj: Openai):
app.add_route(
obj.ROUTE_PREFIX + "/{api_path:path}",
obj.reverse_proxy,
methods=["GET", "POST", "PUT", "DELETE", "OPTIONS", "HEAD", "PATCH", "TRACE"],
)
openai = Openai()
openai.BASE_URL = "https://api.openai.com"
openai.ROUTE_PREFIX = "/openai"
add_route(openai)
localai = Openai()
localai.BASE_URL = "http://localhost:8080"
localai.ROUTE_PREFIX = "/localai"
add_route(localai) Then, remove the After making these changes, you will be able to access the content of api.openai.com under the In the future, it's worth considering making |
Now we can specify multiple targets through configuration, like this: |
I have experimented running multiple proxies in the same fastapi-app, forwarding to multiple endpoints.
E.g.
/v1/audio
could be still forwarded to openai, while/v1/completions
is forwarded to this API:https://github.com/go-skynet/LocalAI
How would you suggest integrating such a feature?
I thought of refactoring ROUTE_PREFIX class OpenaiBase to make openai base configurable.
The text was updated successfully, but these errors were encountered: