Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BOUNTY - $200] make it so process_prompt is cancellable outside of a download (follow up to #305) #307

Open
AlexCheema opened this issue Oct 7, 2024 · 1 comment

Comments

@AlexCheema
Copy link
Contributor

See #305

Currently the fix is to shield process_prompt however this also makes it so that if a chatgpt api request times out, it never gets cancelled even when a download is not in progress.

This can lead to behaviour where exo keeps generating even after a chatgpt api request has timed out.

Gotchas: make sure however exo works right is actually cancellable end-to-end. If an inference is in progress, that needs to be cancellable - InferenceEngines need to support this behaviour.

@ji-cryptocafe
Copy link

Started working on this. But since holidays are near, it could take a few days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants