New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

支持多个不同模型同时推理吗 #6

Open

GitHub-LTQ opened this issue Apr 16, 2024 · 1 comment

GitHub-LTQ commented Apr 16, 2024

大佬，我想写一个接口，根据传过来的模型文件或者标识去推理，可以实现多线程方式去同时推理不

Owner

ben0oil1 commented Apr 16, 2024

这个取决于你的硬件，CPU多线程调度问题不大，实现相对简单。有多个gpu调度的也可以

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment