You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to have a fast service for inference, we need to implement a few optimisations on top of Candle. This will include an implementation of paged attention (see here for an existing implementation using Candle already).
The text was updated successfully, but these errors were encountered:
In order to have a fast service for inference, we need to implement a few optimisations on top of Candle. This will include an implementation of paged attention (see here for an existing implementation using Candle already).
The text was updated successfully, but these errors were encountered: