You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This keeps Executor code clean, especially for serving models that can't benefit from working on batches of documents at the same time.
Parameters can be described as Pydantic models (#6001)
An Executor's parameters argument can now be a Pydantic model rather than a plain Python dictionary. To use a Pydantic model, the parameters argument needs to have the model as a type annotation.
Defining parameters as a Pydantic model instead of a simple Dict has two main benefits:
Validation and default values: You can get validation of the parameters that the Executor expected before the Executor may access any invalid key. You can also easily define defaults.
Descriptive OpenAPI definition when using the HTTP protocol.
Expose richer OpenAPI when serving Executor with HTTP inside a Flow (#5992)
Executors served with Deployments and Flows can now provide a descriptive OpenAPI when using HTTP. The description, examples and other relevant fields are used in the Gateway to provide a complete API.
Support streaming in single-Executor Flows (#5988)
Streaming endpoints now also support the Flow orchestration layer and it is no longer mandatory to use just a Deployment.
A Flow orchestration can accept streaming endpoints under both the gRPC and HTTP protocols.
After adding SSE support to allow streaming documents one by one for the HTTP protocol, we added the same functionality for the gRPC protocol. A Jina server can now stream single Documents to a client, one at a time, using gRPC. This feature relies on streaming gRPC endpoints under the hood.
One typical use-case of this feature is streaming tokens from a Large Language Model. For instance, check our how-to on streaming LLM tokens.
fromjinaimportExecutor, requests, DeploymentfromdocarrayimportBaseDoc# first define schemasclassMyDocument(BaseDoc):
text: str# then define the ExecutorclassMyExecutor(Executor):
@requests(on='/hello')asyncdeftask(self, doc: MyDocument, **kwargs) ->MyDocument:
foriinrange(100):
yieldMyDocument(text=f'hello world {i}')
withDeployment(
uses=MyExecutor,
port=12345,
protocol='grpc', # or 'grpc'
) asdep:
dep.block()
From the client side, you can use the new stream_doc() method to receive documents one by one:
Fix caching models from all endpoints, inputs and outputs (#6005)
An issue was fixed that caused problems when using an Executor inside a Flow where the same document type was used as input and output in different endpoints.
The orchestration layer will use 127.0.0.1 to send health checks to Executors and Gateways when working locally. It previously used 0.0.0.0 as the default host and this caused issues in some configurations.
Release Note
This release contains 5 new features 3 bug fixes and 8 documentation improvements.
🆕 Features
Executor can work on single documents (#5991)
Executors no longer need to work solely on a
DocList
, but can expose endpoints for working on single documents.For this, the method decorated by
requests
must take adoc
argument and an annotation for the input and output types.This keeps Executor code clean, especially for serving models that can't benefit from working on batches of documents at the same time.
Parameters can be described as Pydantic models (#6001)
An Executor's
parameters
argument can now be a Pydantic model rather than a plain Python dictionary. To use a Pydantic model, theparameters
argument needs to have the model as a type annotation.Defining
parameters
as a Pydantic model instead of a simple Dict has two main benefits:Expose richer OpenAPI when serving Executor with HTTP inside a Flow (#5992)
Executors served with Deployments and Flows can now provide a descriptive OpenAPI when using HTTP. The description, examples and other relevant fields are used in the Gateway to provide a complete API.
Support streaming in single-Executor Flows (#5988)
Streaming endpoints now also support the Flow orchestration layer and it is no longer mandatory to use just a Deployment.
A Flow orchestration can accept streaming endpoints under both the gRPC and HTTP protocols.
Streaming endpoints with gRPC protocol (#5921)
After adding SSE support to allow streaming documents one by one for the HTTP protocol, we added the same functionality for the gRPC protocol. A Jina server can now stream single Documents to a client, one at a time, using gRPC. This feature relies on streaming gRPC endpoints under the hood.
One typical use-case of this feature is streaming tokens from a Large Language Model. For instance, check our how-to on streaming LLM tokens.
From the client side, you can use the new
stream_doc()
method to receive documents one by one:Read more in the docs.
🐞 Bug Fixes
Fix caching models from all endpoints, inputs and outputs (#6005)
An issue was fixed that caused problems when using an Executor inside a Flow where the same document type was used as input and output in different endpoints.
Use
127.0.0.1
as local ctrl address (#6004)The orchestration layer will use
127.0.0.1
to send health checks to Executors and Gateways when working locally. It previously used0.0.0.0
as the default host and this caused issues in some configurations.Ignore warnings from Google (#5968)
Warnings that used to appear in relation to the
pkg_resources
deprecated API are now suppressed.📗 Documentation Improvements
🤟 Contributors
We would like to thank all contributors to this release:
The text was updated successfully, but these errors were encountered: