Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Versioning3 proto updates for data plane APIs #463

Merged
merged 3 commits into from
Oct 11, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 37 additions & 3 deletions openapi/openapiv2.json
Original file line number Diff line number Diff line change
Expand Up @@ -11449,6 +11449,16 @@
},
"description": "VersionInfo contains details about current and recommended release versions as well as alerts and upgrade instructions."
},
"v1VersioningBehavior": {
"type": "string",
"enum": [
"VERSIONING_BEHAVIOR_UNSPECIFIED",
"VERSIONING_BEHAVIOR_PINNED",
"VERSIONING_BEHAVIOR_AUTO_UPGRADE"
],
"default": "VERSIONING_BEHAVIOR_UNSPECIFIED",
"description": " - VERSIONING_BEHAVIOR_PINNED: Workflow should be pinned to the current build ID until manually moved.\n - VERSIONING_BEHAVIOR_AUTO_UPGRADE: Workflow automatically moves to the latest version (default build ID of the task queue) when the next\ntask is dispatched."
},
"v1WaitPolicy": {
"type": "object",
"properties": {
Expand All @@ -11469,9 +11479,13 @@
"useVersioning": {
"type": "boolean",
"description": "If set, the worker is opting in to worker versioning, and wishes to only receive appropriate\ntasks."
},
"deploymentName": {
"type": "string",
"description": "Must be sent if user has set a deployment name (versioning-3)."
}
},
"description": "Identifies the version(s) that a worker is compatible with when polling or identifying itself,\nand whether or not this worker is opting into the build-id based versioning feature. This is\nused by matching to determine which workers ought to receive what tasks."
"description": "Identifies the version that a worker is compatible with when polling or identifying itself,\nand whether or not this worker is opting into the build-id based versioning feature. This is\nused by matching to determine which workers ought to receive what tasks."
},
"v1WorkerVersionStamp": {
"type": "object",
Expand All @@ -11483,9 +11497,13 @@
"useVersioning": {
"type": "boolean",
"description": "If set, the worker is opting in to worker versioning. Otherwise, this is used only as a\nmarker for workflow reset points and the BuildIDs search attribute."
},
"deploymentName": {
"type": "string",
"description": "Must be sent if user has set a deployment name (versioning-3)."
}
},
"title": "Identifies the version(s) of a worker that processed a task"
"title": "Identifies the version of a worker that received or processed a task"
},
"v1WorkflowExecution": {
"type": "object",
Expand Down Expand Up @@ -11731,12 +11749,24 @@
},
"assignedBuildId": {
"type": "string",
"description": "The currently assigned build ID for this execution. Presence of this value means worker versioning is used\nfor this execution. Assigned build ID is selected based on Worker Versioning Assignment Rules\nwhen the first workflow task of the execution is scheduled. If the first workflow task fails and is scheduled\nagain, the assigned build ID may change according to the latest versioning rules.\nAssigned build ID can also change in the middle of a execution if Compatible Redirect Rules are applied to\nthis execution."
"title": "The currently assigned build ID for this execution. Presence of this value means worker versioning is used\nfor this execution. Assigned build ID is selected based on Worker Versioning Assignment Rules\nwhen the first workflow task of the execution is scheduled. If the first workflow task fails and is scheduled\nagain, the assigned build ID may change according to the latest versioning rules.\nAssigned build ID can also change in the middle of a execution if Compatible Redirect Rules are applied to\nthis execution.\nDeprecated. This field should be cleaned up when versioning-2 API is removed. [cleanup-experimental-wv]"
},
"inheritedBuildId": {
"type": "string",
"description": "Build ID inherited from a previous/parent execution. If present, assigned_build_id will be set to this, instead\nof using the assignment rules."
},
"buildId": {
"type": "string",
"title": "Current build ID of this workflow execution that determines the matching queue that new tasks of this execution\nwill be scheduled in. Current build ID is set after completion of the first workflow task, and\nwhen workflow moves to a different build ID either automatically or manually. (versioning-3)"
},
"deploymentName": {
"type": "string",
"description": "The deployment name associated with this workflow's task queue + build ID. This is updated when `build_id` is\nupdated. `deployment_name` should be set if `build_id` field is set."
},
"versioningBehavior": {
"$ref": "#/definitions/v1VersioningBehavior",
"description": "Determines how server should treat this execution when workers are upgraded. `versioning_behavior` has to be set\nif the `build_id` field is set."
},
"firstRunId": {
"type": "string",
"title": "The first run ID in the execution chain.\nExecutions created via the following operations are considered to be in the same chain\n- ContinueAsNew\n- Workflow Retry\n- Workflow Reset\n- Cron Schedule"
Expand Down Expand Up @@ -12148,6 +12178,10 @@
"$ref": "#/definitions/v1WorkerVersionStamp",
"title": "Version info of the worker who processed this workflow task. If present, the `build_id` field\nwithin is also used as `binary_checksum`, which may be omitted in that case (it may also be\npopulated to preserve compatibility).\nDeprecated. Use the info inside the corresponding WorkflowTaskStartedEvent"
},
"versioningBehavior": {
"$ref": "#/definitions/v1VersioningBehavior",
"description": "Versioning behavior sent by SDK for this workflow execution. Empty means behavior is not\nchanged in this event."
},
"sdkMetadata": {
"$ref": "#/definitions/v1WorkflowTaskCompletedMetadata",
"description": "Data the SDK wishes to record for itself, but server need not interpret, and does not\ndirectly impact workflow state."
Expand Down
42 changes: 40 additions & 2 deletions openapi/openapiv3.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8917,8 +8917,11 @@ components:
description: |-
If set, the worker is opting in to worker versioning, and wishes to only receive appropriate
tasks.
deploymentName:
type: string
description: Must be sent if user has set a deployment name (versioning-3).
description: |-
Identifies the version(s) that a worker is compatible with when polling or identifying itself,
Identifies the version that a worker is compatible with when polling or identifying itself,
and whether or not this worker is opting into the build-id based versioning feature. This is
used by matching to determine which workers ought to receive what tasks.
WorkerVersionStamp:
Expand All @@ -8934,7 +8937,10 @@ components:
description: |-
If set, the worker is opting in to worker versioning. Otherwise, this is used only as a
marker for workflow reset points and the BuildIDs search attribute.
description: Identifies the version(s) of a worker that processed a task
deploymentName:
type: string
description: Must be sent if user has set a deployment name (versioning-3).
description: Identifies the version of a worker that received or processed a task
WorkflowEvent_EventReference:
type: object
properties:
Expand Down Expand Up @@ -9234,11 +9240,33 @@ components:
again, the assigned build ID may change according to the latest versioning rules.
Assigned build ID can also change in the middle of a execution if Compatible Redirect Rules are applied to
this execution.
Deprecated. This field should be cleaned up when versioning-2 API is removed. [cleanup-experimental-wv]
inheritedBuildId:
type: string
description: |-
Build ID inherited from a previous/parent execution. If present, assigned_build_id will be set to this, instead
of using the assignment rules.
buildId:
type: string
description: |-
Current build ID of this workflow execution that determines the matching queue that new tasks of this execution
will be scheduled in. Current build ID is set after completion of the first workflow task, and
when workflow moves to a different build ID either automatically or manually. (versioning-3)
deploymentName:
type: string
description: |-
The deployment name associated with this workflow's task queue + build ID. This is updated when `build_id` is
updated. `deployment_name` should be set if `build_id` field is set.
versioningBehavior:
enum:
- VERSIONING_BEHAVIOR_UNSPECIFIED
- VERSIONING_BEHAVIOR_PINNED
- VERSIONING_BEHAVIOR_AUTO_UPGRADE
type: string
description: |-
Determines how server should treat this execution when workers are upgraded. `versioning_behavior` has to be set
if the `build_id` field is set.
format: enum
firstRunId:
type: string
description: |-
Expand Down Expand Up @@ -9597,6 +9625,16 @@ components:
within is also used as `binary_checksum`, which may be omitted in that case (it may also be
populated to preserve compatibility).
Deprecated. Use the info inside the corresponding WorkflowTaskStartedEvent
versioningBehavior:
enum:
- VERSIONING_BEHAVIOR_UNSPECIFIED
- VERSIONING_BEHAVIOR_PINNED
- VERSIONING_BEHAVIOR_AUTO_UPGRADE
type: string
description: |-
Versioning behavior sent by SDK for this workflow execution. Empty means behavior is not
changed in this event.
format: enum
sdkMetadata:
allOf:
- $ref: '#/components/schemas/WorkflowTaskCompletedMetadata'
Expand Down
10 changes: 8 additions & 2 deletions temporal/api/common/v1/message.proto
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ message MeteringMetadata {
uint32 nonfirst_local_activity_execution_attempts = 13;
}

// Identifies the version(s) of a worker that processed a task
// Identifies the version of a worker that received or processed a task
message WorkerVersionStamp {
// An opaque whole-worker identifier. Replaces the deprecated `binary_checksum` field when this
// message is included in requests which previously used that.
Expand All @@ -133,10 +133,13 @@ message WorkerVersionStamp {
// marker for workflow reset points and the BuildIDs search attribute.
bool use_versioning = 3;

// Must be sent if user has set a deployment name (versioning-3).
string deployment_name = 4;

// Later, may include bundle id that could be used for WASM and/or JS dynamically loadable bundles.
}

// Identifies the version(s) that a worker is compatible with when polling or identifying itself,
// Identifies the version that a worker is compatible with when polling or identifying itself,
// and whether or not this worker is opting into the build-id based versioning feature. This is
// used by matching to determine which workers ought to receive what tasks.
message WorkerVersionCapabilities {
Expand All @@ -147,6 +150,9 @@ message WorkerVersionCapabilities {
// tasks.
bool use_versioning = 2;

// Must be sent if user has set a deployment name (versioning-3).
string deployment_name = 3;

// Later, may include info like "I can process WASM and/or JS bundles"
}

Expand Down
11 changes: 10 additions & 1 deletion temporal/api/enums/v1/workflow.proto
Original file line number Diff line number Diff line change
Expand Up @@ -137,4 +137,13 @@ enum TimeoutType {
TIMEOUT_TYPE_SCHEDULE_TO_START = 2;
TIMEOUT_TYPE_SCHEDULE_TO_CLOSE = 3;
TIMEOUT_TYPE_HEARTBEAT = 4;
}
}

enum VersioningBehavior {
VERSIONING_BEHAVIOR_UNSPECIFIED = 0;
// Workflow should be pinned to the current build ID until manually moved.
VERSIONING_BEHAVIOR_PINNED = 1;
// Workflow automatically moves to the latest version (default build ID of the task queue) when the next
// task is dispatched.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Auto-upgrade" behavior seemed a better name than "unpinned". But we can change the name later when namings are final. cc @drewhoskins-temporal @antlai-temporal

VERSIONING_BEHAVIOR_AUTO_UPGRADE = 2;
}
4 changes: 4 additions & 0 deletions temporal/api/history/v1/message.proto
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,10 @@ message WorkflowTaskCompletedEventAttributes {
// populated to preserve compatibility).
// Deprecated. Use the info inside the corresponding WorkflowTaskStartedEvent
temporal.api.common.v1.WorkerVersionStamp worker_version = 5;
// Versioning behavior sent by SDK for this workflow execution. Empty means behavior is not
// changed in this event.
temporal.api.enums.v1.VersioningBehavior versioning_behavior = 7;

// Data the SDK wishes to record for itself, but server need not interpret, and does not
// directly impact workflow state.
temporal.api.sdk.v1.WorkflowTaskCompletedMetadata sdk_metadata = 6;
Expand Down
12 changes: 12 additions & 0 deletions temporal/api/workflow/v1/message.proto
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,22 @@ message WorkflowExecutionInfo {
// again, the assigned build ID may change according to the latest versioning rules.
// Assigned build ID can also change in the middle of a execution if Compatible Redirect Rules are applied to
// this execution.
// Deprecated. This field should be cleaned up when versioning-2 API is removed. [cleanup-experimental-wv]
string assigned_build_id = 19;
// Build ID inherited from a previous/parent execution. If present, assigned_build_id will be set to this, instead
// of using the assignment rules.
string inherited_build_id = 20;
// Current build ID of this workflow execution that determines the matching queue that new tasks of this execution
// will be scheduled in. Current build ID is set after completion of the first workflow task, and
// when workflow moves to a different build ID either automatically or manually. (versioning-3)
string build_id = 22;
// The deployment name associated with this workflow's task queue + build ID. This is updated when `build_id` is
// updated. `deployment_name` should be set if `build_id` field is set.
string deployment_name = 23;
// Determines how server should treat this execution when workers are upgraded. `versioning_behavior` has to be set
// if the `build_id` field is set.
temporal.api.enums.v1.VersioningBehavior versioning_behavior = 24;

// The first run ID in the execution chain.
// Executions created via the following operations are considered to be in the same chain
// - ContinueAsNew
Expand Down
9 changes: 9 additions & 0 deletions temporal/api/workflowservice/v1/request_response.proto
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,15 @@ message RespondWorkflowTaskCompletedRequest {
// always be set by SDKs. Workers opting into versioning will also set the `use_versioning`
// field to true. See message docstrings for more.
temporal.api.common.v1.WorkerVersionStamp worker_version_stamp = 10;
// Versioning behavior explicitly specified by user for this workflow type.
// If absent, `default_versioning_behavior` will be used.
// At least one of `versioning_behavior` or `default_versioning_behavior` should be set when
// `worker_version_stamp.use_versioning` is `true` and `worker_version_stamp.deployment_name` is
// provided (which means versioning-3 API is used).
temporal.api.enums.v1.VersioningBehavior versioning_behavior = 14;
// Default versioning behavior that is set at worker server level.
temporal.api.enums.v1.VersioningBehavior default_versioning_behavior = 15;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Asking SDK to send both explicit and default annotations so we can have visibility on what/how users set these.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Default versioning behavior that is set at worker server level.
// Protocol messages piggybacking on a WFT as a transport
repeated temporal.api.protocol.v1.Message messages = 11;
// Data the SDK wishes to record for itself, but server need not interpret, and does not
Expand Down
Loading