[Internal] Update Jobs GetRun API to support paginated responses for jobs and ForEach tasks with >100 runs #725
Conversation
| *, | ||
| include_history: Optional[bool] = None, | ||
| include_resolved_values: Optional[bool] = None, | ||
| page_token: Optional[str] = None) -> jobs.Run: |
There was a problem hiding this comment.
Should we have a doccomment here explaining that this makes multiple requests to return all tasks/iterations when the underlying has more than 100?
| requests_mock.get(make_path_pattern(1337, "tokenToThirdPage"), text=json.dumps(run3)) | ||
| w = WorkspaceClient(config=config) | ||
|
|
||
| run = w.jobs.get_run(1337, page_token="initialToken") |
There was a problem hiding this comment.
Will users ever specify an initial token?
There was a problem hiding this comment.
At this point users cannot get initial page token from any SDK function. So this parameter is redundant. The function that is overloaded is generated from openApi spec and it also comes with the page_token param. So I assumed that it's ok to repeat the same function contract in the mixin as well.
I see two paths:
- we hide the page_token param in the mixin and make our change opaque to the users
- we do not hide this parameter and stay consistent with the openApi spec for this spec and for the other functions that use page_token. Maybe in the future the initial token will be exposed to the users 🤷
There was a problem hiding this comment.
Discussed offline. Let's leave the parameter there so the mixin matches the underlying API. We'll clean up token parameters separately and can remove it here when we do that.
| requests_mock.get(make_path_pattern(1337, "tokenToThirdPage"), text=json.dumps(run3)) | ||
| w = WorkspaceClient(config=config) | ||
|
|
||
| run = w.jobs.get_run(1337, page_token="initialToken") |
There was a problem hiding this comment.
ditto about the initial token?
mgyucht
changed the title
mgyucht
changed the title
Changes
Jobs GetRun 2.2 behind the scenes pagination
Tests
make testrun locallymake fmtapplied