[1.x] Package input files (no autodownload, no multiprocessing DownloadManager) (#510) #511
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rver#425) * modify the commit for 1.x * Fix CI, run lint, reduce end-to-end tests flakiness (jupyter-server#417) * add macos tests * add locally-generated snapshots * check if filebrowser is already open before opening it * add actual snapshoy from CI * Update Playwright Snapshots * use notebook toolbar, mask bttns on the right * correct selector * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * run lint * Update Playwright Snapshots * timestampLocator -> timestamp * wait until fonts are ready before taking a snapshot * add maxDiffPixels * Revert "wait until fonts are ready before taking a snapshot" This reverts commit 9554559. * pin jupyterlab version for e2e tests and snapshots * remove macos tests * rename relevant job to end-to-end test * remove darwin/macos snapshots * specify jupyterlab version correctly * run lint * add jlpm install to Lint the extension step * remove ubuntu postfix from test report * Make e2e tests a separate file * add build steps * pin version of jupyterlab in e2e test file * introduce modifyListResponse, remove maxDiffPixels * modifyListResponse -> setJobList * set expected URL * always set create_time to 1 * update list view snapshot * run check option for lint * run lint locally --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix install chromium command * remove playwrght from dev dependencies * fix comma * Roll back 402 changes * roll back changes to build.yml * run lint * update snapshots * remove integration step * reinstate d423ede --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Archiving all-files scheduler (jupyter-server#388) * Fix typo in comment * WIP: Adds new scheduler * writes individual files * WIP: Write zip file * WIP: Trying to get zip file to be written only on scheduled job runs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * WIP: Removes zip type, incremental work for archiving work dir * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Create tar.gz in staging subdir * Capture side effect files in staging dir * Extracts files * Add filter * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update jupyter_scheduler/job_files_manager.py Co-authored-by: david qiu <[email protected]> * Simplifies cleanup logic * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updates docs, deletes old Archiving*, renames AllFilesArchiving --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: david qiu <[email protected]> * Avoids option compatible only with Python 3.11 * Fix JFM tests (jupyter-server#424) * fix JFM tests * pre-commit * add minor comment --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: david qiu <[email protected]>
* fix JFM tests * pre-commit * add minor comment
SHA256 hashes: jupyter_scheduler-1.4.0-py3-none-any.whl: dfa259225fa6f75e3eb33bab0d0ddc6a87dc2deb847e349ba809e7e070dedb75 jupyter_scheduler-1.4.0.tar.gz: 64d9123b73a5f42efc7ab7ea33209c68644e7785cde7a383595c2ee4dd158822 jupyterlab-scheduler-1.4.0.tgz: 5eb905b49175c626a52162eea7ca93c3a1d3b39f5678f7672f71b7c52045fdcd
* Add readthedocs configuration file v2 * Update per 101458a
SHA256 hashes: jupyter_scheduler-1.5.0-py3-none-any.whl: 81949bb8697c3b41f95c9132b8efa48c6c3ebc972d276a5decea5c656c6d56a8 jupyter_scheduler-1.5.0.tar.gz: 9f9d314639737cdceb2e18d4da19a3a357c28a153e1b18d1623b0c54b1261fb0 jupyterlab-scheduler-1.5.0.tgz: 42b6a618048ff2ff3b0b949983f4675f8f44657b4cbee15e6e81a4210237e3ab
…r#454) * fix event loop is already running bug on Linux * Update Playwright Snapshots --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Added telemetry support. * Fixed lint errors * Refactored to use a logger hook. * Corrected the handler and token names. * Corrected the plugin id to match the token. * Updated names for hook and handler
* Moved event logger handler up the stack. * Fixed lint errors * Combined handler, and plugin. * Fixed lint errors * separate launcher command and rename show-notebook-jobs (jupyter-server#3) --------- Co-authored-by: david qiu <[email protected]>
SHA256 hashes: jupyter_scheduler-1.6.0-py3-none-any.whl: 66177a3c980dbbdfc916bcc32051705c5d7073b97186e7737aa04b4bd809c205 jupyter_scheduler-1.6.0.tar.gz: 88f304b0fb33f7994373a7e33e5c514d94819e9775a0acd1d81c6159011e97ea jupyterlab-scheduler-1.6.0.tgz: b065c2a43a1f0720d75ec19639fef8effb61ea2b753635a933b4e89c356787cb
…erver#463) (jupyter-server#465) * Custom module to allow Pydantic v1 and v2 compatibility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update jupyter_scheduler/pydantic_v1/__init__.py * Removes unneeded logic * Adds dataclasses.py --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: david qiu <[email protected]>
SHA256 hashes: jupyter_scheduler-1.7.0-py3-none-any.whl: 62ca118aab465b3146cf02bed8eeb3e402b3b1ca3e29a80f6d55f73e79151b41 jupyter_scheduler-1.7.0.tar.gz: 5600d4db008ac82d4102f0bfea9960e84e7eff2ab4197ebe7440abebbf945c1b jupyterlab-scheduler-1.7.0.tgz: d5da54cde4be6bcb8ac77dc3c1de301687513590550e206486dc1bf97842beac
…e Job Definition, Create Job from Job Definition hooks (jupyter-server#472) (jupyter-server#475) * add job/job definition creation event success or failure logging * fix create-job logging format * log error message and httpStatusCode when error occurs * Update src/index.tsx * Update src/context.ts * Update src/context.ts * log string for logDetails, not whole error * catch errors as unknown, not Error * Make generic error message more user-frinedly and actionable per @JasonWeill --------- Co-authored-by: Piyush Jain <[email protected]>
SHA256 hashes: jupyter_scheduler-1.8.0-py3-none-any.whl: e79c6556e74820cd665993e2885642fb3b719f16200994c4072b69321e8c4aef jupyter_scheduler-1.8.0.tar.gz: 03e7f2271a3b0d4985c3837519b05ecdae44232b1f4740b2fa95f06616f1a069 jupyterlab-scheduler-1.8.0.tgz: 1dcb38f3ad69ae7f0a360112ce23041cd695f0476383b083c98eaa558c29149e
…p non-blocking (jupyter-server#480) (jupyter-server#481) * Make server extension verification call during extension startup non-blocking (jupyter-server#480) * remove async call, make call sync, refactor into a separate function * don't use abbreviations in the verifyServerExtension function or type naming * Update src/index.tsx Co-authored-by: david qiu <[email protected]> * remove unused type * add @jupyterlab/rendermime-interfaces as a dependency * Remove async from activatePlugin function declaration (makes it not async) * set @jupyterlab/rendermime-interfaces version to ^3.8.0 to support all JLab >= 4 --------- Co-authored-by: david qiu <[email protected]> * Fix translator usage, remove @jupyterlab/rendermime-interfaces dependency (jupyter-server#483) * update snapshots * use lowercase "es" is "es2017" as is convention in Project Jupyter * fix 1.x dependencies * fix package manifest to be compatible with JL3 * Make ErrorBoundary return JSX.Element to avoid type conflicts --------- Co-authored-by: david qiu <[email protected]>
SHA256 hashes: jupyter_scheduler-1.8.1-py3-none-any.whl: c7a44b5935f375247bba44e1430ca1c9c773d54be83f53bb9f78fa033adac52d jupyter_scheduler-1.8.1.tar.gz: b14b5953f2d4dd49cb6173c2258d1d06119a8d876f2c84c6d36306af3cfa3096 jupyterlab-scheduler-1.8.1.tgz: e195bc445e73847f0b0d3a013b3af066fa271a08bb7c50d0cbf137ed7cf4f714
…ns (jupyter-server#496) (jupyter-server#501) * Changed column header to "Input file" in Notebook Job Definitions (jupyter-server#496) * Changed column header to Input file in Notebook Job Definitions * Update list-jobs.tsx for column header change in Notebook Jobs table Changed the column headers for the inout file in both Notebook Jobs and Notebook Job Definitions to 'Input filename" - we choose “filename” over “file” is because we don’t show the absolute or relative path, just the name of the file in any directory. * Updated E2E snapshots for test * Update Playwright Snapshots --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
SHA256 hashes: jupyter_scheduler-1.8.2-py3-none-any.whl: bcf1459627b7221096d42283db9320b2afdfb423f8f8f1d5c4302568898beb09 jupyter_scheduler-1.8.2.tar.gz: 334f736eadcf1904c343df3463708b96e5bb7d9dd11c1fd9089f6d19c4ec09c0 jupyterlab-scheduler-1.8.2.tgz: 174e646cd3645670eba06433e6dbc35145ba6ddb2a1b0d0644c62f3a35822edf
…ger) (jupyter-server#510) * package input files and folders (backend) * package input files and folders (frontend) * remove "input_dir" from staging_paths dict * ensure execution context matches the notebook directory * update snapshots * copy staging folder to output folder after job runs (SUCESS or FAILURE) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * copy staging folder and side effects to output after job runs, track and redownload files * remove staging to output copying logic from executor * refactor output files creation logic into a separate function for clarity * Fix job definition data model * add packaged_files to JobDefinition and DescribeJobDefinition model * fix existing pytests * clarify FilesDirectoryLink title * Dynamically display input folder in the checkbox text * display packageInputFolder parameter as 'Files included' * use helper text with input directory for 'include files' checkbox * Update Playwright Snapshots * add test side effects accountability test for execution manager * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use "Run job with input folder" for packageInputFolder checkbox text * Update Playwright Snapshots * Use "Ran with input folder" in detail page * Update src/components/input-folder-checkbox.tsx Co-authored-by: Jason Weill <[email protected]> * fix lint error * Update Playwright Snapshots * Update existing screenshots * Update "Submit the Create Job" section mentioning “Run job with input folder” option * Update docs/users/index.md Co-authored-by: Jason Weill <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update src/components/input-folder-checkbox.tsx Co-authored-by: Jason Weill <[email protected]> * Update Playwright Snapshots * Describe side effects behavior better --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jason Weill <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports #510