nix run .#jupyter-all generates errors that make it unusable

[teto]

Describe the bug

I've had issues with jupyenv (infinite recursion, solved by tweag/jupyenv#524) and while trying to solve it thought I would use the nixpkgs infra added in #247428

[I 2023-11-01 17:28:35.789 ServerApp] Package notebook took 0.0000s to import
[I 2023-11-01 17:28:35.793 ServerApp] notebook | extension was successfully linked.
[I 2023-11-01 17:28:35.809 ServerApp] notebook | extension was successfully loaded.
[I 2023-11-01 17:28:35.810 ServerApp] The port 8888 is already in use, trying another port.
[I 2023-11-01 17:28:35.810 ServerApp] Serving notebooks from local directory: /home/teto/nixpkgs
[I 2023-11-01 17:28:35.810 ServerApp] Jupyter Server 2.7.3 is running at:
[I 2023-11-01 17:28:35.810 ServerApp] http://localhost:8889/tree?token=0fe1db1eb32be23ee18bf23e00e25e80d3116098fc8447ae
[I 2023-11-01 17:28:35.810 ServerApp]     http://127.0.0.1:8889/tree?token=0fe1db1eb32be23ee18bf23e00e25e80d3116098fc8447ae
[I 2023-11-01 17:28:35.811 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2023-11-01 17:28:35.863 ServerApp] 
    
    To access the server, open this file in a browser:
        file:///home/teto/.local/share/jupyter/runtime/jpserver-296526-open.html
    Or copy and paste one of these URLs:
        http://localhost:8889/tree?token=0fe1db1eb32be23ee18bf23e00e25e80d3116098fc8447ae
        http://127.0.0.1:8889/tree?token=0fe1db1eb32be23ee18bf23e00e25e80d3116098fc8447ae
/nix/store/18636203zf1d2s147gfwc4pxcqxmw3v7-python3.11-ipython-8.15.0/lib/python3.11/site-packages/IPython/paths.py:69: UserWarning: IPython parent '/' is not a writable location, using a temp directory.
  warn("IPython parent '{0}' is not a writable location,"
[W 2023-11-01 17:28:38.294 JupyterNotebookApp] Missing or misshapen translation settings schema:
    HTTP 404: Not Found (Schema not found: /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas/@jupyterlab/translation-extension/plugin.json)
[W 2023-11-01 17:28:38.294 JupyterNotebookApp] Settings directory does not exist at /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas
[W 2023-11-01 17:28:38.497 JupyterNotebookApp] Missing or misshapen translation settings schema:
    HTTP 404: Not Found (Schema not found: /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas/@jupyterlab/translation-extension/plugin.json)
[W 2023-11-01 17:28:38.498 ServerApp] 404 GET /lab/api/settings/@jupyter-notebook/application-extension:shell?1698856118476 (127.0.0.1): Schema not found: /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas/@jupyter-notebook/application-extension/shell.json
[W 2023-11-01 17:28:38.498 JupyterNotebookApp] wrote error: 'Schema not found: /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas/@jupyter-notebook/application-extension/shell.json'
    Traceback (most recent call last):
      File "/nix/store/r2xgr8bhsqp2kb7srw114k2s6wxgl0fh-python3.11-tornado-6.3.3/lib/python3.11/site-packages/tornado/web.py", line 1784, in _execute
        result = method(*self.path_args, **self.path_kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/nix/store/r2xgr8bhsqp2kb7srw114k2s6wxgl0fh-python3.11-tornado-6.3.3/lib/python3.11/site-packages/tornado/web.py", line 3290, in wrapper
        return method(self, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/nix/store/hf0mxpbjfrnnc0inbcgja5qmms0qzac3-python3.11-jupyterlab_server-2.25.0/lib/python3.11/site-packages/jupyterlab_server/settings_handler.py", line 48, in get
        result, warnings = get_settings(
                           ^^^^^^^^^^^^^
      File "/nix/store/hf0mxpbjfrnnc0inbcgja5qmms0qzac3-python3.11-jupyterlab_server-2.25.0/lib/python3.11/site-packages/jupyterlab_server/settings_utils.py", line 370, in get_settings
        schema, version = _get_schema(schemas_dir, schema_name, overrides, labextensions_path)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/nix/store/hf0mxpbjfrnnc0inbcgja5qmms0qzac3-python3.11-jupyterlab_server-2.25.0/lib/python3.11/site-packages/jupyterlab_server/settings_utils.py", line 47, in _get_schema
        raise web.HTTPError(404, notfound_error % path)
    tornado.web.HTTPError: HTTP 404: Not Found (Schema not found: /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas/@jupyter-notebook/application-extension/shell.json)
[W 2023-11-01 17:28:38.500 JupyterNotebookApp] 404 GET /lab/api/settings/@jupyter-notebook/application-extension:shell?1698856118476 ([email protected]) 9.39ms referer=http://localhost:8889/tree
[W 2023-11-01 17:28:38.500 JupyterNotebookApp] Missing or misshapen translation settings schema:
    HTTP 404: Not Found (Schema not found: /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas/@jupyterlab/translation-extension/plugin.json)
[W 2023-11-01 17:28:38.501 ServerApp] 404 GET /lab/api/settings/@jupyterlab/translation-extension:plugin?1698856118476 (127.0.0.1): Schema not found: /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/share/jupyter/lab/schemas/@jupyterlab/translation-extension/plugin.json

Steps To Reproduce

nix run .#jupyter-all

Expected behavior

Be able to use jupyter with the specified kernels.

Screenshots

As jupyter appears in firefox :/

Additional context

It looks like we have a good basis in nixpkgs to build a local jupyenv.
I am working on ihaskell improvements and I am not sure how to best test those but if I could do with native nixpkgs, that would be better than relying on jupyenv (maintained by the external organization Tweag).

Notify maintainers

@thomasjm

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

nixos-unstable 0cbe9f69c234a7700596e943bfae7ef27a31b735

[thomasjm]

Jupyter issue here: jupyter/notebook#6974

Incidentally, it seems really hard to git bisect on Nixpkgs master branch because it has so many merge commits, even with --first-parent. I gave up with 10 steps remaining, after needing to git bisect skip like 15 times on some commit where jupyter-all doesn't exist. Is there some trick I'm missing?

[thomasjm]

As a workaround until this gets fixed, maybe you can use jupyter-console?

nix run --impure --expr 'with import ./. {}; jupyter-console.withSingleKernel your-kernel-definition'

[natsukium]

related to #255923

[thomasjm]

Any objection to closing this in favor of #255923?

[teto]

@thomasjm I am fine with closing this. I am just curious about your commitment to the jupyter env in nixpkgs ? I dont have the capacity to lead such an effort but willing to help anyone interested in this. thanks for linking the issue @natsukium ;)

[thomasjm]

I am just curious about your commitment to the jupyter env in nixpkgs ?

I'm very interested in working on it. The only thing holding me back is that I've had a hard time getting my PRs reviewed in a timely manner. For example:

• xeus-cling: init at 0.15.3 #244777
• coq_kernel: init at 1.6.0 #171202

If you could help me merge some stuff @teto then we could do great things :)

[GTrunSec]

@teto I will test the mentioned packages tomorrow once everything is okay, you can merge them anytime.

[teto]

@thomasjm cool ! let me know when you need help merging sthg. I will defer the reviews to@GTrunSec then who is knowledgeable about jupyenv I've added a "jupyter" label to help with the effort, someone knowledgeable could autolabel PRs via https://github.com/NixOS/nixpkgs/blob/master/.github/labeler.yml. There is also a jupyter team here

nixpkgs/maintainers/team-list.nix

Line 439 in 8f6ee3a

jupyter = {