Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NotImplementedError in BaseEventLoop when running git_clone pull step on Windows #14522

Closed
4 tasks done
kevingrismore opened this issue Jul 8, 2024 · 1 comment
Closed
4 tasks done
Labels
2.x arch:windows Related to the Windows OS bug Something isn't working

Comments

@kevingrismore
Copy link
Contributor

kevingrismore commented Jul 8, 2024

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I searched the Prefect documentation for this issue.
  • I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

When upgrading from 2.11.5 to 2.19.6, a Process worker running on Windows fails at the git_clone pull step during a deployment run.

From the initial report, it appears that the error is arising from the event loop base class, which has unimplemented methods, so this may be an issue of OS detection somewhere along the way.

Reproduction

import asyncio
import shutil

from prefect.runner.storage import GitRepository

async def clone():
    storage = GitRepository(url="https://github.com/kevingrismore/ecs-example.git")
    await storage.pull_code()
    shutil.rmtree("ecs-example")

if __name__ == "__main__":
    asyncio.run(clone())

Error

Traceback (most recent call last):
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\deployments\steps\core.py", line 154, in run_steps
    step_output = await run_step(step, upstream_outputs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\deployments\steps\core.py", line 125, in run_step
    result = await from_async.call_soon_in_new_thread(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 327, in aresult
    return await asyncio.wrap_future(self.future)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 352, in _run_sync
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\utilities\asyncutils.py", line 304, in coroutine_wrapper
    return call()
           ^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 432, in __call__
    return self.result()
           ^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 318, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 179, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 389, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\deployments\steps\pull.py", line 123, in git_clone
    await storage.pull_code()
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\runner\storage.py", line 233, in pull_code
    await self._clone_repo()
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\prefect\runner\storage.py", line 257, in _clone_repo
    await run_process(cmd)
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\anyio\_core\_subprocesses.py", line 62, in run_process
    async with await open_process(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\anyio\_core\_subprocesses.py", line 126, in open_process
    return await get_asynclib().open_process(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\1_CODE\prefect2\.venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 1041, in open_process
    process = await asyncio.create_subprocess_exec(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\AppData\Local\Programs\Python\Python311\Lib\asyncio\subprocess.py", line 223, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 1694, in subprocess_exec
    transport = await self._make_subprocess_transport(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\s21913\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 502, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError

Versions (prefect version output)

Version:             2.19.6
API version:         0.8.4
Python version:      3.11.7
Git commit:          9d938fe7
Built:               Mon, Jun 24, 2024 10:23 AM
OS/Arch:             win32/AMD64
Profile:             default
Server type:         cloud

Additional context

After several attempts, we haven't been able to reproduce this independently. This issue is for tracking purposes in the case that this issue can be reproduced and is subsequently resolved.

@kevingrismore kevingrismore added bug Something isn't working arch:windows Related to the Windows OS needs:triage 2.x labels Jul 8, 2024
@kevingrismore
Copy link
Contributor Author

The reporting user fixed by reinstalling python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x arch:windows Related to the Windows OS bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants