Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flow with keys as argument makes all run failed #15610

Closed
ChillPC opened this issue Oct 8, 2024 · 1 comment · Fixed by #15611
Closed

Flow with keys as argument makes all run failed #15610

ChillPC opened this issue Oct 8, 2024 · 1 comment · Fixed by #15611
Labels
upstream dependency An upstream issue caused by a bug in one of our dependencies

Comments

@ChillPC
Copy link

ChillPC commented Oct 8, 2024

Bug summary

When you try to launch a deployed flow with one of its argument being keys, the flow fail with a log in the worker only being :

13:31:45.141 | ERROR   | prefect.engine - Validation of flow parameters failed with error: 'str' object is not callable
13:31:45.213 | ERROR   | prefect.engine - Finished in state Failed("Validation of flow parameters failed with error: TypeError: 'str' object is not callable")
13:31:45.214 | ERROR   | prefect.engine - Engine execution of flow run 'eaea1104-4a59-408f-925a-0d26f5b7b348' exited with unexpected exception
Traceback (most recent call last):
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/engine.py", line 40, in <module>
    run_coro_as_sync(run_flow(flow, flow_run=flow_run))
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 243, in run_coro_as_sync
    return call.result()
           ^^^^^^^^^^^^^
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 312, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 182, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 383, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 225, in coroutine_wrapper
    return await task
           ^^^^^^^^^^
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/flow_engine.py", line 717, in run_flow_async
    return engine.state if return_type == "state" else engine.result()
                                                       ^^^^^^^^^^^^^^^
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/flow_engine.py", line 255, in result
    raise self._raised
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/flow_engine.py", line 203, in begin_run
    self.parameters = self.flow.validate_parameters(self.parameters or {})
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/c/.cache/pypoetry/virtualenvs/pysae-izMesTf_-py3.11/lib/python3.11/site-packages/prefect/flows.py", line 600, in validate_parameters
    for k, v in dict(model).items()
                ^^^^^^^^^^^
TypeError: 'str' object is not callable

The reason is that in src/prefect/flows.py, function validate_parameters, dict(<a_pydantic_base_model>) is used instead of a <a_pydantic_base_model>.model_dump() :

        cast_parameters = {
            k: v
            for k, v in dict(model).items()
            if k in model.model_fields_set or model.model_fields[k].default_factory
        }

Here is a reproducible test of pydantic :

from pydantic import BaseModel

class C(BaseModel):
    a: int = 1
    keys: str

dict(C(keys="hello")) # TypeError: 'str' object is not callable
C(keys="hello").model_dump() # {'a': 1, 'keys': 'hello'}

Please see if there is other misuses of pydantic in the code :)

Version info (prefect version output)

Version:             3.0.4
API version:         0.8.4
Python version:      3.11.9
Git commit:          c068d7e2
Built:               Tue, Oct 1, 2024 11:54 AM
OS/Arch:             linux/x86_64
Profile:             local
Server type:         server
Pydantic version:    2.9.2

Additional context

No response

@ChillPC ChillPC added the bug Something isn't working label Oct 8, 2024
@zzstoatzz
Copy link
Collaborator

zzstoatzz commented Oct 8, 2024

hi @ChillPC - dict(model) is a valid use of pydantic and in this case is actually important that we use it because dict does not recursively cast submodels to dictionaries

The ultimate bug seems like its coming from pydantic, as I see no reason you can't have keys as a field name

In [3]: class D(BaseModel):
   ...:     not_keys: str
   
In [4]: dict(D(not_keys="hello"))
Out[4]: {'not_keys': 'hello'}

PR to follow

@zzstoatzz zzstoatzz added upstream dependency An upstream issue caused by a bug in one of our dependencies and removed bug Something isn't working labels Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
upstream dependency An upstream issue caused by a bug in one of our dependencies
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants