-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Return default value if the input is None
#8972
Return default value if the input is None
#8972
Comments
None
None
I think it probably makes sense to use a custom validator for this. For example: from pydantic import BaseModel, ValidationInfo, field_validator
class Foo(BaseModel):
bar: list[int] | None = [0,1,2]
baz: list[str] | None = ["a", "b", "c"]
@field_validator('bar', 'baz')
@classmethod
def check_alphanumeric(cls, v: str, info: ValidationInfo) -> str:
if v is None:
return cls.model_fields[info.field_name].default
return v
print(repr(Foo()))
# Foo(bar=[0, 1, 2], baz=['a', 'b', 'c'])
print(repr(Foo(bar=[1,2], baz=["a","b"])))
# Foo(bar=[1, 2], baz=['a', 'b'])
print(repr(Foo(bar=None, baz=None))) # Explicitly pass `None`
# Foo(bar=[0, 1, 2], baz=['a', 'b', 'c']) I haven't seen an abundance of demand for this feature. If there's significant community interest, we could reconsider making this a flag on a field, but for now I think the field validator approach (a change in user code) makes the most sense! |
That gets the job done, thank you! |
@sydney-runkle Can we do the same in BeforeValidator? I love the Annotated + Validator pattern as it provides stronger reusability. But I couldn't access the default value information in the validator function in this case. Specifically I'd like to do something like this. def validator(v: Any, info: ValidationInfo) -> Any:
print(
info
) # ValidationInfo(config={'title': 'MyClass'}, context=None, data={}, field_name='field')
if v is None:
default = get_default_somehow()
return default
return v
class MyClass(BaseModel):
field: Annotated[int, Field(default=0), BeforeValidator(validator)]
MyClass.model_validate({"field": None}) Although I can explicitly add BeforeValidator like following, writing default value twice looks a bit weird. def null_default_validator(default: Any) -> BeforeValidator:
def validator(v: Any) -> Any:
if v is None:
return default
return v
return BeforeValidator(validator)
class MyClass(BaseModel):
field: Annotated[int, Field(default=0), null_default_validator(default=0)]
MyClass.model_validate({"field": None}) |
I have to say that I would have wanted this feature. The thing is - that I already have a default value which I want this field to take in the case it is None. I want an option to configure this behavior so it'll use the default on None as well - it will give me more freedom. |
@yotaro-shimose and @tsalex1992 - Just in case you might find it helpful, our team encountered a slightly similar issue and came up with another possible solution. Sadly, it doesn't fit with the Annotated + Validator pattern but it is fairly reusable. With your # some/path/to/utils.py
def generic_validator_function_with_default(class_ref, value, info):
return (
value
if value is not None
else get_default_value_from_field_config(class_ref, info)
)
def get_default_value_from_field_config(class_ref, info):
field = class_ref.model_fields[info.field_name]
return field.default_factory() if callable(field.default_factory) else field.default
# some/path/to/models/foo.py
from some.path.to.utils import generic_validator_function_with_default
class Foo(BaseModel):
bar: list[int] | None = [0,1,2]
baz: list[str] | None = ["a", "b", "c"]
@field_validator('bar', 'baz')
@classmethod
def handle_field_default(cls, value, info):
return generic_validator_function_with_default(cls, value, info)
|
@saurookadook As your workaround says, it might be good to have reference to the class to be wrapped in function validators. Hopefully it works with python Annotated syntax so that pydantic maintainers can implement it. |
Having this syntax would be great for shared library code as well |
I'd be happy to review a PR with support for this via a field flag if the API is pretty clean, given that there seems to be a decent amount of requests for this. I think we want to do this on the field level and not the config level, but I'm not sure... |
In the API I'm dealing with, I needed it globally on every model so a model_config level would be appreciated |
Thank you @sydney-runkle |
Being able to set defaults like this would also be useful for the cases when using |
After a bit of tinkering, I arrived independently arrived at a similar solution to @saurookadook which is added to our common @field_validator("*", mode="before")
@classmethod
def not_none(cls, v, val_info):
"""
Convert None to Default on optional fields.
Why doesn't Pydantic have this option?
"""
field = cls.__fields__[val_info.field_name]
if v is None and (default := field.get_default(call_default_factory=True)) is not None:
return default
return v This could also be broken out into a function and called with I'd still rather have a |
Here's another solution. from pydantic import BaseModel, model_validator
from typing import Optional
class TestModel(BaseModel):
name: str
age: Optional[int]=0
@model_validator(mode='before')
@classmethod
def remove_none(cls, data):
if isinstance(data, dict):
return {k: v for k, v in data.items() if v is not None}
return data
data = {"name": "test", "age": None}
model = TestModel(**data) |
See also this: #8585 |
After some studies I came to this result, it works, but it is extremely inelegant, it would be great if pydantic already did this by default, having to create this subclass is not the best thing in the world as I have to duplicate this in all my python packages, or create another python package just for this code below, either way, it's not elegant from pydantic import BaseModel
from pydantic_core import PydanticUndefinedType
class CustomBaseModel(BaseModel):
@model_validator(mode='before')
def set_defaults_for_none(cls, values):
fields = cls.model_fields
for field_name, field_info in fields.items():
value = values.get(field_name)
default_value = (
field_info.default
if not isinstance(field_info.default, PydanticUndefinedType)
else None
)
default_factory = (
field_info.default_factory
if not isinstance(
field_info.default_factory, PydanticUndefinedType
)
else None
)
if value is None:
if default_value is not None:
values[field_name] = default_value
elif default_factory is not None:
values[field_name] = default_factory()
return values |
I typically use Pydantic schema classes in GPT prompts to ensure the response is in JSON format. However, when the GPT model doesn't find a value for a field, it initializes the field with |
Semi-related, the |
Going back to the @Ravencentric's original example, IMO the current behavior is actually preferable as written because the type annotation marked the field as optional ( However I think in most cases if you have a default value to use for a field then you don't want that field to be optional. For example: class Foo(BaseModel):
bar: list[int]= [0,1,2]
Foo()
# Foo(bar=[1,2,3])
Foo(bar=None)
# raises ValidationError In that case it would be preferable if it used the default value instead of raising an error. The model_validator workaround that others mentioned does work, but that also raises the question why doesn't a mode="before" field validator work? That part is pretty unexpected that the None type validation runs before the "before" field validators but not before the "before" model validator. |
Based on the example suggested by @sydney-runkle, this seems to be working well and should contend everyone in this thread 🙂 : from pydantic import BaseModel, field_validator
from pydantic_core import PydanticUseDefault
class Foo(BaseModel):
bar: list[int] = [0,1,2]
baz: list[str] = ["a", "b", "c"]
@field_validator("*", mode="before")
@classmethod
def none_to_default(cls, v):
if v is None:
raise PydanticUseDefault()
return v
assert Foo(bar=None, baz=None) == Foo() |
Hey @sydney-runkle, I'd like to work on this. I agree with @dcosson's suggestion, so I made some local changes to pydantic-core, and this is what I have so far: from typing import Optional
from pydantic import BaseModel
class Foo(BaseModel):
bar: list[int] = [0, 1]
baz: list[int] = [2, 3]
daz: list[int] = [4, 6]
caz: Optional[list[int]] = [0, 0]
faz: Optional[list[int]] = [7, 8]
print(Foo(bar=[9, 0], daz=None, caz=None))
# bar=[9, 0] baz=[2, 3] daz=[4, 6] caz=None faz=[7, 8] I think an API like the following would work well for everyone, class Foo(BaseModel):
# explicitly `True`, overrides `Config`
bar : list[int] = Field(default=[1,2], none_as_default=True)
# explicitly `False`, overrides `Config`
baz : list[int] = Field(default=[1,2], none_as_default=False)
# behaviour depends on `Config`
bar : list[int] = Field(default=[1,2])
class Config:
# default `True`
none_as_default = False |
Hi! I was just facing the same issue and thought I'd hack something up. The following is what I came up with. It mainly just expands on the code from @k4nar class Foo(BaseModel):
use_default_for_none: int = 1
preserve_none_value: Optional[int] = 1
no_default: int
@field_validator("*", mode="before")
@classmethod
def _usedefault_for_none(cls, value, ctx: ValidationInfo) -> Any:
"""
Will use the default value for the field if the value is None and the annotation doesn't allow for a None input.
:param data: The data to be validated.
:return: The data with the None values replaced with the default value.
"""
if value is None and not isinstance(value, cls.model_fields[ctx.field_name].annotation): # Check if the value is None and the annotation doesn't allow for None.
raise PydanticUseDefault()
return value I think the way this handles the conversion would be fitting as it still allows for None-values in the model if the annotation allows for them. Best, Edit: This will cause Issues if you inherit from your Model and use discriminator fields. You would have to call this field_validator on every field separately because you can't exclude just one field from the special "*" field name. |
I'd love to see the suggestion from @Meetesh-Saini implemented so I can get rid of my validator |
While raising
|
Initial Checks
Description
Let's take this simple model as an example
There already seems to be some demand for this:
Affected Components
.model_dump()
and.model_dump_json()
model_construct()
, pickling, private attributes, ORM modeThe text was updated successfully, but these errors were encountered: