@Ravencentric,

I think it probably makes sense to use a custom validator for this. For example:

from pydantic import BaseModel, ValidationInfo, field_validator


class Foo(BaseModel):
    bar: list[int] | None = [0,1,2]
    baz: list[str] | None = ["a", "b", "c"]

    @field_validator('bar', 'baz')
    @classmethod
    def check_alphanumeric(cls, v: str, info: ValidationInfo) -> str:
        if v is None:
            return cls.model_fields[info.field_name].default
        return v

print(repr(Foo()))
# Foo(bar=[0, 1, 2], baz=['a', 'b', 'c'])

print(repr(Foo(bar=[1,2], baz=["a","b"])))
# Foo(bar=[1, 2], baz=['a', 'b'])

print(repr(Foo(bar=None, baz=None))) # Explicitly pass `None`
# Foo(bar=[0, 1, 2], baz=['a', 'b', 'c'])

I haven't seen an abundance of demand for this feature. If there's significant community interest, we could reconsider making this a flag on a field, but for now I think the field validator approach (a change in user code) makes the most sense!

That gets the job done, thank you!

@sydney-runkle
Thank you for the example.

Can we do the same in BeforeValidator? I love the Annotated + Validator pattern as it provides stronger reusability. But I couldn't access the default value information in the validator function in this case.

Specifically I'd like to do something like this.

def validator(v: Any, info: ValidationInfo) -> Any:
    print(
        info
    )  # ValidationInfo(config={'title': 'MyClass'}, context=None, data={}, field_name='field')
    if v is None:
        default = get_default_somehow()
        return default
    return v


class MyClass(BaseModel):
    field: Annotated[int, Field(default=0), BeforeValidator(validator)]


MyClass.model_validate({"field": None})

Although I can explicitly add BeforeValidator like following, writing default value twice looks a bit weird.

def null_default_validator(default: Any) -> BeforeValidator:
    def validator(v: Any) -> Any:
        if v is None:
            return default
        return v

    return BeforeValidator(validator)


class MyClass(BaseModel):
    field: Annotated[int, Field(default=0), null_default_validator(default=0)]


MyClass.model_validate({"field": None})

I have to say that I would have wanted this feature.
Here is my usecase - I'm getting a serialized api request to my service - and the sender doesn't have the option to exclude undefined fields, so they have to send them as nulls which get serialized to None in python.

The thing is - that I already have a default value which I want this field to take in the case it is None.
Right now the default value works only for the case it haven't been sent(undefined).

I want an option to configure this behavior so it'll use the default on None as well - it will give me more freedom.
Otherwise what I need to do is to manually populate the field in the validator - or in usage always check if the field is not None which is redundant.

@yotaro-shimose and @tsalex1992 - Just in case you might find it helpful, our team encountered a slightly similar issue and came up with another possible solution.

Sadly, it doesn't fit with the Annotated + Validator pattern but it is fairly reusable. With your Foo example above, for instance:

# some/path/to/utils.py
def generic_validator_function_with_default(class_ref, value, info):
    return (
        value
        if value is not None
        else get_default_value_from_field_config(class_ref, info)
    )


def get_default_value_from_field_config(class_ref, info):
    field = class_ref.model_fields[info.field_name]
    return field.default_factory() if callable(field.default_factory) else field.default


# some/path/to/models/foo.py
from some.path.to.utils import generic_validator_function_with_default

class Foo(BaseModel):
    bar: list[int] | None = [0,1,2]
    baz: list[str] | None = ["a", "b", "c"]

    @field_validator('bar', 'baz')
    @classmethod
    def handle_field_default(cls, value, info):
        return generic_validator_function_with_default(cls, value, info)
        

@saurookadook
Thank you for your suggestion. I love that example as it shows good reusability.

As your workaround says, it might be good to have reference to the class to be wrapped in function validators. Hopefully it works with python Annotated syntax so that pydantic maintainers can implement it.

Having this syntax would be great for shared library code as well
Is there anything one can do to help out on this issue?

@MarkusSagen,

I'd be happy to review a PR with support for this via a field flag if the API is pretty clean, given that there seems to be a decent amount of requests for this. I think we want to do this on the field level and not the config level, but I'm not sure...

In the API I'm dealing with, I needed it globally on every model so a model_config level would be appreciated

Thank you @sydney-runkle
Do you have any pointers for which file the changes would be placed in
I've not yet worked with the internals of Pyadntic

Being able to set defaults like this would also be useful for the cases when using ConfigDict(from_attributes=True) mode, which is going to remove the need to validate each field which could possibly be None in the database.

After a bit of tinkering, I arrived independently arrived at a similar solution to @saurookadook which is added to our common BaseModel that contains our configuration and util functions:

    @field_validator("*", mode="before")
    @classmethod
    def not_none(cls, v, val_info):
        """
        Convert None to Default on optional fields.
        Why doesn't Pydantic have this option?
        """
        field = cls.__fields__[val_info.field_name]
        if v is None and (default := field.get_default(call_default_factory=True)) is not None:
            return default
        return v

This could also be broken out into a function and called with _not_none = field_validator("my_field", "my_field2")(not_none) or used inside a WrapValidator, but in our case any Optional field can potentially be populated with None when we'd rather have our default values.

I'd still rather have a Field(replace_none_with_default=True) option though.

Here's another solution.

from pydantic import BaseModel, model_validator

from typing import Optional

class TestModel(BaseModel):
    name: str
    age: Optional[int]=0
    
    @model_validator(mode='before')
    @classmethod
    def remove_none(cls, data):
        if isinstance(data, dict):
            return {k: v for k, v in data.items() if v is not None}
        return data

data = {"name": "test", "age": None}
model = TestModel(**data)

See also this: #8585
But for that, we would need to have access to the model's class inside ValidationInfo.

After some studies I came to this result, it works, but it is extremely inelegant, it would be great if pydantic already did this by default, having to create this subclass is not the best thing in the world as I have to duplicate this in all my python packages, or create another python package just for this code below, either way, it's not elegant

from pydantic import BaseModel
from pydantic_core import PydanticUndefinedType

class CustomBaseModel(BaseModel):
    @model_validator(mode='before')
    def set_defaults_for_none(cls, values):
        fields = cls.model_fields
        for field_name, field_info in fields.items():
            value = values.get(field_name)
            default_value = (
                field_info.default
                if not isinstance(field_info.default, PydanticUndefinedType)
                else None
            )
            default_factory = (
                field_info.default_factory
                if not isinstance(
                    field_info.default_factory, PydanticUndefinedType
                )
                else None
            )

            if value is None:
                if default_value is not None:
                    values[field_name] = default_value
                elif default_factory is not None:
                    values[field_name] = default_factory()

        return values

@Ravencentric,

I haven't seen an abundance of demand for this feature. If there's significant community interest, we could reconsider making this a flag on a field, but for now I think the field validator approach (a change in user code) makes the most sense!

I typically use Pydantic schema classes in GPT prompts to ensure the response is in JSON format. However, when the GPT model doesn't find a value for a field, it initializes the field with None instead of using the default value. This issue suggests the need for a flag in the Field class to handle such cases appropriately.

Semi-related, the example in this section: https://docs.pydantic.dev/latest/concepts/json/#partial-json-parsing

Going back to the @Ravencentric's original example, IMO the current behavior is actually preferable as written because the type annotation marked the field as optional (list[int] | None). In that case it makes sense that None should be an allowed value that you can explicitly pass in.

However I think in most cases if you have a default value to use for a field then you don't want that field to be optional.

For example:

class Foo(BaseModel):
    bar: list[int]= [0,1,2]

Foo()
# Foo(bar=[1,2,3])

Foo(bar=None)
# raises ValidationError 

In that case it would be preferable if it used the default value instead of raising an error.

The model_validator workaround that others mentioned does work, but that also raises the question why doesn't a mode="before" field validator work? That part is pretty unexpected that the None type validation runs before the "before" field validators but not before the "before" model validator.

Based on the example suggested by @sydney-runkle, this seems to be working well and should contend everyone in this thread 🙂 :

from pydantic import BaseModel, field_validator
from pydantic_core import PydanticUseDefault


class Foo(BaseModel):
    bar: list[int] = [0,1,2]
    baz: list[str] = ["a", "b", "c"]

    @field_validator("*", mode="before")
    @classmethod
    def none_to_default(cls, v):
        if v is None:
            raise PydanticUseDefault()
        return v

assert Foo(bar=None, baz=None) == Foo()

Hey @sydney-runkle, I'd like to work on this. I agree with @dcosson's suggestion, so I made some local changes to pydantic-core, and this is what I have so far:

from typing import Optional
from pydantic import BaseModel

class Foo(BaseModel):
    bar: list[int] = [0, 1]
    baz: list[int] = [2, 3]
    daz: list[int] = [4, 6]
    caz: Optional[list[int]] = [0, 0]
    faz: Optional[list[int]] = [7, 8]

print(Foo(bar=[9, 0], daz=None, caz=None))
# bar=[9, 0] baz=[2, 3] daz=[4, 6] caz=None faz=[7, 8]

I think an API like the following would work well for everyone,

class Foo(BaseModel):
    # explicitly `True`, overrides `Config`
    bar : list[int] = Field(default=[1,2], none_as_default=True)

    # explicitly `False`, overrides `Config`
    baz : list[int] = Field(default=[1,2], none_as_default=False)
    
    # behaviour depends on `Config`
    bar : list[int] = Field(default=[1,2])

    class Config:
        # default `True`
        none_as_default = False

Hi! I was just facing the same issue and thought I'd hack something up. The following is what I came up with. It mainly just expands on the code from @k4nar

class Foo(BaseModel):
    use_default_for_none: int = 1
    preserve_none_value: Optional[int] = 1
    no_default: int

    @field_validator("*", mode="before")
    @classmethod
    def _usedefault_for_none(cls, value, ctx: ValidationInfo) -> Any:
        """
        Will use the default value for the field if the value is None and the annotation doesn't allow for a None input.

        :param data: The data to be validated.
        :return: The data with the None values replaced with the default value.
        """
        if value is None and not isinstance(value, cls.model_fields[ctx.field_name].annotation):  # Check if the value is None and the annotation doesn't allow for None.
            raise PydanticUseDefault()
        return value

I think the way this handles the conversion would be fitting as it still allows for None-values in the model if the annotation allows for them.

Best,
Gab

Edit: This will cause Issues if you inherit from your Model and use discriminator fields. You would have to call this field_validator on every field separately because you can't exclude just one field from the special "*" field name.
I'm hope we can get @Meetesh-Saini's solution soon.

I'd love to see the suggestion from @Meetesh-Saini implemented so I can get rid of my validator

While raising PydanticUseDefault() works well for us in practice, this is now causing errors in our test suite:

pydantic_core._pydantic_core.SchemaError: Uncaught UseDefault error, please check your usage of default validators.