feat: Add allow_missing_columns option to read/scan_parquet

[nameexhaustion]

Let me know if there is a better name for this parameter

[codecov]

Codecov Report

Attention: Patch coverage is 50.31847% with 78 lines in your changes missing coverage. Please review.

Project coverage is 80.03%. Comparing base (011e366) to head (9701b30).
Report is 38 commits behind head on main.

Files with missing lines	Patch %	Lines
...tream/src/nodes/parquet_source/row_group_decode.rs	0.00%	54 Missing ⚠️
crates/polars-io/src/parquet/read/read_impl.rs	78.72%	10 Missing ⚠️
...-stream/src/nodes/parquet_source/metadata_fetch.rs	0.00%	10 Missing ⚠️
...m/src/nodes/parquet_source/row_group_data_fetch.rs	0.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #18922      +/-   ##
==========================================
+ Coverage   79.88%   80.03%   +0.14%     
==========================================
  Files        1521     1523       +2     
  Lines      206849   208786    +1937     
  Branches     2906     2906              
==========================================
+ Hits       165238   167092    +1854     
- Misses      41063    41146      +83     
  Partials      548      548              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[coastalwhite]

nit: if cfg!(debug_assertions)

[nameexhaustion]

I bumped the minor version for NodeTraverser as this exposes a new field in FileScanOptions

[coastalwhite]

I feel like strict_schema is a better name for this, but that is a bit subjective

[alexander-beedie]

How about allow_missing_columns?

Including the word "rows" in the parameter name feels a bit off, as it's really a column-oriented parameter 🤔

[ritchie46]

How about allow_missing_columns?

Yeah, let's go for that one. And default to False.

[coastalwhite]

I think this should also be added to read_parquet if possible

[nameexhaustion]

I think this should also be added to read_parquet if possible

I present a magic trick that adds it by changing the title of the PR 😀

[nameexhaustion]

Some technical detail - we do this by making our decode functions (i.e. row_groups_to_dfs) always return empty columns if the column is not found. The control over this behavior is done one level higher when you configure the parquet reader - it checks the schema of the file depending on the allow_missing_columns parameter

[ritchie46]

Nice, can we also refer to the solution in the error when the schema resolving fails in strict mode?