Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Write row group statistics #92

Merged
merged 26 commits into from
Sep 22, 2024
Merged

Write row group statistics #92

merged 26 commits into from
Sep 22, 2024

Commits on Sep 15, 2024

  1. Write null_count per column chunk

    @gaborcsardi
    gaborcsardi committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    d1c5a01 View commit details
    Browse the repository at this point in the history

  2. Report min/max stats in read_parquet_metadata()

    @gaborcsardi
    gaborcsardi committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    4a1fc57 View commit details
    Browse the repository at this point in the history

  3. Improve docs 

    [ci skip]
    @gaborcsardi
    gaborcsardi committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    d6ab47b View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. Update test snapshot

    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    a669d90 View commit details
    Browse the repository at this point in the history

  2. Writer: pass row group & page to callbacks 

    So they can calculate min and max values easily.
    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    1b5d7ec View commit details
    Browse the repository at this point in the history

  3. Min/max for PLAIN INT32 and DATE

    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    55f126f View commit details
    Browse the repository at this point in the history

  4. Work around an unrelated crash

    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    b6ada24 View commit details
    Browse the repository at this point in the history

  5. Min/max for integer dictionaries

    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    87a3689 View commit details
    Browse the repository at this point in the history

  6. Fix a bug with string dicts and row groups

    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    5dae2d8 View commit details
    Browse the repository at this point in the history

  7. Test case for writing min/max for DATEs

    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    12016fa View commit details
    Browse the repository at this point in the history

  8. Fix tests on older R

    @gaborcsardi
    gaborcsardi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    f71bf85 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Test min/max when writing uncompressed data

    @gaborcsardi
    gaborcsardi committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    d999659 View commit details
    Browse the repository at this point in the history

  2. Minor refactor 

    Move function to better place.
    @gaborcsardi
    gaborcsardi committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    d46c91c View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2024

  1. Fixes for writing min/max for integers 

    - fix double -> int32 conversion
- fixes for NA values
    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    6262342 View commit details
    Browse the repository at this point in the history

  2. Fix writing min/max for TIME

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    4b128d0 View commit details
    Browse the repository at this point in the history

  3. Write min/max for DOUBLE

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    98b6425 View commit details
    Browse the repository at this point in the history

  4. Min/max for FLOAT

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    6dc4840 View commit details
    Browse the repository at this point in the history

  5. Write min/max for INT64

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    649b7fb View commit details
    Browse the repository at this point in the history

  6. Fix metadata snapshot tests

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    16d3c1e View commit details
    Browse the repository at this point in the history

  7. Ooops, really fix snapshots

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    ca834ce View commit details
    Browse the repository at this point in the history

  8. Fix compilation on Linux

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    105c7d1 View commit details
    Browse the repository at this point in the history

  9. Better error from min/max conversion

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    07338d1 View commit details
    Browse the repository at this point in the history

  10. Implement min/max for various string (BYTE_ARRAY) types

    @gaborcsardi
    gaborcsardi committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    3d0f67b View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2024

  1. Min/max for TIMESTAMP

    @gaborcsardi
    gaborcsardi committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    f9e5428 View commit details
    Browse the repository at this point in the history

  2. Document types without min/max write support

    @gaborcsardi
    gaborcsardi committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    3f96d6c View commit details
    Browse the repository at this point in the history

  3. NEWS for writing min/max 

    [CI skip]
    @gaborcsardi
    gaborcsardi committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    e43fc37 View commit details
    Browse the repository at this point in the history