Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add experiment analysis functionalities #185

Draft
wants to merge 49 commits into
base: main
Choose a base branch
from
Draft

feat: add experiment analysis functionalities #185

wants to merge 49 commits into from

Commits on Jul 19, 2024

  1. add metric and variant classes

    @ludovico-lanni
    ludovico-lanni committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    bf3bf2e View commit details
    Browse the repository at this point in the history

  2. add dimension class

    @ludovico-lanni
    ludovico-lanni committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    5514c7e View commit details
    Browse the repository at this point in the history

  3. add hypothesis test class

    @ludovico-lanni
    ludovico-lanni committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    0a786b3 View commit details
    Browse the repository at this point in the history

  4. add hypothesis test class

    @ludovico-lanni
    ludovico-lanni committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    d830e8e View commit details
    Browse the repository at this point in the history

  5. add analysis plan class

    @ludovico-lanni
    ludovico-lanni committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    1477e8a View commit details
    Browse the repository at this point in the history

  6. add analysis results data structures

    @ludovico-lanni
    ludovico-lanni committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    0f51902 View commit details
    Browse the repository at this point in the history

  7. add analysis plan analyze signature

    @ludovico-lanni
    ludovico-lanni committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    4524381 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2024

  1. add analysis plan analyze draft

    @ludovico-lanni
    ludovico-lanni committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    45fd7a2 View commit details
    Browse the repository at this point in the history

  2. change signature for metric components

    @ludovico-lanni
    ludovico-lanni committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    278929c View commit details
    Browse the repository at this point in the history

  3. fix bugs

    @ludovico-lanni
    ludovico-lanni committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    7fb2a15 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2024

  1. improve metric system

    @ludovico-lanni
    ludovico-lanni committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    5f9e1a2 View commit details
    Browse the repository at this point in the history

  2. add confidence interval dataclass

    @ludovico-lanni
    ludovico-lanni committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    62562a3 View commit details
    Browse the repository at this point in the history

  3. add confidence interval logic

    @ludovico-lanni
    ludovico-lanni committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    55ee65b View commit details
    Browse the repository at this point in the history

  4. add alpha in results contract

    @ludovico-lanni
    ludovico-lanni committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    8345596 View commit details
    Browse the repository at this point in the history

  5. add support for default total dimension

    @ludovico-lanni
    ludovico-lanni committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    2933355 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. refactor directory structure into analytics submodule

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    85e12aa View commit details
    Browse the repository at this point in the history

  2. edit docstring for analysis plan

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    423d285 View commit details
    Browse the repository at this point in the history

  3. make Variant a dataclass

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    5ba0e29 View commit details
    Browse the repository at this point in the history

  4. make Dimension a dataclass

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    d55a56d View commit details
    Browse the repository at this point in the history

  5. create inference results logic for abstract experiment analysis class

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    d924165 View commit details
    Browse the repository at this point in the history

  6. add inference results schema and computation method

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    bda6e0c View commit details
    Browse the repository at this point in the history

  7. refactor analysis plan to use inference results

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    a711995 View commit details
    Browse the repository at this point in the history

  8. move analysis class selection to hypothesis test class

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    1da6b84 View commit details
    Browse the repository at this point in the history

  9. added support for cupac

    @ludovico-lanni
    ludovico-lanni committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    00b06d2 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. refactor directory name

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    53cc381 View commit details
    Browse the repository at this point in the history

  2. change validation method for variant

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    85f1e61 View commit details
    Browse the repository at this point in the history

  3. change validation method for dimension and rename default dimension

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    fda49f3 View commit details
    Browse the repository at this point in the history

  4. change validation method for analysis plan

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    ac5cb9a View commit details
    Browse the repository at this point in the history

  5. define variant properties

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    54b24ea View commit details
    Browse the repository at this point in the history

  6. define variant properties

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    045d4a1 View commit details
    Browse the repository at this point in the history

  7. move inference results to hypothesis test

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    abade82 View commit details
    Browse the repository at this point in the history

  8. move inference results to hypothesis test

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    c5f4b16 View commit details
    Browse the repository at this point in the history

  9. move cupac logic and config logic from plan to test

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    c98639d View commit details
    Browse the repository at this point in the history

  10. move prepare data from plan to test

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    ceda3b0 View commit details
    Browse the repository at this point in the history

  11. fix bug in cupac handler instantiation

    @ludovico-lanni
    ludovico-lanni committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    d48b621 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. raise error to handle missing cupac covariate

    @ludovico-lanni
    ludovico-lanni committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    3b7cf5b View commit details
    Browse the repository at this point in the history

  2. raise error to handle missing cupac covariate

    @ludovico-lanni
    ludovico-lanni committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    e618629 View commit details
    Browse the repository at this point in the history

  3. restructure loops for readability

    @ludovico-lanni
    ludovico-lanni committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    42a8297 View commit details
    Browse the repository at this point in the history

  4. refactor analysis results data structures

    @ludovico-lanni
    ludovico-lanni committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    4bf5e47 View commit details
    Browse the repository at this point in the history

  5. add data checks in analyse method

    @ludovico-lanni
    ludovico-lanni committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    98b490c View commit details
    Browse the repository at this point in the history

  6. add simplified from_metrics interface for analysis plan

    @ludovico-lanni
    ludovico-lanni committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    83791ed View commit details
    Browse the repository at this point in the history

  7. add verbose logging

    @ludovico-lanni
    ludovico-lanni committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    08943bb View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. fix logging config bad practise

    @ludovico-lanni
    ludovico-lanni committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    d20e10b View commit details
    Browse the repository at this point in the history

  2. make use of class property for metric's target_column

    @ludovico-lanni
    ludovico-lanni committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    05be002 View commit details
    Browse the repository at this point in the history

  3. move add_covariates logic to hypothesis test

    @ludovico-lanni
    ludovico-lanni committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    ccba301 View commit details
    Browse the repository at this point in the history

  4. move get_test_results code to hypothesis test

    @ludovico-lanni
    ludovico-lanni committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    697c5fe View commit details
    Browse the repository at this point in the history

  5. make use of default fields for AnalysisPlanResults dataclass

    @ludovico-lanni
    ludovico-lanni committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    0cc21d3 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. add check for analysis type allowed values

    @ludovico-lanni
    ludovico-lanni committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    8d12b88 View commit details
    Browse the repository at this point in the history

  2. fix analysis class handling to ensure flexibility and extensibility o… 

    …f the library
    @ludovico-lanni
    ludovico-lanni committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    e4f14e6 View commit details
    Browse the repository at this point in the history