Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support url for input asset paths #327

Merged
merged 1 commit into from
Sep 25, 2024
Merged

Support url for input asset paths #327

merged 1 commit into from
Sep 25, 2024

Conversation

iamksuresh
Copy link
Collaborator

Description

Support urls for input assets paths (data,model,background_path, ground_truth)

Type of Change

feat: A new feature

How to Test

[Provide clear instructions on how to test and verify the changes introduced by this pull request, including any specific unit tests you have created to demonstrate your changes.]

Checklist

Please check all the boxes that apply to this pull request using "x":

  • I have tested the changes locally and verified that they work as expected.
  • I have added or updated the necessary documentation (README, API docs, etc.).
  • I have added appropriate unit tests or functional tests for the changes made.
  • I have followed the project's coding conventions and style guidelines.
  • I have rebased my branch onto the latest commit of the main branch.
  • I have squashed or reorganized my commits into logical units.
  • I have added any necessary dependencies or packages to the project's build configuration.
  • I have performed a self-review of my own code.
  • I have read, understood and agree to the Developer Certificate of Origin below, which this project utilises.

Screenshots (if applicable)

[If the changes involve visual modifications, include screenshots or GIFs that demonstrate the changes.]

Additional Notes

[Add any additional information or context that might be relevant to reviewers.]

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
   have the right to submit it under the open source license
   indicated in the file; or

(b) The contribution is based upon previous work that, to the best
   of my knowledge, is covered under an appropriate open source
   license and I have the right under that license to submit that
   work with modifications, whether created in whole or in part
   by me, under the same open source license (unless I am
   permitted to submit under a different license), as indicated
   in the file; or

(c) The contribution was provided directly to me by some other
   person who certified (a), (b) or (c) and I have not modified
   it.

(d) I understand and agree that this project and the contribution
   are public and that a record of the contribution (including all
   personal information I submit with it, including my sign-off) is
   maintained indefinitely and may be redistributed consistent with
   this project or the open source license(s) involved.

@iamksuresh iamksuresh changed the title Remote url Support url for input asset paths Sep 25, 2024
@timlrx timlrx changed the base branch from main to v2.x September 25, 2024 04:44
@iamksuresh
Copy link
Collaborator Author

@timlrx / @imda-peckyoke : For now, data_path can only support files download and not folder(Zip). This will be fixed later in test-engine-core module.

@timlrx
Copy link
Collaborator

timlrx commented Sep 25, 2024

looks right to me. Note that the refactored shap plugin already includes these changes so that should cover all of them. Could you include a few example test in the PR for ease of replication purposes?

@iamksuresh
Copy link
Collaborator Author

Few samples to test -

Partial dependence plot

python -m aiverify_partial_dependence_plot --data_path https://raw.githubusercontent.com/aiverify-foundation/aiverify/refs/heads/v2.x/stock-plugins/user_defined_files/data/sample_bc_credit_data.sav --model_path https://raw.githubusercontent.com/iamksuresh/sample-data-for-aiv/refs/heads/main/stock-plugins/user_defined_files/model/sample_bc_credit_sklearn_linear.LogisticRegression.sav --ground_truth_path https://raw.githubusercontent.com/aiverify-foundation/aiverify/refs/heads/v2.x/stock-plugins/user_defined_files/data/sample_bc_credit_data.sav --ground_truth default --model_type CLASSIFICATION --no-run_pipeline

fairness metrics for regression

python -m aiverify_fairness_metrics_toolbox_for_regression --data_path https://raw.githubusercontent.com/iamksuresh/sample-data-for-aiv/refs/heads/main/stock-plugins/user_defined_files/data/sample_reg_pipeline_data.sav --model_path https://raw.githubusercontent.com/iamksuresh/sample-data-for-aiv/refs/heads/main/stock-plugins/user_defined_files/pipeline.zip --ground_truth_path https://raw.githubusercontent.com/iamksuresh/sample-data-for-aiv/refs/heads/main/stock-plugins/user_defined_files/data/sample_reg_pipeline_ytest_data.sav --ground_truth donation --model_type REGRESSION --run_pipeline --sensitive_features_list gender

fairness metrics for classification

python -m aiverify_fairness_metrics_toolbox_for_classification --data_path https://raw.githubusercontent.com/aiverify-foundation/aiverify/refs/heads/v2.x/stock-plugins/user_defined_files/data/sample_mc_pipeline_toxic_data.sav --ground_truth_path https://raw.githubusercontent.com/aiverify-foundation/aiverify/refs/heads/v2.x/stock-plugins/user_defined_files/data/sample_mc_pipeline_toxic_ytest_data.sav --ground_truth toxic --model_type CLASSIFICATION --run_pipeline --sensitive_features_list gender --model_path https://raw.githubusercontent.com/iamksuresh/sample-data-for-aiv/refs/heads/main/stock-plugins/user_defined_files/pipeline/mc_tabular_toxic.zip

Copy link
Contributor

@imda-peckyoke imda-peckyoke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved

@imda-peckyoke imda-peckyoke merged commit 8fc5f82 into v2.x Sep 25, 2024
3 of 6 checks passed
@imda-peckyoke imda-peckyoke deleted the remote_url branch September 25, 2024 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants