Robustness Toolbox: algorithm output schema validation failed

[gang-yi]

Hi, I was running the robustness test for tabular dataset (serialised with pickle, pandas version 1.5.3).

As per the suggested algorithm user inputs, I set the annotated ground truth path to the ground truth dataset (testing dataset), and keyed in "NA" for the name of column containing image file name.

Attached error notification and snippet of log below:

[imda-kelvinkok]

Hello. Thanks for your feedback. Can I know what dataset and ground truth dataset are you using, and are you running AI Verify via Docker?

[gang-yi]

Hi, I'm not able to provide the data or model as they are confidential. Yes, via Docker.
The model type is xgboost.sklearn.XGBClassifier, xgboost==1.1.0

I was previously able to run the global explainability test. (also, would like check if it's possible to run both local and global explainability test for the same report?)
Had tried running the tests earlier but also encountered #135

In beta v0.3.0, I was able to run the robustness test, but had to make some changes for that:

• change to pandas==1.3.5 for the data
• add attributes to the model - 'use_label_encoder': None, 'enable_categorical': False, 'predictor': 'auto'

[kimeetok]

Hi @gang-yi , apologies we currently support xgboost 1.7.5 and have not tested for 1.1.0, suggest for you to upgrade if possible and try again. List of supported frameworks
As for #135 it has been fixed in our recent v0.9.2 release here.
Please try it out and let us know if your issues persist, thank you! :)

[yschan-ntu]

Hi @kimeetok, quick check-in to let you know that this issue has not resolved with xgboost 1.7.5. The same error is still showing Error log attached.

robustness_err_20220810.txt

[imda-benedictlee]

Hi @yschan-ntu and @gang-yi, we have released AI Verify v0.9.3. It should solve the XGBoost Issue. Do give it a try and let us know if it works for you. Thanks!