playwright tests

[sreeramnitin]

Playwright Smoke testing for the notebooks.

Skipping few files for following reasons.

Error in the execution

../ModelOps/09_ModelOps_GIT_PIMA_Python_XGboost.ipynb
../ModelOps/05_ModelOps_BYOM_PIMA_H2O.ipynb
../ModelOps/04_ModelOps_BYOM_PIMA_ONNX.ipynb
../ModelOps/03_ModelOps_BYOM_PIMA_PMML.ipynb
../UseCases/Recommendations_product_search/Recommendations_product_search_OpenSource_Python.ipynb
../ModelOps/08_ModelOps_GIT_PIMA_Python_H2OAutoML.ipynb
../UseCases/Credit_Card_Data_Preparation/Credit_Card_Data_Preparation_SQL.ipynb
../UseCases/Chat_Powered_Product_Recommendation/Chat_powered_product_recommendation_based_on_search_output.ipynb
../UseCases/Heart_Failure_Prediction/Heart_Failure_Prediction_Python.ipynb
../UseCases/CancerPrediction_TDApiClient/CancerPrediction_tdapiClient_VertexAI_CustomModel.ipynb
../UseCases/Parkinsons_Disease_Prediction/Parkinsons_Disease_Prediction_PY_SQL.ipynb
../UseCases/Remaining_Useful_Life_Forecasting/Remaining_Useful_Life_Forecasting_PY_SQL.ipynb
../UseCases/Green_Manufacturing/Green_Manufacturing_Python.ipynb
../UseCases/startup.ipynb
../UseCases/Sales_Forecasting_UAF/Sales_Forecasting_UAF_PY_SQL.ipynb
../UseCases/Vertex_AI_Integration/Vertex_AI_Integration_Python.ipynb
../UseCases/Vantage_Analytics_Library/VAL_teradataml_Demo_Python.ipynb
../UseCases/Credit_Risk_Assessment/Credit_Risk_Assessment_Python.ipynb
../UseCases/Chat_wth_docs_GenAI/Chat_with_docs_VantageDB_GenAI_Python.ipynb
../Getting_Started/Data_Loading/Data_Loading_SQL.ipynb
../UseCases/Store_Sales_Forecasting_Prophet_STO/Store_Sales_Forecasting_Prophet_STO_PY_SQL.ipynb
../ModelOps/10_ModelOps_GIT_PIMA_R_GBM.ipynb

Requiring additional inputs from user

../ModelOps/10_ModelOps_GIT_PIMA_R_GBM.ipynb
../UseCases/Recommendations_product_search/Recommendations_product_search_OpenAI_using_TDAPIClient_Python.ipynb
../UseCases/Recommendations_product_search/Recommendations_product_search_OpenAI_Python.ipynb
../ModelOps/06_ModelOps_GIT_Project_Setup.ipynb
../UseCases/CancerPrediction_TDApiClient/CancerPrediction_tdapiClient_Sagemaker.ipynb
../UseCases/Mortgage_Calculator_GenAI/Mortgage_Calculator_Python.ipynb
../UseCases/Mortgage_Calculator_GenAI/Mortgage_Calculator_Python_TrustedAI_RAG.ipynb
../UseCases/Chat_wth_docs_GenAI/Chat_with_docs_OpenSource_GenAI_Python.ipynb
../UseCases/Energy_Consumption_Forecasting_AzureML/Energy_Consumption_Forecasting_AzureML.ipynb
../Getting_Started/PySpark_to_Teradataml/HousingPrices_pyspark_to_tdml.ipynb

Pop up error dialogs

../UseCases/.Parkinsons_Disease_Prediction/Parkinsons_Disease_Prediction_PY_SQL.ipynb model error (Directory not found)
../UseCases/Signal_Processing_and_Classification/Signal_Processing_and_Classification_Python.ipynb (Cannot open file)
../UseCases/Natural_Language_Processing/Natural_Language_Processing_Python.ipynb 
../UseCases/Data_Dictionary/Data_Dictionary.ipynb 
../UseCases/AnomalyDetection_OutstandingAmount_FS/AnomalyDetection_OutstandingAmount_FeatureStore.ipynb 

One or multiple steps taking to long to complete

../UseCases/Data_Dictionary/Data_Dictionary_Raw.ipynb
../UseCases/Query_Service/Query_Service.ipynb
../UseCases/Grocery_Recommendation/Grocery_Recommendation_PY_SQL.ipynb
../UseCases/Fine_Tuning_OpenAI_Model/Fine_Tuning_OpenAI_Model_Python.ipynb

Long running

../UseCases/MultiTouch_Attribution/Analyst_MultiTouch_Attribution_PY_SQL.ipynb
../UseCases/MultiTouch_Attribution/MultiTouch_Attribution_PY_SQL.ipynb

[DougEbel]

There are 3 groups of queries. 1. Those with pop-up chat boxes or paragraphs with non-standard text that request API keys or other credentials to run. These will need to be tested manually. 2. Those with very long run times (could be 30 minutes). Those need to be on a separate automated test list that could be initiated manually. I would separate these to allow the rest of the notebook testing to have a reasonable run time and to prevent these notebooks with a questionable user experience be blended with the majority of the notebooks. There are 7 notebooks running over 1,000 seconds. The next 7 run between 350 and 600 seconds. There are 142 notebooks running less than 350 seconds. 3. Those with reasonable run times that should be tested any time we are making a Vantage software release change or a change to libraries Dallas will need some documentation: * How to add notebooks to the list to test * Procedure for changing the requirements.txt and baking a new image in CI * Initiating the test procedure in CI or in production. Doug From: Nitin s ***@***.***> Sent: Sunday, June 23, 2024 1:27 PM To: Teradata/jupyter-demos ***@***.***> Cc: Ebel, Douglas H ***@***.***>; Review requested ***@***.***> Subject: [EXTERNAL] Re: [Teradata/jupyter-demos] playwright tests (PR #650) [CAUTION: External Email] @sreeramnitin commented on this pull request.

________________________________ In .tests/tests/jupytertest.spec.ts<#650 (comment)>:

+const files = readFileIntoArray(FILE_NAME).filter((name) => skipfiles.indexOf(name) === -1);

+ +const testCount = Math.ceil(files.length / CSAE_CI_JOB_COUNT); + +for (let i = 0; i < testCount; i++) { + const idx = i * CSAE_CI_JOB_COUNT + CSAE_CI_JOB_IDX; + if(idx >= files.length) { + break; + } + + const name = files[idx]; + if(name === '') { + continue; + } + test(`test ${i}: ${name}`, async ({page},testInfo) => { + test.setTimeout(10800000); There are some really long test. I have noted them and skipped for now. We can discus about them. Even Dough agreed that we do them manually. - Reply to this email directly, view it on GitHub<#650 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AYNBXG3Z2HRZAYKSQ3UMQMLZI4AONAVCNFSM6AAAAABJWOMTGWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCMZUGM2TKNRXGM>. You are receiving this because your review was requested.Message ID: ***@***.******@***.***>>