Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: big query procedure for comparing validator versions #770

Merged
merged 4 commits into from
Oct 22, 2024
Merged

Conversation

cka-y
Copy link
Contributor

@cka-y cka-y commented Oct 21, 2024

Summary:
This PR introduces a new BigQuery function and updates the infrastructure to enable comparison of validation reports between two specified versions.

BigQuery Table Value Function

  • bigquery/compare-validation-reports.sql: Added a new function, compare_validation_reports, designed to compare validation reports between a previous and a current version. This procedure generates a merged report highlighting the differences in validation notices, including new and dropped notices.

Terraform Configuration

  • infra/metrics/main.tf: Updated Terraform configuration to include a new BigQuery routine resource, compare_validation_reports. This change ensures the deployment and management of the new function in the infrastructure.

Testing Instructions:
To test the function, execute the following call in the dev environment:

SELECT * FROM `mobility-feeds-dev.data_analytics.compare_validation_reports`('5.0.1', '5.0.2-SNAPSHOT');

Example of connected sheet with the results

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@cka-y cka-y linked an issue Oct 21, 2024 that may be closed by this pull request
Copy link
Member

@davidgamez davidgamez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

IFNULL(most_recent_notices.totalNotices, 0) AS totalNotices_current
FROM
old_version_notices
FULL OUTER JOIN
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@cka-y cka-y merged commit 12eedfb into main Oct 22, 2024
3 checks passed
@cka-y cka-y deleted the feat/753-3 branch October 22, 2024 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generate validator analytics from our API pipeline
2 participants