Skip to content
This repository has been archived by the owner on Nov 24, 2021. It is now read-only.

Perform a daily comparison of a new simulation and historical simulation #41

Open
aeisenberg opened this issue May 26, 2020 · 0 comments

Comments

@aeisenberg
Copy link
Contributor

As a form of regression testing, we should be keeping track of how simulation results change over time. As we update the model and make changes to the connector code, we should be able to see how each change affects the output.

To start with, we can look at the Imperial model. See #40 for how we can generate the comparisons. This issue is about automating the process for one or more sets of runs and doing this comparison daily.

For each of these runs, choose a region, a set of interventions and a calibration date (along with the death count). Perform the run in a github action. After the run completes, perform a comparison with the previous day's run. Store the results as an artifact. And if the results indicate a large enough difference, fail the run.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant