Permutations

[sverhoeven]

Refs #24

TODO

• - An experiment can consist of multiple runs
• - Each experiment has one reference configuration and zero or more permutations
• [x] - A reference configuration is complete (doesn't depend on defaults, i.e. all fields are defined explicitly) A reference configuration is complete (doesn't depend on defaults, i.e. all fields are defined explicitly) #32
• - A permutation defines one or more fields that should deviate from the reference. They are named.
• - A run is a single execution of class with a given configuration.
• - There are runs associated with both the reference and each permutation
• - To execute a permutation, the reference config should be combined with the permutations to obtain a new, complete, configuration.
• - ui
  • [X] - Visual hierarchy: reference should be above permutations Improve layout experiment card #31
  • - Permutations can display the difference from reference
  • - Adding a permutation should clearly be separated from cloning an experiment
  • - There should be an option to download an experiment as a json file
  • - There should be an option to download an experiment as zip, including the json file + one output csv per run
  • - Upload button should support uploading an experiment config json file
  • [x] - Removing a permutation is okay. Removing an experiment is also okay, but should warn! Removing an experiment is also okay, but should warn! #33
  • - It should be possible to promote a permutation to reference
• Address TODO comments in code
• Given you have a permutation what should happen when you change the reference configuration?

[Peter9192]

• Given you have a permutation what should happen when you change the reference configuration?

The full config of the permutation then becomes the new reference config + permutation config. That should probably trigger a re-run.

[Peter9192]

Awesome progress! Here are some comments on the functionality so far, we can discuss them Thursday:

• Duplicate experiment should also duplicate permutations
• Shall we name the experiments as well, in the same way we now name the permutations
• I think we would want to be able to duplicate a permutation as well
• For the promote button, I envisioned it would replace the current reference instead of starting a new experiment. Perhaps we should support both?

There's a few open tasks including the UI, not relying on defaults in reference config, and todo's in code. I'd be happy to play a bit more with the layout later on.

[Peter9192]

I quickly looked through the code as well.

[sverhoeven]

Awesome progress! Here are some comments on the functionality so far, we can discuss them Thursday:

• Duplicate experiment should also duplicate permutations
• Shall we name the experiments as well, in the same way we now name the permutations
• I think we would want to be able to duplicate a permutation as well
• For the promote button, I envisioned it would replace the current reference instead of starting a new experiment. Perhaps we should support both?

There's a few open tasks including the UI, not relying on defaults in reference config, and todo's in code. I'd be happy to play a bit more with the layout later on.

1. yep
2. yep, will also make permutation name editable
3. will do
4. will add swap with refence and promote to new experiment buttons. last buttonis current promote button.