Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random Order for Testing 2 Different Versions #131

Open
SaitTalhaNisanci opened this issue Dec 13, 2019 · 0 comments
Open

Random Order for Testing 2 Different Versions #131

SaitTalhaNisanci opened this issue Dec 13, 2019 · 0 comments

Comments

@SaitTalhaNisanci
Copy link
Contributor

Sometimes there are factors in our machines that can slow down performance a bit. It might be after a certain point. For example with AWS we have seen that IOPS would drop after some credits is reached.

We currently do not know anything like that for Azure, but there could be some 'yet-unknown' factors. I guess to be on the safe side, it would make sense to choose the order of versions randomly. So currently if we always run the later version first. We can decide that randomly so that this could bring us more confidence about performance. If a version is faster than another in both cases(running first and last) then it is very likely that it is faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant