Allure Framework
Different testCaseId when Retrying; Results in Duplicated Test Case #2653
Unanswered
Staicul
asked this question in
Questions & Support
Replies: 1 comment 3 replies
-
|
From the code, the |
Beta Was this translation helpful? Give feedback.
3 replies
-
Beta Was this translation helpful? Give feedback.
-
|
Thanks! |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
We have a WebdriverIO TS test automation project. For mobile native tests (executed on SauceLabs), we added 2 retries. So, in case a test fails, it's retried (max 2 times) until it passes.
The retry mechanism works, but reporting is broken. Every retry is considered by Allure a new test, and if a (Jenkins) job has tests that didn't pass on the first try, but passed on retries, the build is not green.
The reporting works well for web. The test is marked as passed and retries appear in the retry tab. However, when running on real devices on SauceLabs, the retry is considered a new test case. Check attached screenshots.
I was able to isolate the issue by going through the Allure result.json files and the problem seems to be that the retry gets a different 'testCaseId'. On web, the retry has the same 'testCaseId' as the original run.
Any explanations for this behaviour?
Is there a way to force the
testCaseIdto be the same for test cases with the same name? Or any other workaround?Beta Was this translation helpful? Give feedback.
All reactions