Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show test results as soon as they occur #1061

Open
ffMathy opened this issue Sep 4, 2023 · 11 comments · May be fixed by #1075
Open

Show test results as soon as they occur #1061

ffMathy opened this issue Sep 4, 2023 · 11 comments · May be fixed by #1075
Labels
need-discussion needs more in-depth discussion

Comments

@ffMathy
Copy link

ffMathy commented Sep 4, 2023

It would be awesome if the extension would not wait until all tests have run to show the status of individual tests.

This especially happens if the tests are in the same file.

@connectdotz
Copy link
Collaborator

This extension processes test results from a json file generated by jest after it runs all the tests in the given run. Changing it would mean the extension needs to parse the output for test results, errors, coverage reporting, etc., which is not trivial and would be fragile whenever jest changes its output messaging. Jest created the stable json file precisely to spare the tooling systems from such pain...

Having said that, the delay should not be too noticeable... unless there is a bug? Does this happen more noticeably in certain files or situations?

@ffMathy
Copy link
Author

ffMathy commented Sep 14, 2023

It's not the extension's fault. I am running some integration tests that spin up a lot of stuff, which just takes time.

However, would it be possible somehow to hook into the reporters feature of Jest or something similar? There must be some way to get the status while it's running.

Maybe this can help?
jestjs/jest#5292

@ffMathy
Copy link
Author

ffMathy commented Sep 15, 2023

I just confirmed that Jest reporters can get the status after each test. So it should be doable for the reporter to communicate back.

@ffMathy
Copy link
Author

ffMathy commented Sep 15, 2023

Here's a sample reporter that can get the results of tests as they occur.

https://github.com/ryparker/jest-reporter-template/blob/master/src/reporter.ts

I really think this could be useful. Also for Playwright tests etc which may also take some time. Currently when you have hundreds of tests, it is super useful.

@ffMathy
Copy link
Author

ffMathy commented Sep 15, 2023

Using this approach, it wouldn't be fragile. In fact, it would be officially supported.

I'm even willing to do a PR on this if you can point me to where and how you'd approach this 🙏

@connectdotz
Copy link
Collaborator

Indeed, this is promising! We already have a reporter that was included in all test run; you can start from there.

The most challenging part, I think, is to match the assertion to the source code blocks (parsed by @babel/parser in jest-editor-support), because the location info do not always match, and the title match also will not match for dynamic title like test.each. So we used something we called "context match" (match-by-context.ts) that combines with multiple match mechanisms to provide a more robust match. In processing one test block at a time, we might not have the full context, which will most likely fail the match logic and require some non-trivial fixes...

The way we notify the testing system (for visual update) is done in test-item-date.ts, you can take a look at the event processing there to get a sense how things wire together.

I think full feature parity with the current process might be pretty involved... But do feel free to take a look of the code and let me know what you think and we can take from there.

@connectdotz connectdotz added need-discussion needs more in-depth discussion and removed stage: awaiting response labels Sep 15, 2023
@ffMathy
Copy link
Author

ffMathy commented Sep 17, 2023

That's great. I'll see what I can do. But either way, let's leave this issue open until it is fixed. Not sure I'll get the time.

@ffMathy
Copy link
Author

ffMathy commented Sep 29, 2023

I got it working! Just need to clean it up now. Pull request coming soon! 🥳

@ffMathy
Copy link
Author

ffMathy commented Sep 30, 2023

May I ask... Is there a reason that Runner.js, Settings.js, Snapshot.js and test_reconciler.js are not in TypeScript?

@ffMathy ffMathy linked a pull request Sep 30, 2023 that will close this issue
@ffMathy
Copy link
Author

ffMathy commented Sep 30, 2023

@connectdotz the PR is over at #1075!

Let me know what you think. It currently breaks some tests, and I could use some assistance figuring out why. But the feature works.

@lonevvolf
Copy link

This would be a really great feature to have - can be a big time saver for large suites of tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need-discussion needs more in-depth discussion
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants