Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update code coverage metric #162

Open
sgherbst opened this issue Feb 23, 2021 · 0 comments
Open

Update code coverage metric #162

sgherbst opened this issue Feb 23, 2021 · 0 comments

Comments

@sgherbst
Copy link
Contributor

The code coverage displayed on the front page of the project is currently 27%, but this is a misleading number. It is the percentage of Python code exercised in the dragonphy directory, despite the fact that the design is mostly described in Verilog and TCL code. Plus, there is a bunch of code in the dragonphy directory that isn't used in generating the design, while there is additional mflowgen Python code that is not included in the coverage statistic.

We should fix these problems by adjusting the scope of the code that is included: count only related Python code, and count Verilog and TCL. Generating coverage for Verilog is straightforward, but integrating it into the codecov analysis may not be, because Verilog is not currently one of the supported languages. TCL is even more unknown, since I'm not sure if synthesis and PnR tools can produce a coverage output (TCL isn't supported by codecov, either).

This might seem like a cosmetic issue, but the purpose of coverage analysis is to find gaps in testing, so if we're not considering the right source code, we don't really learn anything from the analysis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant