Skip to content

Commit

Permalink
chore: replaces c8 with node native coverage (#203)
Browse files Browse the repository at this point in the history
## Description

- removes c8 dependency &  config
- uses  the --experimental-test-coverage cli switch to get coverage
- updates `./tools/dot-with-summary.reporter.js` reporter to emit
coverage information
- updates the ci workflow to emit information from that report instead
of the markdown derived from the codecov run (on node 20 only for now as
there it's stable).

## Motivation and Context

In our setup c8 doesn't provide any real coverage information anymore
(see below - note the number of lines, functions and branches). Root
cause probably lies [how node handles ESM/ how v8-to-istanbul can
process that](bcoe/c8#34). As nodejs now also
has a --experimental-test-coverage built in we migrate this to that.

> --experimental-test-coverage isn't perfect either b.t.w. In node 20
it's stable, but on node 22 (at least in our set up) it tends to report
different results on each run with unchanged source code.

<details>
<summary>recent c8 output on virtual-code-owners main branch</summary>

```shell
$ npm test

> [email protected] test
> c8 tsx --test-reporter ./tools/dot-with-summary.reporter.js --test src/*.test.ts src/**/*.test.ts

.....................................................

53 passing (1.159 ms)

=============================== Coverage summary ===============================
Statements   : 100% ( 11/11 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 0/0 )
Lines        : 100% ( 11/11 )
================================================================================
```

</details>

## How Has This Been Tested?

- [x] green ci
- [x] manually (the script in tools should probably be put in a separate
repo and get its own set of unit tests c.s., but for fixing the issue at
hand a bit out of scope.

## Screenshots

```
.....................................................

53 passing (1.199 ms)

=============================== Coverage summary ===============================
Branches     : 100 % (205/205)
Functions    : 100 % (123/123)
Lines        : 100 % (1.533/1.533)
================================================================================
```

With uncovered lines/ functions (b.t.w. same, unchanged codebase as
above ..., but using node 22, after a run or two)

```shell
.....................................................

53 passing (1.216 ms)

=============================== Coverage summary ===============================
Branches     : 100 % (184/184)
Functions    : 98,31 % (116/118) NOK
Lines        : 99,74 % (1.529/1.533)
================================================================================

Uncovered lines:
  /Users/sander/prg/js/virtual-code-owners/src/labeler-yml/generate.ts:102,103,104
  /Users/sander/prg/js/virtual-code-owners/src/virtual-code-owners/anomalies.ts:40

Uncovered functions:
  /Users/sander/prg/js/virtual-code-owners/src/virtual-code-owners/anomalies.ts:40,40
```

## Types of changes

<!--- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Documentation only change
- [ ] Refactor (non-breaking change which fixes an issue without
changing functionality)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Chore (thing that needs doing in e.g. coding infrastructure)
  • Loading branch information
sverweij committed May 11, 2024
1 parent 8233e66 commit 9f61fb9
Show file tree
Hide file tree
Showing 6 changed files with 137 additions and 669 deletions.
9 changes: 0 additions & 9 deletions .c8rc.json

This file was deleted.

13 changes: 10 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ jobs:
matrix:
node-version:
- 18.x
- 20.12.2
- 22.x
platform:
- ubuntu-latest
Expand All @@ -53,12 +54,18 @@ jobs:
- run: npm install
- run: npm run build
- run: npm run depcruise -- --no-cache --progress performance-log
if: always() && matrix.node-version != '20.12.2'
- run: npm test
- name: emit coverage results to step summary
if: always() && matrix.node-version == env.NODE_LATEST
if: always() && matrix.node-version != '20.12.2'
- name: run test & emit results to step summary
# code coverage report on 20.x only, as --experimental-test-coverage on node > 20.12.2
# is not stable (gives varying results over identical runs)
if: always() && matrix.node-version == '20.12.2'
run: |
echo '## Code coverage' >> $GITHUB_STEP_SUMMARY
npx tsx tools/istanbul-json-summary-to-markdown.ts < coverage/coverage-summary.json >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
npm test >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
- name: on pushes to the default branch emit full depcruise results to step summary
if: always() && matrix.node-version == env.NODE_LATEST && github.event_name == 'push' && github.ref_name == github.event.repository.default_branch
run: |
Expand Down
Loading

0 comments on commit 9f61fb9

Please sign in to comment.