Combination of component and e2e test seems random. vite-plugin-istanbul

[hamidmayeli]

Logs and screenshots
I won't be able to add logs as it's over GitHub limits

Versions

• What is this plugin's version? ^3.11.0
• What is Cypress version? ^12.17.2
• What is your operating system? Windows 10 x64 / Linux Ubuntu
• What is the shell?
• What is the Node version? 20.3.1
• What is the NPM version? 9.7.2
• How do you instrument your application? Using vite-plugin-istanbul verionn ^5.0.0.
• When running tests, if you open the web application in regular browser, and open DevTools, do you see window.__coverage__ object? Yes
• Is there .nyc_output folder? Is there .nyc_output/out.json file. Is it empty? It's not empty and has +31K lines content.
• Do you have any custom NYC settings in package.json (nyc object) or in other [NYC config files] Yes

   istanbul({
        include: 'src/*',
        exclude: ['node_modules'],
        extension: [ '.js', '.ts', '.jsx', '.tsx',  ],
        requireEnv: !process.env.CYPRESS,
        checkProd: true,
      }),
  

• Do you run Cypress tests in a Docker container? Yes

Describe the bug
I am trying to combine the unit test coverage and e2e. But the coverage is different in different environments. I have tested this in 3 different environments.

1. My Windows (Win 10 x64) machine, which normally shows 5 uncovered branches. FYI, I delete .nyc and coverage folder before running the commands.
2. A docker file which I use to test with. This one show 3 uncovered branches.
3. The pipeline (GitHub Actions) which build the same docker file. This one show 8 uncovered branches.

I have RUN npm run components:ci && npm run e2e:ci && npm run build in my docker file and run the same commands in the same order in windows too. However, if I run the e2e before components I will end up with 32 uncovered branches.

I am testing a page using component test which results in 100% code coverage. When I run the e2e (to test the routing to this page), it will be reduced to just over 80%.

Link to the repo
Sadly, I cannot make the project public, and if I remove the codes/tests the issue with go away. But the setup can be find in this template. But, I will be able to give access to a contributor.

package.json

{
  "name": "my-vite-app",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "concurrently --kill-others \"npx json-server -w json-server/data.cjs -r json-server/routing.json -m json-server/middleware.cjs --host 127.0.0.1 -p 4000\" \"vite\"",
    "build": "tsc && vite build",
    "lint": "eslint src --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
    "preview": "vite preview",
    "integration": "start-server-and-test dev \"http://localhost:5173|http://127.0.0.1:4000\" \"cypress open --e2e\"",
    "integration:ci": "start-server-and-test dev \"http://localhost:5173|http://127.0.0.1:4000\" \"cypress run --e2e\"",
    "unit": "cypress open --component",
    "unit:ci": "cypress run --component"
  },
  "dependencies": {
    "jalaali-js": "^1.2.6",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-easy-crop": "^5.0.0",
    "react-router-dom": "^6.14.2"
  },
  "devDependencies": {
    "@cypress/code-coverage": "^3.11.0",
    "@faker-js/faker": "^8.0.2",
    "@types/jalaali-js": "^1.1.1",
    "@types/react": "^18.2.16",
    "@types/react-dom": "^18.2.7",
    "@typescript-eslint/eslint-plugin": "^6.2.0",
    "@typescript-eslint/parser": "^6.2.0",
    "@vitejs/plugin-react": "^4.0.3",
    "autoprefixer": "^10.4.14",
    "concurrently": "^8.2.0",
    "cypress": "^12.17.2",
    "cypress-file-upload": "^5.0.8",
    "eslint": "^8.45.0",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-react-refresh": "^0.4.1",
    "json-server": "^0.17.3",
    "postcss": "^8.4.27",
    "start-server-and-test": "^2.0.0",
    "tailwindcss": "^3.3.3",
    "typescript": "^5.0.2",
    "vite": "^4.4.7",
    "vite-plugin-istanbul": "^5.0.0",
    "vite-plugin-pwa": "^0.16.4",
    "vite-tsconfig-paths": "^4.2.0"
  }
}

vite.ts

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react';
import tsconfigPaths from 'vite-tsconfig-paths';
import istanbul from 'vite-plugin-istanbul';
import { VitePWA } from 'vite-plugin-pwa'

// https://vitejs.dev/config/
export default defineConfig({
  build: {
    sourcemap: true,
  },
  plugins: [
    react(),
    tsconfigPaths(),
    istanbul({
      include: 'src/*',
      exclude: ['node_modules'],
      extension: [ '.js', '.ts', '.jsx', '.tsx',  ],
      requireEnv: !process.env.CYPRESS,
      checkProd: true,
    }),
    VitePWA({ 
      registerType: 'autoUpdate',
      injectRegister: 'auto',
      includeAssets: ['favicon.ico', 'apple-touch-icon.png', 'mask-icon.svg'],
      manifestFilename: "manifest.json",
      manifest: {
        name: '*****',
        short_name: '****',
        description: '*****',
        theme_color: '#ffffff',
        icons: [
          {
            src: 'logo192.png',
            sizes: '192x192',
            type: 'image/png'
          },
          {
            src: 'logo512.png',
            sizes: '512x512',
            type: 'image/png'
          }
        ]
      }
     }),
  ],
})

cypress.config.ts

import { defineConfig } from "cypress";
import registerCodeCoverageTasks from "@cypress/code-coverage/task";

export default defineConfig({
  e2e: {
    screenshotOnRunFailure: false,
    video: false,
    retries: {
      runMode: 2,
    },
    baseUrl: "http://localhost:5173",
    setupNodeEvents(on, config) {
      registerCodeCoverageTasks(on, config);
      // implement node event listeners here

      return config;
    },
  },

  component: {
    screenshotOnRunFailure: false,
    retries: {
      runMode: 2,
    },
    video: false,
    devServer: {
      framework: "react",
      bundler: "vite",
    },
    setupNodeEvents(on, config) {
      registerCodeCoverageTasks(on, config);
      // implement node event listeners here

      return config;
    },
  },
});

Dockerfile

FROM node:20-slim as build

RUN apt-get update && apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb jq -y

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .
RUN npm run unit:ci && npm run integration:ci && npm run build

RUN echo "Current branch coverage is $(cat ./coverage/coverage-summary.json | jq ".total.branches.total - .total.branches.covered")."

RUN [ $(cat ./coverage/coverage-summary.json | jq ".total.branches.total - .total.branches.covered") -le 4 ] \
   || touch ./coverage/failed.flag

FROM scratch as test-result
COPY --from=build /app/coverage /

FROM scratch as deployable
COPY --from=build /app/dist /

FROM nginx:1.25-alpine-slim as deployment
COPY ./nginx.conf /etc/nginx/conf.d/default.conf
CMD ["/bin/sh", "-c", "nginx -g 'daemon off;'"]
COPY --from=build /app/dist /usr/share/nginx/html

[nagash77]

Hi @hamidmayeli issues in the repo are reserved for bugs and feature requests. Please check out the Cypress Discord Community , it can be helpful for debugging or answering questions on how to use Cypress and Cypress plugins.