chore: refactor e2e tests to playwright #6540
Annotations
10 errors and 1 notice
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L83
1) [chromium] › server.test.js:62:7 › server option › as string › http › should handle GET request to index route (/)
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium/server-option-as-string-ht-299e6-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium/server-option-as-string-ht-299e6-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
81 | expect(response.status()).toBe(200);
82 |
> 83 | await expect(page).toHaveScreenshot();
| ^
84 |
85 | expect(consoleMessages.map((message) => message.text())).toMatchSnapshotWithArray("console messages");
86 |
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:83:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L83
1) [chromium] › server.test.js:62:7 › server option › as string › http › should handle GET request to index route (/)
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium-retry1/server-option-as-string-ht-299e6-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium-retry1/server-option-as-string-ht-299e6-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
81 | expect(response.status()).toBe(200);
82 |
> 83 | await expect(page).toHaveScreenshot();
| ^
84 |
85 | expect(consoleMessages.map((message) => message.text())).toMatchSnapshotWithArray("console messages");
86 |
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:83:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L83
1) [chromium] › server.test.js:62:7 › server option › as string › http › should handle GET request to index route (/)
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium-retry2/server-option-as-string-ht-299e6-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium-retry2/server-option-as-string-ht-299e6-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
81 | expect(response.status()).toBe(200);
82 |
> 83 | await expect(page).toHaveScreenshot();
| ^
84 |
85 | expect(consoleMessages.map((message) => message.text())).toMatchSnapshotWithArray("console messages");
86 |
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:83:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L83
1) [chromium] › server.test.js:62:7 › server option › as string › http › should handle GET request to index route (/)
Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium-retry3/server-option-as-string-ht-299e6-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-5962b-GET-request-to-index-route--chromium-retry3/server-option-as-string-ht-299e6-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
81 | expect(response.status()).toBe(200);
82 |
> 83 | await expect(page).toHaveScreenshot();
| ^
84 |
85 | expect(consoleMessages.map((message) => message.text())).toMatchSnapshotWithArray("console messages");
86 |
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:83:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L138
2) [chromium] › server.test.js:117:7 › server option › as string › custom-http › should handle GET request to index route (/)
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-custom-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium/server-option-as-string-cu-7ce33-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium/server-option-as-string-cu-7ce33-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
136 | expect(response.status()).toBe(200);
137 |
> 138 | await expect(page).toHaveScreenshot();
| ^
139 |
140 | expect(
141 | consoleMessages.map((message) => message.text())
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:138:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L138
2) [chromium] › server.test.js:117:7 › server option › as string › custom-http › should handle GET request to index route (/)
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-custom-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium-retry1/server-option-as-string-cu-7ce33-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium-retry1/server-option-as-string-cu-7ce33-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
136 | expect(response.status()).toBe(200);
137 |
> 138 | await expect(page).toHaveScreenshot();
| ^
139 |
140 | expect(
141 | consoleMessages.map((message) => message.text())
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:138:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L138
2) [chromium] › server.test.js:117:7 › server option › as string › custom-http › should handle GET request to index route (/)
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-custom-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium-retry2/server-option-as-string-cu-7ce33-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium-retry2/server-option-as-string-cu-7ce33-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
136 | expect(response.status()).toBe(200);
137 |
> 138 | await expect(page).toHaveScreenshot();
| ^
139 |
140 | expect(
141 | consoleMessages.map((message) => message.text())
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:138:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L138
2) [chromium] › server.test.js:117:7 › server option › as string › custom-http › should handle GET request to index route (/)
Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-custom-http-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium-retry3/server-option-as-string-cu-7ce33-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-f28a2-GET-request-to-index-route--chromium-retry3/server-option-as-string-cu-7ce33-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
136 | expect(response.status()).toBe(200);
137 |
> 138 | await expect(page).toHaveScreenshot();
| ^
139 |
140 | expect(
141 | consoleMessages.map((message) => message.text())
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:138:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L195
3) [chromium] › server.test.js:174:7 › server option › as string › https › should handle GET request to index route (/)
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-https-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-fa7de-GET-request-to-index-route--chromium/server-option-as-string-ht-6d576-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-fa7de-GET-request-to-index-route--chromium/server-option-as-string-ht-6d576-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
193 | expect(response.status()).toBe(200);
194 |
> 195 | await expect(page).toHaveScreenshot();
| ^
196 |
197 | expect(consoleMessages.map((message) => message.text())).toMatchSnapshotWithArray("console messages");
198 |
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:195:28
|
|
Run E2E tests for webpack version latest:
test/e2e/server.test.js#L195
3) [chromium] › server.test.js:174:7 › server option › as string › https › should handle GET request to index route (/)
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
69 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/__snapshots__/server.test.js/server-option-as-string-https-should-handle-GET-request-to-index-route-1.png
Received: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-fa7de-GET-request-to-index-route--chromium-retry1/server-option-as-string-ht-6d576-equest-to-index-route-1-actual.png
Diff: /Users/runner/work/webpack-dev-server/webpack-dev-server/test-results/server-server-option-as-st-fa7de-GET-request-to-index-route--chromium-retry1/server-option-as-string-ht-6d576-equest-to-index-route-1-diff.png
Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 69 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 69 pixels (ratio 0.01 of all image pixels) are different.
193 | expect(response.status()).toBe(200);
194 |
> 195 | await expect(page).toHaveScreenshot();
| ^
196 |
197 | expect(consoleMessages.map((message) => message.text())).toMatchSnapshotWithArray("console messages");
198 |
at /Users/runner/work/webpack-dev-server/webpack-dev-server/test/e2e/server.test.js:195:28
|
|
Run E2E tests for webpack version latest
40 failed
[chromium] › server.test.js:62:7 › server option › as string › http › should handle GET request to index route (/)
[chromium] › server.test.js:117:7 › server option › as string › custom-http › should handle GET request to index route (/)
[chromium] › server.test.js:174:7 › server option › as string › https › should handle GET request to index route (/)
[chromium] › server.test.js:229:7 › server option › as string › spdy › should handle GET request to index route (/)
[chromium] › server.test.js:323:7 › server option › as object › ca, pfx, key and cert are array of buffers › should handle GET request to index route (/)
[chromium] › server.test.js:403:7 › server option › as object › ca, pfx, key and cert are strings › should handle GET request to index route (/)
[chromium] › server.test.js:492:7 › server option › as object › ca, pfx, key and cert are array of strings › should handle GET request to index route (/)
[chromium] › server.test.js:558:7 › server option › as object › ca, pfx, key and cert are paths to files › should handle GET request to index route (/)
[chromium] › server.test.js:624:7 › server option › as object › ca, pfx, key and cert are array of paths to files › should handle GET request to index route (/)
[chromium] › server.test.js:771:7 › server option › as object › ca, pfx, key and cert are buffer › should handle GET request to index route (/)
[chromium] › server.test.js:852:7 › server option › as object › ca, pfx, key and cert are buffer, key and pfx are objects › should handle GET request to index route (/)
[chromium] › server.test.js:938:7 › server option › as object › ca, pfx, key and cert are strings, key and pfx are objects › should handle GET request to index route (/)
[chromium] › server.test.js:1015:7 › server option › as object › allow to pass more options › should handle GET request to index route (/)
[chromium] › server.test.js:1147:7 › server option › as object › spdy server with options › should handle GET request to index route (/)
[chromium] › server.test.js:1216:7 › server option › as object › custom server with options › should handle GET request to index route (/)
[chromium] › setup-middlewares.test.js:81:3 › setupMiddlewares option › should handle GET request to /setup-middleware/some/path route
[chromium] › setup-middlewares.test.js:159:3 › setupMiddlewares option › should handle POST request to /setup-middleware/some/path route
[chromium] › static-directory.test.js:52:5 › static.directory option › to directory › should handle request to index route
[chromium] › static-directory.test.js:76:5 › static.directory option › to directory › should handle request to other file
[chromium] › static-directory.test.js:199:5 › static.directory option › test listing files in folders without index.html using the option static.serveIndex: false › should show Heyo. because bar has index.html inside it (200)
[chromium] › static-directory.test.js:286:5 › static.directory option › test listing files in folders without index.html using the option static.serveIndex: true › should show Heyo. because bar has index.html inside it (200)
[chromium] › static-directory.test.js:372:5 › static.directory option › test listing files in folders without index.html using the default static.serveIndex option (true) › should show Heyo. because bar has index.html inside it (200)
[chromium] › static-directory.test.js:426:5 › static.directory option › to multiple directories › should handle request first directory
[chromium] › static-directory.test.js:450:5 › static.directory option › to multiple directories › should handle request to second directory
[chromium] › static-directory.test.js:598:5 › static.directory option › defaults to PWD › should handle request to /index.html
[chromium] › static-public-path.test.js:50:5 › static.publicPath option › to directory › should handle request to index
[chromium] › static-public-path.test.js:77:5 › static.publicPath option › to directory › s
|
Loading