Skip to content

chore(test): fix Codegen "should save assets via SIGINT" #36798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

agg23
Copy link
Contributor

@agg23 agg23 commented Jul 25, 2025

I'm unsure if we consider it acceptable for a forceful termination of the browser during this test, but that's what is happening in these test failures. Consider exit code 1 acceptable.

@agg23 agg23 added the CQ1 label Jul 25, 2025

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@agg23 agg23 added CQ1 and removed CQ1 labels Jul 28, 2025

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@agg23 agg23 marked this pull request as ready for review July 28, 2025 18:50
@agg23 agg23 added CQ1 and removed CQ1 labels Jul 28, 2025
Copy link
Contributor

Test results for "tests 1"

1 failed
❌ [default-reuse] › highlight-locators.spec.ts:25:5 › should work @vscode-extension

3 flaky ⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-ubuntu-22.04-node18
⚠️ [webkit-library] › library/browsercontext-viewport-mobile.spec.ts:124:5 › mobile viewport › respect meta viewport tag @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › library/browsercontext-viewport-mobile.spec.ts:206:5 › mobile viewport › view scale should reset after navigation @webkit-ubuntu-22.04-node18

46500 passed, 804 skipped
✔️✔️✔️

Merge workflow run.

Copy link
Contributor

Test results for "tests others"

1 fatal errors, not part of any test
20187 passed, 467 skipped
✔️✔️✔️

Merge workflow run.

Copy link
Contributor

Test results for "tests 2"

8 fatal errors, not part of any test
4 failed
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:93:7 › cli codegen › should not lead to an error if html gets clicked @channel-chromium-macos-latest
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:93:7 › cli codegen › should not lead to an error if html gets clicked @chrome-beta-macos-latest
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:93:7 › cli codegen › should not lead to an error if html gets clicked @chrome-macos-latest
❌ [chromium-library] › library/proxy.spec.ts:238:3 › should use socks proxy in second page @msedge-dev-macos-latest

131 flaky ⚠️ [chromium-library] › library/browsertype-connect.spec.ts:647:5 › launchServer › should record trace with sources @channel-chromium-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28:3 › context.cookies() should work @smoke @channel-chromium-macos-latest
⚠️ [chromium-library] › library/emulation-focus.spec.ts:202:12 › should trigger hover state concurrently @channel-chromium-macos-latest
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @channel-chromium-macos-latest
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @channel-chromium-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-pages.spec.ts:142:3 › should keep selection in multiple pages @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:250:5 › reuse launch › should ignore binding from beforeunload @chrome-beta-macos-latest
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:151:15 › should close service worker together with the context @chrome-beta-macos-latest
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:160:5 › should connect to existing page with iframe and navigate @chrome-beta-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:166:1 › should highlight all @chrome-beta-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:89:3 › should support viewport option @chrome-beta-macos-latest
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @chrome-beta-macos-latest
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @chrome-beta-macos-latest
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:299:5 › should send default User-Agent header with connect request @chrome-beta-windows-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @chrome-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:671:5 › launchServer › should fulfill with global fetch result @chrome-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:739:5 › launchServer › setInputFiles should preserve lastModified timestamp @chrome-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:138:1 › should navigate all @chrome-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:166:1 › should highlight all @chrome-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28:3 › context.cookies() should work @smoke @chrome-macos-latest
⚠️ [chromium-library] › library/emulation-focus.spec.ts:202:12 › should trigger hover state concurrently @chrome-macos-latest
⚠️ [chromium-library] › library/fetch-proxy.spec.ts:21:3 › context request should pick up proxy credentials @chrome-macos-latest
⚠️ [chromium-library] › library/headful.spec.ts:233:3 › should click in OOPIF @chrome-macos-latest
⚠️ [chromium-library] › library/logger.spec.ts:19:3 › should log @smoke @chrome-macos-latest
⚠️ [chromium-library] › library/tracing.spec.ts:208:5 › should respect tracesDir and name @chrome-macos-latest
⚠️ [chromium-library] › library/tracing.spec.ts:369:5 › should survive browser.close with auto-created traces dir @chrome-macos-latest
⚠️ [chromium-library] › library/video.spec.ts:379:5 › screencast › should capture navigation @chrome-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-2.spec.ts:111:3 › should restore state from userDataDir @chromium-headed-macos-14-xlarge
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @chromium-headed-macos-14-xlarge
⚠️ [chromium-library] › library/chromium/tracing.spec.ts:99:3 › should support a buffer without a path @chromium-macos-13-xlarge
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @chromium-tip-of-tree-macos-13
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @chromium-tip-of-tree-macos-13
⚠️ [chromium-library] › library/chromium/oopif.spec.ts:284:3 › should click @chromium-tip-of-tree-macos-13--headed
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @chromium-tip-of-tree-macos-13--headed
⚠️ [chromium-library] › library/inspector/cli-codegen-1.spec.ts:94:7 › cli codegen › should click twice @chromium-tip-of-tree-ubuntu-22.04--headed
⚠️ [chromium-library] › library/trace-viewer.spec.ts:472:1 › should show snapshot URL @chromium-tip-of-tree-ubuntu-22.04--headed
⚠️ [chromium-library] › library/trace-viewer.spec.ts:982:1 › should show action source @chromium-tip-of-tree-ubuntu-22.04--headed
⚠️ [chromium-page] › page/page-drag.spec.ts:103:5 › Drag and drop › should cancel on escape @chromium-tip-of-tree-windows-latest--headed
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-beta-macos-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-aria.spec.ts:76:7 › should update aria snapshot highlight @firefox-beta-macos-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-pick-locator.spec.ts:35:7 › should update locator highlight @firefox-beta-macos-latest
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144:3 › should keep reduced motion and color emulation after reload @firefox-beta-macos-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-beta-ubuntu-22.04
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-beta-windows-latest
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-beta-windows-latest
⚠️ [firefox-library] › library/capabilities.spec.ts:244:3 › requestFullscreen @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/trace-viewer.spec.ts:1646:1 › canvas clipping @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-headed-ubuntu-24.04
⚠️ [firefox-library] › library/browsercontext-add-cookies.spec.ts:207:3 › should isolate persistent cookies @firefox-headed-windows-latest
⚠️ [firefox-library] › library/browsercontext-add-cookies.spec.ts:263:3 › should set multiple cookies @firefox-headed-windows-latest
⚠️ [firefox-library] › library/client-certificates.spec.ts:747:3 › browser › should handle rejected certificate in handshake with HTTP/2 @firefox-headed-windows-latest
⚠️ [firefox-library] › library/har.spec.ts:163:3 › should include form params @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:672:7 › cli codegen › should consume contextmenu events, despite a custom context menu @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:214:7 › should work with --save-har in nunit @firefox-headed-windows-latest
⚠️ [firefox-page] › page/page-aria-snapshot-ai.spec.ts:358:3 › return empty snapshot when iframe is not loaded @firefox-headed-windows-latest
⚠️ [firefox-page] › page/page-click-scroll.spec.ts💯3 › should scroll into view element in iframe @firefox-headed-windows-latest
⚠️ [firefox-page] › page/page-click.spec.ts:919:3 › should not hang when frame is detached @firefox-headed-windows-latest
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-13-large
⚠️ [firefox-library] › library/inspector/cli-codegen-pick-locator.spec.ts:35:7 › should update locator highlight @firefox-macos-13-large
⚠️ [firefox-library] › library/video.spec.ts:379:5 › screencast › should capture navigation @firefox-macos-13-large
⚠️ [firefox-page] › page/page-event-request.spec.ts:182:3 › should return response body when Cross-Origin-Opener-Policy is set @firefox-macos-13-large
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-13-xlarge
⚠️ [firefox-page] › page/page-goto.spec.ts:81:3 › should work with Cross-Origin-Opener-Policy @firefox-macos-13-xlarge
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-14-large
⚠️ [firefox-page] › page/page-event-request.spec.ts:182:3 › should return response body when Cross-Origin-Opener-Policy is set @firefox-macos-14-large
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-14-xlarge
⚠️ [firefox-page] › page/page-event-request.spec.ts:182:3 › should return response body when Cross-Origin-Opener-Policy is set @firefox-macos-14-xlarge
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-macos-14-xlarge
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-ubuntu-24.04
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-ubuntu-24.04
⚠️ [firefox-library] › library/debug-controller.spec.ts:79:1 › should pick element @firefox-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-windows-latest
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-windows-latest
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-fetch.spec.ts:1262:3 › should work with connectOverCDP @msedge-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:114:5 › reuse launch › should reset serviceworker @msedge-beta-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:149:1 › should reset for reuse @msedge-beta-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:142:3 › should support acceptDownloads option @msedge-beta-macos-latest
⚠️ [chromium-library] › library/headful.spec.ts:114:3 › should close browser after context menu was triggered @msedge-beta-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:296:7 › cli codegen › should handle history.postData @msedge-beta-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-beta-macos-latest
⚠️ [chromium-library] › library/video.spec.ts:379:5 › screencast › should capture navigation @msedge-beta-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-beta-ubuntu-22.04
⚠️ [chromium-library] › library/debug-controller.spec.ts:79:1 › should pick element @msedge-beta-windows-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-beta-windows-latest
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-fetch.spec.ts:1262:3 › should work with connectOverCDP @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:94:5 › reuse launch › should re-add binding after reset @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:150:5 › reuse launch › should reset serviceworker that hangs in importScripts @msedge-dev-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:101:3 › should support userAgent option @msedge-dev-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-1.spec.ts:468:7 › cli codegen › should update selected element after pressing Tab @msedge-dev-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:342:7 › cli codegen › should not clash pages @msedge-dev-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-3.spec.ts:226:7 › cli codegen › should generate frame locators (4) @msedge-dev-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-dev-macos-latest
⚠️ [chromium-library] › library/proxy.spec.ts:178:3 › should exclude patterns @msedge-dev-macos-latest
⚠️ [chromium-library] › library/tracing.spec.ts:369:5 › should survive browser.close with auto-created traces dir @msedge-dev-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-dev-ubuntu-22.04
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-macos-latest
⚠️ [chromium-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-cookies-third-party.spec.ts:526:1 › should(not) block third party cookies - persistent context @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-fetch.spec.ts:1262:3 › should work with connectOverCDP @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:94:5 › reuse launch › should re-add binding after reset @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:270:5 › reuse launch › should reset mouse position @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:333:5 › reuse launch › should continue issuing events after closing the reused page @msedge-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:149:1 › should reset for reuse @msedge-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:249:1 › should reset routes before reuse @msedge-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28:3 › context.cookies() should work @smoke @msedge-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-2.spec.ts:70:3 › should support geolocation and permissions options @msedge-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-2.spec.ts:211:3 › should respect selectors @msedge-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-macos-latest
⚠️ [chromium-library] › library/proxy.spec.ts:93:11 › should proxy local network requests › with other bypasses › link-local @msedge-macos-latest
⚠️ [chromium-library] › library/client-certificates.spec.ts:400:3 › browser › should not hang on tls errors during TLS 1.2 handshake @msedge-ubuntu-22.04
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-ubuntu-22.04
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @tracing-firefox
⚠️ [webkit-library] › library/browsercontext-reuse.spec.ts:114:5 › reuse launch › should reset serviceworker @webkit-headed-windows-latest
⚠️ [webkit-library] › library/inspector/cli-codegen-3.spec.ts:74:7 › cli codegen › should click locator.nth @webkit-headed-windows-latest
⚠️ [webkit-library] › library/inspector/cli-codegen-3.spec.ts:640:7 › cli codegen › should consume pointer events @webkit-headed-windows-latest
⚠️ [webkit-library] › library/inspector/cli-codegen-pick-locator.spec.ts:35:7 › should update locator highlight @webkit-headed-windows-latest
⚠️ [webkit-page] › page/page-click.spec.ts:251:3 › should click on checkbox input and toggle @webkit-headed-windows-latest
⚠️ [webkit-page] › page/wheel.spec.ts:70:3 › should dispatch wheel events after context menu was opened @webkit-macos-14-xlarge
⚠️ [webkit-library] › library/trace-viewer.spec.ts:1313:1 › should pick locator in iframe @webkit-macos-15-large
⚠️ [webkit-page] › page/page-request-continue.spec.ts:191:5 › post data › should amend post data @webkit-macos-15-large
⚠️ [webkit-page] › page/wheel.spec.ts:70:3 › should dispatch wheel events after context menu was opened @webkit-macos-15-xlarge
⚠️ [webkit-library] › library/browsercontext-reuse.spec.ts:114:5 › reuse launch › should reset serviceworker @webkit-windows-latest

239960 passed, 9548 skipped
✔️✔️✔️

Merge workflow run.

@pavelfeldman
Copy link
Member

I think there is an underlying cause, I don't remember what it is, but it is not easy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants