Skip to content

e2e-flakiness-detector #192

e2e-flakiness-detector

e2e-flakiness-detector #192

Triggered via schedule September 17, 2024 14:04
Status Failure
Total duration 1h 27m 24s
Artifacts 2
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

20 errors and 2 notices
command-commit.test.ts:24:3 › use terminal output as context: vscode\test\e2e\helpers.ts#L331
1) command-commit.test.ts:24:3 › use terminal output as context ────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceqWWKDZ after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceqWWKDZ\logs\20240917T141225\window1\exthost\extensionTelemetry.log' at helpers.ts:331 329 | } catch (error) { 330 | if (attempts === 1) { > 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 332 | } 333 | 334 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
command-commit.test.ts:24:3 › use terminal output as context: vscode\test\e2e\helpers.ts#L331
2) command-commit.test.ts:24:3 › use terminal output as context ────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceZat4KT after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceZat4KT\logs\20240917T142152\window1\exthost\extensionTelemetry.log' at helpers.ts:331 329 | } catch (error) { 330 | if (attempts === 1) { > 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 332 | } 333 | 334 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
command-commit.test.ts:24:3 › use terminal output as context: vscode\test\e2e\helpers.ts#L331
3) command-commit.test.ts:24:3 › use terminal output as context ────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVtCHMf after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVtCHMf\logs\20240917T143152\window1\exthost\extensionTelemetry.log' at helpers.ts:331 329 | } catch (error) { 330 | if (attempts === 1) { > 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 332 | } 333 | 334 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
command-custom.test.ts:283:17 › use terminal output as context: vscode\test\e2e\helpers.ts#L331
4) command-custom.test.ts:283:17 › use terminal output as context ──────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceuJtbE9 after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceuJtbE9\logs\20240917T141419\window1\exthost\extensionTelemetry.log' at helpers.ts:331 329 | } catch (error) { 330 | if (attempts === 1) { > 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 332 | } 333 | 334 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
command-custom.test.ts:283:17 › use terminal output as context: vscode\test\e2e\helpers.ts#L331
5) command-custom.test.ts:283:17 › use terminal output as context ──────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceyPUGLm after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceyPUGLm\logs\20240917T142345\window1\exthost\extensionTelemetry.log' at helpers.ts:331 329 | } catch (error) { 330 | if (attempts === 1) { > 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 332 | } 333 | 334 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
command-edit.test.ts:133:1 › edit (fixup) input - model selection: vscode\test\e2e\command-edit.test.ts#L149
6) command-edit.test.ts:133:1 › edit (fixup) input - model selection ───────────────────────────── Error: expect(locator).toBeVisible() Locator: getByLabel('$(anthropic-logo) Claude 3.5').locator('a') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('$(anthropic-logo) Claude 3.5').locator('a') - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" 147 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5').locator('a') 148 | await nap() > 149 | expect(modelItem).toBeVisible() | ^ 150 | 151 | // Open the model input and check it has the correct item selected 152 | await modelItem.click() at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:149:23
command-edit.test.ts:133:1 › edit (fixup) input - model selection: vscode\test\e2e\command-edit.test.ts#L154
7) command-edit.test.ts:133:1 › edit (fixup) input - model selection ───────────────────────────── Error: expect(locator).toBeVisible() Locator: getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic') - locator resolved to <div role="option" data-index="0" aria-level="1" aria…>…</div> - unexpected value "hidden" - locator resolved to <div role="option" data-index="0" aria-level="1" aria…>…</div> - unexpected value "hidden" 152 | await modelItem.click() 153 | const selectedModelItem = page.getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic') > 154 | expect(selectedModelItem).toBeVisible() | ^ 155 | }) 156 | at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:154:31
command-edit.test.ts:95:1 › edit (fixup) input - range selection: vscode\test\e2e\command-edit.test.ts#L113
8) command-edit.test.ts:95:1 › edit (fixup) input - range selection ────────────────────────────── Error: expect(locator).toBeVisible() Locator: getByText('Nearest Code Block') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 111 | // Check the correct range item is auto-selected 112 | const rangeItem = page.getByText('Nearest Code Block') > 113 | expect(rangeItem).toBeVisible() | ^ 114 | 115 | // Open the range input and check it has the correct item selected 116 | await rangeItem.click() at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:113:23
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error: vscode\test\e2e\enterprise-cody-ignore.test.ts#L1
9) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error Test timeout of 30000ms exceeded.
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error: vscode\test\e2e\enterprise-cody-ignore.test.ts#L73
9) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error Error: locator.hover: Application exited Call log: - waiting for getByRole('button', { name: 'cody-logo-heavy-slash File Ignored, The current file is ignored by Cody' }) 71 | name: 'cody-logo-heavy-slash File Ignored, The current file is ignored by Cody', 72 | }) > 73 | await statusBarButton.hover() | ^ 74 | await expect(statusBarButton).toBeVisible() 75 | 76 | // Clicking on the Cody icon shows a message at D:\a\cody\cody\vscode\test\e2e\enterprise-cody-ignore.test.ts:73:31
auth.test.ts:22:3 › requires a valid auth token and allows logouts: vscode/test/e2e/auth.test.ts#L1
1) auth.test.ts:22:3 › requires a valid auth token and allows logouts ──────────────────────────── Test timeout of 30000ms exceeded.
auth.test.ts:22:3 › requires a valid auth token and allows logouts: vscode/test/e2e/auth.test.ts#L34
1) auth.test.ts:22:3 › requires a valid auth token and allows logouts ──────────────────────────── Error: locator.click: Application exited Call log: - waiting for getByRole('option', { name: 'Sign In with URL and Access Token' }) 32 | 33 | await sidebar?.getByRole('button', { name: 'Sign In to Your Enterprise Instance' }).click() > 34 | await page.getByRole('option', { name: 'Sign In with URL and Access Token' }).click() | ^ 35 | await page.getByRole('combobox', { name: 'input' }).fill(SERVER_URL) 36 | await page.getByRole('combobox', { name: 'input' }).press('Enter') 37 | await page.getByRole('combobox', { name: 'input' }).fill(VALID_TOKEN) at /home/runner/work/cody/cody/vscode/test/e2e/auth.test.ts:34:83
chat-input.test.ts:26:3 › chat input focus: vscode/test/e2e/chat-input.test.ts#L61
2) chat-input.test.ts:26:3 › chat input focus ──────────────────────────────────────────────────── Error: locator.click: Error: strict mode violation: getByLabel(/Commands \(/) resolved to 3 elements: 1) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Cody: Chat actions').getByLabel('Cody Commands (Alt+C)') 2) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Chat Section').getByLabel('Cody Commands (Alt+C)') 3) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka locator('[id="workbench\\.parts\\.editor"]').getByLabel('Cody Commands (Alt+C)') Call log: - waiting for getByLabel(/Commands \(/) 59 | 60 | // Submit a new chat question from the command menu. > 61 | await page.getByLabel(/Commands \(/).click() | ^ 62 | await page.waitForTimeout(100) 63 | 64 | // HACK: The 'delay' command is used to make sure the response is streamed 400ms after at /home/runner/work/cody/cody/vscode/test/e2e/chat-input.test.ts:61:42
chat-input.test.ts:26:3 › chat input focus: vscode/test/e2e/chat-input.test.ts#L61
3) chat-input.test.ts:26:3 › chat input focus ──────────────────────────────────────────────────── Error: locator.click: Error: strict mode violation: getByLabel(/Commands \(/) resolved to 3 elements: 1) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Cody: Chat actions').getByLabel('Cody Commands (Alt+C)') 2) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Chat Section').getByLabel('Cody Commands (Alt+C)') 3) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka locator('[id="workbench\\.parts\\.editor"]').getByLabel('Cody Commands (Alt+C)') Call log: - waiting for getByLabel(/Commands \(/) 59 | 60 | // Submit a new chat question from the command menu. > 61 | await page.getByLabel(/Commands \(/).click() | ^ 62 | await page.waitForTimeout(100) 63 | 64 | // HACK: The 'delay' command is used to make sure the response is streamed 400ms after at /home/runner/work/cody/cody/vscode/test/e2e/chat-input.test.ts:61:42
chat-input.test.ts:26:3 › chat input focus: vscode/test/e2e/chat-input.test.ts#L61
4) chat-input.test.ts:26:3 › chat input focus ──────────────────────────────────────────────────── Error: locator.click: Error: strict mode violation: getByLabel(/Commands \(/) resolved to 3 elements: 1) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Cody: Chat actions').getByLabel('Cody Commands (Alt+C)') 2) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Chat Section').getByLabel('Cody Commands (Alt+C)') 3) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka locator('[id="workbench\\.parts\\.editor"]').getByLabel('Cody Commands (Alt+C)') Call log: - waiting for getByLabel(/Commands \(/) 59 | 60 | // Submit a new chat question from the command menu. > 61 | await page.getByLabel(/Commands \(/).click() | ^ 62 | await page.waitForTimeout(100) 63 | 64 | // HACK: The 'delay' command is used to make sure the response is streamed 400ms after at /home/runner/work/cody/cody/vscode/test/e2e/chat-input.test.ts:61:42
initial-context.test.ts:83:82 › chat try-again actions: vscode/test/e2e/initial-context.test.ts#L1
5) initial-context.test.ts:83:82 › chat try-again actions ──────────────────────────────────────── Test timeout of 30000ms exceeded.
initial-context.test.ts:83:82 › chat try-again actions: vscode/test/e2e/common.ts#L173
5) initial-context.test.ts:83:82 › chat try-again actions ──────────────────────────────────────── Error: locator.fill: Application exited Call log: - waiting for getByPlaceholder('Type the name of a command to run.') - locator resolved to <input wrap="off" type="text" class="input" role="combo…/> - fill("main.c") - attempting fill action - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #1 - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #2 - waiting 20ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #3 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #4 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #5 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #6 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #7 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #8 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #9 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #10 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #11 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #12 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #13 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #14 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #15 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #16 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #17 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #18 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #19 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #20 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #21 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #22 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill
initial-context.test.ts:83:82 › chat try-again actions: vscode/test/e2e/initial-context.test.ts#L1
6) initial-context.test.ts:83:82 › chat try-again actions ──────────────────────────────────────── Test timeout of 30000ms exceeded.
initial-context.test.ts:83:82 › chat try-again actions: vscode/test/e2e/common.ts#L173
6) initial-context.test.ts:83:82 › chat try-again actions ──────────────────────────────────────── Error: locator.fill: Application exited Call log: - waiting for getByPlaceholder('Type the name of a command to run.') - locator resolved to <input wrap="off" type="text" class="input" role="combo…/> - fill("main.c") - attempting fill action - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #1 - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #2 - waiting 20ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #3 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #4 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #5 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #6 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #7 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #8 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #9 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #10 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #11 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #12 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #13 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #14 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #15 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #16 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #17 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #18 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #19 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #20 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #21 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #22 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill
test-e2e (ubuntu)
Process completed with exit code 1.
🎭 Playwright Run Summary
10 failed command-commit.test.ts:24:3 › use terminal output as context ─────────────────────────────────── command-commit.test.ts:24:3 › use terminal output as context ─────────────────────────────────── command-commit.test.ts:24:3 › use terminal output as context ─────────────────────────────────── command-custom.test.ts:283:17 › use terminal output as context ───────────────────────────────── command-custom.test.ts:283:17 › use terminal output as context ───────────────────────────────── command-edit.test.ts:133:1 › edit (fixup) input - model selection ────────────────────────────── command-edit.test.ts:133:1 › edit (fixup) input - model selection ────────────────────────────── command-edit.test.ts:95:1 › edit (fixup) input - range selection ─────────────────────────────── enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error initial-context.test.ts:83:82 › chat try-again actions ───────────────────────────────────────── 12 skipped 158 passed (38.5m)
🎭 Playwright Run Summary
6 failed auth.test.ts:22:3 › requires a valid auth token and allows logouts ───────────────────────────── chat-input.test.ts:26:3 › chat input focus ───────────────────────────────────────────────────── chat-input.test.ts:26:3 › chat input focus ───────────────────────────────────────────────────── chat-input.test.ts:26:3 › chat input focus ───────────────────────────────────────────────────── initial-context.test.ts:83:82 › chat try-again actions ───────────────────────────────────────── initial-context.test.ts:83:82 › chat try-again actions ───────────────────────────────────────── 30 skipped 414 passed (1.4h)

Artifacts

Produced during runtime
Name Size
playwright-recordings ubuntu
4.64 MB
playwright-recordings windows
1.66 MB