e2e-flakiness-detector #192
Annotations
20 errors and 2 notices
command-commit.test.ts:24:3 › use terminal output as context:
vscode\test\e2e\helpers.ts#L331
1) command-commit.test.ts:24:3 › use terminal output as context ──────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceqWWKDZ after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceqWWKDZ\logs\20240917T141225\window1\exthost\extensionTelemetry.log'
at helpers.ts:331
329 | } catch (error) {
330 | if (attempts === 1) {
> 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
332 | }
333 |
334 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
|
command-commit.test.ts:24:3 › use terminal output as context:
vscode\test\e2e\helpers.ts#L331
2) command-commit.test.ts:24:3 › use terminal output as context ──────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceZat4KT after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceZat4KT\logs\20240917T142152\window1\exthost\extensionTelemetry.log'
at helpers.ts:331
329 | } catch (error) {
330 | if (attempts === 1) {
> 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
332 | }
333 |
334 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
|
command-commit.test.ts:24:3 › use terminal output as context:
vscode\test\e2e\helpers.ts#L331
3) command-commit.test.ts:24:3 › use terminal output as context ──────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVtCHMf after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVtCHMf\logs\20240917T143152\window1\exthost\extensionTelemetry.log'
at helpers.ts:331
329 | } catch (error) {
330 | if (attempts === 1) {
> 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
332 | }
333 |
334 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
|
command-custom.test.ts:283:17 › use terminal output as context:
vscode\test\e2e\helpers.ts#L331
4) command-custom.test.ts:283:17 › use terminal output as context ────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceuJtbE9 after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceuJtbE9\logs\20240917T141419\window1\exthost\extensionTelemetry.log'
at helpers.ts:331
329 | } catch (error) {
330 | if (attempts === 1) {
> 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
332 | }
333 |
334 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
|
command-custom.test.ts:283:17 › use terminal output as context:
vscode\test\e2e\helpers.ts#L331
5) command-custom.test.ts:283:17 › use terminal output as context ────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceyPUGLm after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceyPUGLm\logs\20240917T142345\window1\exthost\extensionTelemetry.log'
at helpers.ts:331
329 | } catch (error) {
330 | if (attempts === 1) {
> 331 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
332 | }
333 |
334 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:331:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:212:13)
|
command-edit.test.ts:133:1 › edit (fixup) input - model selection:
vscode\test\e2e\command-edit.test.ts#L149
6) command-edit.test.ts:133:1 › edit (fixup) input - model selection ─────────────────────────────
Error: expect(locator).toBeVisible()
Locator: getByLabel('$(anthropic-logo) Claude 3.5').locator('a')
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('$(anthropic-logo) Claude 3.5').locator('a')
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
147 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5').locator('a')
148 | await nap()
> 149 | expect(modelItem).toBeVisible()
| ^
150 |
151 | // Open the model input and check it has the correct item selected
152 | await modelItem.click()
at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:149:23
|
command-edit.test.ts:133:1 › edit (fixup) input - model selection:
vscode\test\e2e\command-edit.test.ts#L154
7) command-edit.test.ts:133:1 › edit (fixup) input - model selection ─────────────────────────────
Error: expect(locator).toBeVisible()
Locator: getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic')
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic')
- locator resolved to <div role="option" data-index="0" aria-level="1" aria…>…</div>
- unexpected value "hidden"
- locator resolved to <div role="option" data-index="0" aria-level="1" aria…>…</div>
- unexpected value "hidden"
152 | await modelItem.click()
153 | const selectedModelItem = page.getByLabel('check anthropic-logo Claude 3.5 Sonnet, by Anthropic')
> 154 | expect(selectedModelItem).toBeVisible()
| ^
155 | })
156 |
at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:154:31
|
command-edit.test.ts:95:1 › edit (fixup) input - range selection:
vscode\test\e2e\command-edit.test.ts#L113
8) command-edit.test.ts:95:1 › edit (fixup) input - range selection ──────────────────────────────
Error: expect(locator).toBeVisible()
Locator: getByText('Nearest Code Block')
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('Nearest Code Block')
111 | // Check the correct range item is auto-selected
112 | const rangeItem = page.getByText('Nearest Code Block')
> 113 | expect(rangeItem).toBeVisible()
| ^
114 |
115 | // Open the range input and check it has the correct item selected
116 | await rangeItem.click()
at D:\a\cody\cody\vscode\test\e2e\command-edit.test.ts:113:23
|
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error:
vscode\test\e2e\enterprise-cody-ignore.test.ts#L1
9) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error
Test timeout of 30000ms exceeded.
|
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error:
vscode\test\e2e\enterprise-cody-ignore.test.ts#L73
9) enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error
Error: locator.hover: Application exited
Call log:
- waiting for getByRole('button', { name: 'cody-logo-heavy-slash File Ignored, The current file is ignored by Cody' })
71 | name: 'cody-logo-heavy-slash File Ignored, The current file is ignored by Cody',
72 | })
> 73 | await statusBarButton.hover()
| ^
74 | await expect(statusBarButton).toBeVisible()
75 |
76 | // Clicking on the Cody icon shows a message
at D:\a\cody\cody\vscode\test\e2e\enterprise-cody-ignore.test.ts:73:31
|
auth.test.ts:22:3 › requires a valid auth token and allows logouts:
vscode/test/e2e/auth.test.ts#L1
1) auth.test.ts:22:3 › requires a valid auth token and allows logouts ────────────────────────────
Test timeout of 30000ms exceeded.
|
auth.test.ts:22:3 › requires a valid auth token and allows logouts:
vscode/test/e2e/auth.test.ts#L34
1) auth.test.ts:22:3 › requires a valid auth token and allows logouts ────────────────────────────
Error: locator.click: Application exited
Call log:
- waiting for getByRole('option', { name: 'Sign In with URL and Access Token' })
32 |
33 | await sidebar?.getByRole('button', { name: 'Sign In to Your Enterprise Instance' }).click()
> 34 | await page.getByRole('option', { name: 'Sign In with URL and Access Token' }).click()
| ^
35 | await page.getByRole('combobox', { name: 'input' }).fill(SERVER_URL)
36 | await page.getByRole('combobox', { name: 'input' }).press('Enter')
37 | await page.getByRole('combobox', { name: 'input' }).fill(VALID_TOKEN)
at /home/runner/work/cody/cody/vscode/test/e2e/auth.test.ts:34:83
|
chat-input.test.ts:26:3 › chat input focus:
vscode/test/e2e/chat-input.test.ts#L61
2) chat-input.test.ts:26:3 › chat input focus ────────────────────────────────────────────────────
Error: locator.click: Error: strict mode violation: getByLabel(/Commands \(/) resolved to 3 elements:
1) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Cody: Chat actions').getByLabel('Cody Commands (Alt+C)')
2) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Chat Section').getByLabel('Cody Commands (Alt+C)')
3) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka locator('[id="workbench\\.parts\\.editor"]').getByLabel('Cody Commands (Alt+C)')
Call log:
- waiting for getByLabel(/Commands \(/)
59 |
60 | // Submit a new chat question from the command menu.
> 61 | await page.getByLabel(/Commands \(/).click()
| ^
62 | await page.waitForTimeout(100)
63 |
64 | // HACK: The 'delay' command is used to make sure the response is streamed 400ms after
at /home/runner/work/cody/cody/vscode/test/e2e/chat-input.test.ts:61:42
|
chat-input.test.ts:26:3 › chat input focus:
vscode/test/e2e/chat-input.test.ts#L61
3) chat-input.test.ts:26:3 › chat input focus ────────────────────────────────────────────────────
Error: locator.click: Error: strict mode violation: getByLabel(/Commands \(/) resolved to 3 elements:
1) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Cody: Chat actions').getByLabel('Cody Commands (Alt+C)')
2) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Chat Section').getByLabel('Cody Commands (Alt+C)')
3) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka locator('[id="workbench\\.parts\\.editor"]').getByLabel('Cody Commands (Alt+C)')
Call log:
- waiting for getByLabel(/Commands \(/)
59 |
60 | // Submit a new chat question from the command menu.
> 61 | await page.getByLabel(/Commands \(/).click()
| ^
62 | await page.waitForTimeout(100)
63 |
64 | // HACK: The 'delay' command is used to make sure the response is streamed 400ms after
at /home/runner/work/cody/cody/vscode/test/e2e/chat-input.test.ts:61:42
|
chat-input.test.ts:26:3 › chat input focus:
vscode/test/e2e/chat-input.test.ts#L61
4) chat-input.test.ts:26:3 › chat input focus ────────────────────────────────────────────────────
Error: locator.click: Error: strict mode violation: getByLabel(/Commands \(/) resolved to 3 elements:
1) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Cody: Chat actions').getByLabel('Cody Commands (Alt+C)')
2) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka getByLabel('Chat Section').getByLabel('Cody Commands (Alt+C)')
3) <a tabindex="0" role="button" aria-label="Cody Comm…></a> aka locator('[id="workbench\\.parts\\.editor"]').getByLabel('Cody Commands (Alt+C)')
Call log:
- waiting for getByLabel(/Commands \(/)
59 |
60 | // Submit a new chat question from the command menu.
> 61 | await page.getByLabel(/Commands \(/).click()
| ^
62 | await page.waitForTimeout(100)
63 |
64 | // HACK: The 'delay' command is used to make sure the response is streamed 400ms after
at /home/runner/work/cody/cody/vscode/test/e2e/chat-input.test.ts:61:42
|
initial-context.test.ts:83:82 › chat try-again actions:
vscode/test/e2e/initial-context.test.ts#L1
5) initial-context.test.ts:83:82 › chat try-again actions ────────────────────────────────────────
Test timeout of 30000ms exceeded.
|
initial-context.test.ts:83:82 › chat try-again actions:
vscode/test/e2e/common.ts#L173
5) initial-context.test.ts:83:82 › chat try-again actions ────────────────────────────────────────
Error: locator.fill: Application exited
Call log:
- waiting for getByPlaceholder('Type the name of a command to run.')
- locator resolved to <input wrap="off" type="text" class="input" role="combo…/>
- fill("main.c")
- attempting fill action
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #1
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #2
- waiting 20ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #3
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #4
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #5
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #6
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #7
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #8
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #9
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #10
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #11
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #12
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #13
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #14
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #15
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #16
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #17
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #18
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #19
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #20
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #21
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #22
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill
|
initial-context.test.ts:83:82 › chat try-again actions:
vscode/test/e2e/initial-context.test.ts#L1
6) initial-context.test.ts:83:82 › chat try-again actions ────────────────────────────────────────
Test timeout of 30000ms exceeded.
|
initial-context.test.ts:83:82 › chat try-again actions:
vscode/test/e2e/common.ts#L173
6) initial-context.test.ts:83:82 › chat try-again actions ────────────────────────────────────────
Error: locator.fill: Application exited
Call log:
- waiting for getByPlaceholder('Type the name of a command to run.')
- locator resolved to <input wrap="off" type="text" class="input" role="combo…/>
- fill("main.c")
- attempting fill action
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #1
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #2
- waiting 20ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #3
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #4
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #5
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #6
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #7
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #8
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #9
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #10
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #11
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #12
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #13
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #14
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #15
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #16
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #17
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #18
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #19
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #20
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #21
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #22
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill
|
test-e2e (ubuntu)
Process completed with exit code 1.
|
🎭 Playwright Run Summary
10 failed
command-commit.test.ts:24:3 › use terminal output as context ───────────────────────────────────
command-commit.test.ts:24:3 › use terminal output as context ───────────────────────────────────
command-commit.test.ts:24:3 › use terminal output as context ───────────────────────────────────
command-custom.test.ts:283:17 › use terminal output as context ─────────────────────────────────
command-custom.test.ts:283:17 › use terminal output as context ─────────────────────────────────
command-edit.test.ts:133:1 › edit (fixup) input - model selection ──────────────────────────────
command-edit.test.ts:133:1 › edit (fixup) input - model selection ──────────────────────────────
command-edit.test.ts:95:1 › edit (fixup) input - range selection ───────────────────────────────
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error
initial-context.test.ts:83:82 › chat try-again actions ─────────────────────────────────────────
12 skipped
158 passed (38.5m)
|
🎭 Playwright Run Summary
6 failed
auth.test.ts:22:3 › requires a valid auth token and allows logouts ─────────────────────────────
chat-input.test.ts:26:3 › chat input focus ─────────────────────────────────────────────────────
chat-input.test.ts:26:3 › chat input focus ─────────────────────────────────────────────────────
chat-input.test.ts:26:3 › chat input focus ─────────────────────────────────────────────────────
initial-context.test.ts:83:82 › chat try-again actions ─────────────────────────────────────────
initial-context.test.ts:83:82 › chat try-again actions ─────────────────────────────────────────
30 skipped
414 passed (1.4h)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
playwright-recordings ubuntu
|
4.64 MB |
|
playwright-recordings windows
|
1.66 MB |
|