Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stops after a failing test #394

Closed
alissa-ferro opened this issue Nov 8, 2017 · 8 comments
Closed

Stops after a failing test #394

alissa-ferro opened this issue Nov 8, 2017 · 8 comments

Comments

@alissa-ferro
Copy link

alissa-ferro commented Nov 8, 2017 •
edited by DanielMSchmidt
Loading

Hey!
When I have a failing test the other tests doesn't run. In my case, there are 4 tests but only 2 are executed.

  • Detox: 5.9.2
  • Node: 8.8.1
  • Device: iPhone 6 (simulator)
  • Xcode: 9.0.1
  • macOS: Sierra 10.12.6

Here are my verbose logs:

MacGeint:BB30Detox Geint$ detox test --loglevel verbose --reuse
node_modules/.bin/mocha e2e --opts e2e/mocha.opts  --loglevel verbose  --reuse
detox-server info 10:52:04: server listening on localhost:50549...
detox verb ws onOpen [object Object]
detox verb ws send: {"type":"login","params":{"sessionId":"16c734c5-cf92-946a-4d87-039faf65b65c","role":"tester"},"messageId":0}
detox verb ws onMessage: {"type":"loginSuccess","params":{"sessionId":"16c734c5-cf92-946a-4d87-039faf65b65c","role":"tester"},"messageId":0}
detox verb ws
detox verb 1: applesimutils --list "iPhone 6" --maxResults=1
detox info 1: Searching for device matching iPhone 6...
detox verb 1: stdout: [
detox verb 1: stdout:   {
detox verb 1: stdout:     "state" : "Booted",
detox verb 1: stdout:     "availability" : "(available)",
detox verb 1: stdout:     "name" : "iPhone 6",
detox verb 1: stdout:     "udid" : "985AF84C-02CD-4127-937E-4D78BAF115D4",
detox verb 1: stdout:     "os" : {
detox verb 1: stdout:       "version" : "11.0.1",
detox verb 1: stdout:       "availability" : "(available)",
detox verb 1: stdout:       "name" : "iOS 11.0",
detox verb 1: stdout:       "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-11-0",
detox verb 1: stdout:       "buildversion" : "15A8401"
detox verb 1: stdout:     }
detox verb 1: stdout:   }
detox verb 1: stdout: ]
detox verb 2: stdout:
detox verb 3: /usr/bin/xcrun simctl terminate 985AF84C-02CD-4127-937E-4D78BAF115D4 com.bb.mobile30piloto
detox info 3: Terminating com.bb.mobile30piloto...
detox info 3: com.bb.mobile30piloto terminated
detox verb 4: /bin/cat /dev/null >$HOME/Library/Developer/CoreSimulator/Devices/985AF84C-02CD-4127-937E-4D78BAF115D4/data/tmp/detox.last_launch_app_log.out 2>$HOME/Library/Developer/CoreSimulator/Devices/985AF84C-02CD-4127-937E-4D78BAF115D4/data/tmp/detox.last_launch_app_log.err && SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/Geint/Library/Detox/ios/e08fce8f9431386e13e9ac2a4666c32c183d7979/Detox.framework/Detox" /usr/bin/xcrun simctl launch --stdout=/tmp/detox.last_launch_app_log.out --stderr=/tmp/detox.last_launch_app_log.err 985AF84C-02CD-4127-937E-4D78BAF115D4 com.bb.mobile30piloto --args -detoxServer ws://localhost:50549 -detoxSessionId 16c734c5-cf92-946a-4d87-039faf65b65c
detox info 4: Launching com.bb.mobile30piloto...
detox verb 4: stdout: com.bb.mobile30piloto: 56360
detox verb 4: stdout:
detox info 4: com.bb.mobile30piloto launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F $HOME/Library/Developer/CoreSimulator/Devices/985AF84C-02CD-4127-937E-4D78BAF115D4/data/tmp/detox.last_launch_app_log.{out,err}
detox verb ws send: {"type":"isReady","params":{},"messageId":-1000}
detox verb ws onMessage: {"type":"ready","messageId":-1000,"params":{}}
detox verb ws
  Consultas
detox verb ws send: {"type":"reactNativeReload","params":{},"messageId":-1000}
detox verb ws onMessage: {"type":"ready","messageId":-1000,"params":{}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"EarlGrey","value":"instance"},"method":"detox_selectElementWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"detoxMatcherForText:","args":[{"type":"NSString","value":"Senha de 8 dígitos"}]}}]}},"method":"assertWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"matcherForSufficientlyVisible","args":[]}}]},"messageId":1}
detox verb ws onMessage: {"type":"invokeResult","messageId":1,"params":{"result":"(GREYElementInteraction)"}}
detox verb ws
    ✓ test (59947ms)
detox verb ws send: {"type":"reactNativeReload","params":{},"messageId":-1000}
detox verb ws onMessage: {"type":"ready","messageId":-1000,"params":{}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"EarlGrey","value":"instance"},"method":"detox_selectElementWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"detoxMatcherForText:","args":[{"type":"NSString","value":"Senha de 8 dígitos"}]}}]}},"method":"performAction:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYActions"},"method":"actionForTypeText:","args":[{"type":"NSString","value":"88888888"}]}}]},"messageId":2}
detox verb ws onMessage: {"type":"invokeResult","messageId":2,"params":{"result":"(GREYElementInteraction)"}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYConfiguration"},"method":"sharedInstance","args":[]}},"method":"disableSynchronization","args":[]},"messageId":3}
detox verb ws onMessage: {"type":"invokeResult","messageId":3,"params":{"result":"(null)"}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"EarlGrey","value":"instance"},"method":"detox_selectElementWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"detox_matcherForAccessibilityLabel:","args":[{"type":"NSString","value":" ENTRAR"}]}}]}},"method":"performAction:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYActions"},"method":"actionForTap","args":[]}}]},"messageId":4}
detox verb ws onMessage: {"type":"invokeResult","messageId":4,"params":{"result":"(GREYElementInteraction)"}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYConfiguration"},"method":"sharedInstance","args":[]}},"method":"enableSynchronization","args":[]},"messageId":5}
detox verb ws onMessage: {"type":"invokeResult","messageId":5,"params":{"result":"(null)"}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"Invocation","value":{"target":{"type":"EarlGrey","value":"instance"},"method":"detox_selectElementWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"matcherForAccessibilityTraits:","args":[{"type":"NSInteger","value":1}]}}]}},"method":"atIndex:","args":[{"type":"NSInteger","value":3}]}},"method":"performAction:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYActions"},"method":"actionForTap","args":[]}}]},"messageId":6}
detox verb ws onMessage: {"type":"invokeResult","messageId":6,"params":{"result":"(GREYElementInteraction)"}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"EarlGrey","value":"instance"},"method":"detox_selectElementWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"detox_matcherForAccessibilityLabel:","args":[{"type":"NSString","value":"Conta corrente"}]}}]}},"method":"performAction:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYActions"},"method":"actionForTap","args":[]}}]},"messageId":7}
detox verb ws onMessage: {"type":"invokeResult","messageId":7,"params":{"result":"(GREYElementInteraction)"}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"Invocation","value":{"target":{"type":"EarlGrey","value":"instance"},"method":"detox_selectElementWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"detox_matcherForAccessibilityLabel:","args":[{"type":"NSString","value":"Saldo de conta-corrente"}]}}]}},"method":"atIndex:","args":[{"type":"NSInteger","value":0}]}},"method":"performAction:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYActions"},"method":"actionForTap","args":[]}}]},"messageId":8}
detox verb ws onMessage: {"type":"invokeResult","messageId":8,"params":{"result":"(GREYElementInteraction)"}}
detox verb ws
detox verb ws send: {"type":"invoke","params":{"target":{"type":"Invocation","value":{"target":{"type":"EarlGrey","value":"instance"},"method":"detox_selectElementWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"detoxMatcherForText:","args":[{"type":"NSString","value":"Transação efetivada!"}]}}]}},"method":"assertWithMatcher:","args":[{"type":"Invocation","value":{"target":{"type":"Class","value":"GREYMatchers"},"method":"matcherForSufficientlyVisible","args":[]}}]},"messageId":9}
detox verb ws onMessage: {"type":"testFailed","messageId":9,"params":{"details":"Cannot find UI Element.\nException with Assertion: {\n  \"Assertion Criteria\" : \"assertWithMatcher:matcherForSufficientlyVisible(>=0.750000)\",\n  \"Element Matcher\" : \"(((((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('Transação efetivada!')) || (kindOfClass('RCTText') && an object with accessibilityLabel \"Transação efetivada!\")) && !(kindOfClass('RCTScrollView'))) || (kindOfClass('UIScrollView') && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && ancestorThatMatches(((((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('Transação efetivada!')) || (kindOfClass('RCTText') && an object with accessibilityLabel \"Transação efetivada!\")) && kindOfClass('RCTScrollView'))))))\",\n  \"Recovery Suggestion\" : \"Check if the element exists in the UI hierarchy printed below. If it exists, adjust the matcher so that it accurately matches element.\"\n}\n\nError Trace: [\n  {\n    \"Description\" : \"Interaction cannot continue because the desired element was not found.\",\n    \"Error Domain\" : \"com.google.earlgrey.ElementInteractionErrorDomain\",\n    \"Error Code\" : \"0\",\n    \"File Name\" : \"GREYElementInteraction.m\",\n    \"Function Name\" : \"-[GREYElementInteraction matchedElementsWithTimeout:error:]\",\n    \"Line\" : \"124\"\n  }\n]"}}
detox verb ws
    1) Saldo conta corrente
detox verb ws send: {"type":"cleanup","params":{"stopRunner":false},"messageId":10}
detox verb ws onMessage: {"type":"cleanupDone","messageId":10,"params":{}}
detox verb ws
  1 passing (1m)
  1 failing
  1) Consultas Saldo conta corrente:
     Error: Error: Cannot find UI Element.
Exception with Assertion: {
  "Assertion Criteria" : "assertWithMatcher:matcherForSufficientlyVisible(>=0.750000)",
  "Element Matcher" : "(((((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('Transação efetivada!')) || (kindOfClass('RCTText') && an object with accessibilityLabel "Transaçãoefetivada!")) && !(kindOfClass('RCTScrollView'))) || (kindOfClass('UIScrollView') && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && ancestorThatMatches(((((kindOfClass('UILabel') || kindOfClass('UITextField') || kindOfClass('UITextView')) && hasText('Transação efetivada!')) || (kindOfClass('RCTText') && an object with accessibilityLabel "Transação efetivada!")) && kindOfClass('RCTScrollView'))))))",
  "Recovery Suggestion" : "Check if the element exists in the UI hierarchy printed below. If it exists, adjust the matcher so that it accurately matches element."
}
Error Trace: [
  {
    "Description" : "Interaction cannot continue because the desired element was not found.",
    "Error Domain" : "com.google.earlgrey.ElementInteractionErrorDomain",
    "Error Code" : "0",
    "File Name" : "GREYElementInteraction.m",
    "Function Name" : "-[GREYElementInteraction matchedElementsWithTimeout:error:]",
    "Line" : "124"
  }
]
      at Client._callee8$ (node_modules/detox/lib/client/Client.js:57:19)
      at tryCatch (node_modules/babel-polyfill/node_modules/regenerator-runtime/runtime.js:65:40)
      at Generator.invoke [as _invoke] (node_modules/babel-polyfill/node_modules/regenerator-runtime/runtime.js:303:22)
      at Generator.prototype.(anonymous function) [as throw] (node_modules/babel-polyfill/node_modules/regenerator-runtime/runtime.js:117:21)
      at step (node_modules/detox/lib/client/Client.js:1:809)
      at node_modules/detox/lib/client/Client.js:1:1008
      at <anonymous>
      at process._tickCallback (internal/process/next_tick.js:188:7)
child_process.js:641
    throw err;
    ^
Error: Command failed: node_modules/.bin/mocha e2e --opts e2e/mocha.opts  --loglevel verbose  --reuse
    at checkExecSyncError (child_process.js:598:13)
    at Object.execSync (child_process.js:638:13)
    at runMocha (/Users/Geint/BB30Detox/node_modules/detox/local-cli/detox-test.js:56:6)
    at Object.<anonymous> (/Users/Geint/BB30Detox/node_modules/detox/local-cli/detox-test.js:36:5)
    at Module._compile (module.js:612:30)
    at Object.Module._extensions..js (module.js:623:10)
    at Module.load (module.js:531:32)
    at tryModuleLoad (module.js:494:12)
    at Function.Module._load (module.js:486:3)
    at Function.Module.runMain (module.js:653:10)
MacGeint:BB30Detox Geint$eraction cannot continue because the desired element was not found.\",\n    \"Error Domain\" : \"com.google.earlgrey.ElementInteractionErrorDomain\",\n    \"Error Code\" : \"0\",\n    \"File Name\" : \"GREYElementInteract
@LeoNatan
Copy link
Contributor

LeoNatan commented Nov 9, 2017

@DanielMSchmidt I am not sure this is a bug. I think it's the intended behavior.
@rotemmiz What say you?

@alissa-ferro
Copy link
Author

@LeoNatan, I don´t think it's intended behavior. At least for me, I wanna now if besides any bug everything else is working or not. That's why makes sense for me testing everything and at the end show what worked and what didn't.

@LeoNatan
Copy link
Contributor

LeoNatan commented Nov 9, 2017 •
edited
Loading

Yes, I understand what you meant. I was just responding to @DanielMSchmidt adding the "bug" label. This could be an "enhancement" or something that we will not implement.

For example, what should the expected behavior be if an app crashes between tests? Crashes are already difficult to handle—we don't handle them properly in Detox right now—what should the behavior be for the rest of the tests? I assume you don't relaunch your app before each test? Changing the existing behavior requires investigating all problems that could arise.

@DanielMSchmidt
Copy link
Contributor

@LeoNatan Sorry, I thought we already had something to handle this scenario. I removed the bug label.

@alissa-ferro
Copy link
Author

Got you.

I used to test my app with Appium, and I'm trying to use detox due my company decided to use react native. With appium, in case of crashes between tests it restart the app and continue. I guess this is the expected, this way you can see if the problem is especifically in one testcase or with something else.

In my user opinion, it's something to be implemented (I really need it... )
I'll wait and see :)

@alissa-ferro alissa-ferro reopened this Nov 9, 2017
@LeoNatan
Copy link
Contributor

LeoNatan commented Nov 9, 2017

Let's keep the issue open for now. Let's see what @rotemmiz says.

@DanielMSchmidt We may have. Awaiting mizmiz comment. 😉

@rotemmiz
Copy link
Member

@alissa-ferro this is out of Detox's scope. The test runner's job to decide if it should bail or continue to test the other tests in the suite.
If you want tests to continue running remove the --bail from your mocha.opts

@alissa-ferro
Copy link
Author

That´s perfect! Thanks!

@wix wix locked and limited conversation to collaborators Jul 23, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@DanielMSchmidt @rotemmiz @LeoNatan @alissa-ferro