Home » Nodejs » Node.js + mocha + webdriverjs: Failed tests kill suite

Node.js + mocha + webdriverjs: Failed tests kill suite

Posted by: admin November 30, 2017 Leave a comment

Questions:

I am testing a web application using Mocha and WebDriverJS, more or less as decribed here. When the tests pass, everything is fine. However, if one test fails, the rest of the tests in the suite will timeout and the runner will exit at the end of the suite, without closing the Webdriver instance. Example test case:

var assert = require('assert'),
    client = require("webdriverjs").remote({
        logLevel: 'silent'
    });

describe('Self-test', function() {

    before(function(done) {
        client
            .init()
            .url('http://www.wikipedia.org/', function() {
                done();
            });
    });

    after(function(done) {
        client.end(function() {
            done();
        });
    });

    // tests

    it('should fail properly', function(done) {
        client.getTitle(function(result) {
            assert(false, 'This should fail');
            done();
        });
    });

    it('should pass afterwards', function(done) {
        client.getTitle(function(result) {
            assert(true, 'This should still pass');
            done();
        });
    });

});

output:

~> mocha  test/self-test.js

    Self-test
  1) should fail properly
  2) should pass afterwards
  3) "after all" hook

✖ 3 of 2 tests failed:

1) Self-test should fail properly:
   AssertionError: This should fail
    at null.<anonymous> (./test/self-test.js:24:17)
    at QueueItem (./node_modules/webdriverjs/lib/webdriverjs.js:242:15)
    at null.<anonymous> (./node_modules/webdriverjs/lib/commands/getTitle.js:12:6)
    at QueueItem (./node_modules/webdriverjs/lib/webdriverjs.js:242:15)
    at IncomingMessage.WebdriverJs.proxyResponse (./node_modules/webdriverjs/lib/webdriverjs.js:782:6)
    at IncomingMessage.EventEmitter.emit (events.js:115:20)
    at IncomingMessage._emitEnd (http.js:366:10)
    at HTTPParser.parserOnMessageComplete [as onMessageComplete] (http.js:149:23)
    at Socket.socketOnData [as ondata] (http.js:1366:20)
    at TCP.onread (net.js:402:27)

2) Self-test should pass afterwards:
   Error: timeout of 10000ms exceeded
    at Object.<anonymous> (./node_modules/mocha/lib/runnable.js:158:14)
    at Timer.list.ontimeout (timers.js:101:19)

3) Self-test "after all" hook:
   Error: timeout of 10000ms exceeded
    at Object.<anonymous> (./node_modules/mocha/lib/runnable.js:158:14)
    at Timer.list.ontimeout (timers.js:101:19)

As far as I can tell, this is because the WebDriverJS queue gets stalled when a test fails. Is there any way to fix this? It’s sub-optimal even for local command-line testing, and it makes running tests automatically and/or in the background difficult-to-impossible.

Update: I think I can fix the queue failure by instantiating a new client for every test, but this would make things much slower (as the WebDriver instance would need to start up from scratch each time) and would leave WebDriver processes hanging around unkilled on test failure. Ideally, I’d like something like the structure offered by Soda, where a failure somewhere in the queue skips to the end of the queue and then throws the error for the test framework to catch.

Answers:

You should setup and tear down with beforeEach() and afterEach() instead of before() and after() if each test is dependent upon the state of WebDriverJS queue.

Questions:
Answers:

The way webdriverjs is written each test should be a seperate session, and therefore start with init() and end with end(). An exception in your test will bubble up to mocha and mess up the webdriverjs queue. You are therefore trying to us webdriverjs in a way that isn’t supported.

Looking at the source code of soda this also seems to be the case. In client.js:223 exceptions are handled and the error passed back to the callback. This callback is the function set with .end() and therefore not having a .end() for every test will simply skip all following tests of the same session until calling the last test’s .end() callback.

Is this what you mean by ‘a failure somewhere in the queue skips to the end of the queue’? If this is the behaviour you want, you’d have to enclose webdriver.js:244 with a try catch and when an exception is caught, call the .end() command.

Questions:
Answers:

Selenium has their own official webdriverjs bindings that run on node now

http://code.google.com/p/selenium/wiki/WebDriverJs

On npm https://npmjs.org/package/selenium-webdriver

Questions:
Answers:

you would need a listner to catch the uncaught exception

webdriver.promise.controlFlow().on('uncaughtException', function(e) {
     console.error('Unhandled error: ' + e);
     driver.quit();
});

this is what i am doing. The only issue with this is it also catches assertion error.
hope this helps.