What is Nightwatch?

Nightwatch.js is an automated testing framework for web application and websites, written in Node.js and using the Selenium WebDriver API.

It is a complete browser automation (End-to-End) solution which aims to simplify the process of setting up Continous Integration and writing of automated tests.

Nightwatch got its name from the famous painting The Night Watch by Dutch painter Rembrandt van Rijn. The masterpiece is prominently displayed in the Rijksmuseum, in Amsterdam - The Netherlands.

Overview of Selenium

Selenium is a very popular and comprehensive set of tools for browser automation, initially written for Java but now with support for most programming languages.

Selenium's main projects are:

Nightwatch uses the Selenium WebDriver, specifically the WebDriver Wire Protocol to perform the browser automation related tasks.

Theory of Operation

Nightwatch works by sending HTTP requests to the Selenium server with the right parameters and interpreting the response. The restful API protocol is defined the Selenium JsonWireProtocol. See below for an example workflow for browser initialization.

Most of the times, Nightwatch needs to send at least 2 requests to the Selenium server in order to perform a command or assetion, the first one being the request to locate an element given a CSS selector (or Xpath expression) and the next to perform the actual command/assertion on the given element.

Install Node.js

From nodejs.org:

"Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices."
There are installation packages and instructions for most major Operating systems on its website nodejs.org. Remember to install also the npm tool, which is the node package manager and is distributed with the Node.js installer.

Install Nightwatch

To install the latest version using the npm command line tool, run the following:

$ npm install nightwatch

To make nightwatch runner available globally in your system, add the -g option after npm install and sudo if you're on Mac OS X / Linux or other Unix based OS.

Running the Selenium Server

The Selenium WebDriver server is simply a Java servlet which runs separately on the machine with the browser you want to test.

Download Selenium

Download the latest version of the selenium-server-standalone-{VERSION}.jar file from the Selenium downloads page and place it on the computer with the browser you want to test.

Running Selenium Automatically

If the server is on the same machine where Nightwatch is running, it can be started/stopped directly by the Nightwatch Test Runner.

Running Selenium Manually

To run the selenium server manually, from the directory with the jar run the following:

$ java -jar selenium-server-standalone-{VERSION}.jar
More info about running the Selenium server can be found here:
http://code.google.com/p/selenium/wiki/RemoteWebDriverServer

For viewing all the run-time options, run the previous command adding the -help:

$ java -jar selenium-server-standalone-{VERSION}.jar -help

Writing Tests

Using the prefered CSS selector model to locate elements on a page, Nightwatch makes it very easy to write automated End-to-End test cases.

Create a separate folder for tests in your project, e.g.: tests. Each file inside it will be loaded as a test by the Nightwatch test runner. A basic test will look like this:


module.exports = {
  "Demo test Google" : function (browser) {
    browser
      .url("http://www.google.com")
      .waitForElementVisible('body', 1000)
      .setValue('input[type=text]', 'nightwatch')
      .waitForElementVisible('button[name=btnG]', 1000)
      .click('button[name=btnG]')
      .pause(1000)
      .assert.containsText('#main', 'The Night Watch')
      .end();
  }
};

Remember always to call the .end() method when you want to close your test, in order for the selenium session to be properly closed.

A test can have multiple steps, if needed:


module.exports = {
  "step one" : function (browser) {
    browser
      .url("http://www.google.com")
      .waitForElementVisible('body', 1000)
      .setValue('input[type=text]', 'nightwatch')
      .waitForElementVisible('button[name=btnG]', 1000)
  },
  
  "step two" : function (browser) {
    browser
      .click('button[name=btnG]')
      .pause(1000)
      .assert.containsText('#main', 'The Night Watch')
      .end();
  }
};

Tests can also be written in this format:


this.demoTestGoogle = function (browser) {
  browser
    .url("http://www.google.com")
    .waitForElementVisible('body', 1000)
    .setValue('input[type=text]', 'nightwatch')
    .waitForElementVisible('button[name=btnG]', 1000)
    .click('button[name=btnG]')
    .pause(1000)
    .assert.containsText('#main', 'The Night Watch')
    .end();
};

Using XPath selectors

Nightwatch supports xpath selectors also. To switch to xpath instead of css selectors as the locate strategy, call the method useXpath(), as seen in the example below.

To switch back to CSS, call useCss().


this.demoTestGoogle = function (browser) {
  browser
    .useXpath() // every selector now must be xpath
    .click("//tr[@data-recordid]/span[text()='Search Text']")
    .useCss() // we're back to CSS now
    .setValue('input[type=text]', 'nightwatch')
};

Test Runner

Nightwatch includes a command-line test runner which makes it easy to run tests and generate useful output.

Example usage:

$ ./nightwatch -t tests/demotest.js

If you have installed nightwatch with -g option you can skip this.

To use the test runner in your project simply create a new file called nightwatch and add the following:

For Linux and MacOSX:
#!/usr/bin/env node
require('nightwatch/bin/runner.js');

Then set the permissions:

chmod a+x nightwatch

For Windows:

Name the file nightwatch.js and add the following line:


require('nightwatch/bin/runner.js');

Then run as follow:

C:\workspace\project> node nightwatch.js

Command-line Options

The test runner supports a number of run-time options to be passed at. To view all, run the following:

$ ./nightwatch --help
Name Shortname default description
--config -c ./nightwatch.json The location of the nightwatch.json file - the configuration file which the runner uses and which also includes the Selenium WebDriver options.
--output -o tests_output The location where the JUnit XML reports will be saved.
--env -e default Which testing environment to use - defined in nightwatch.json
--verbose -v Shows extended selenium command logging during the session
--test -t Runs only the specified test. By default it will attempt to run all tests in the folder and its subfolders.
--group -g Runs only the specified group of tests (subfolder). Tests are grouped by being placed in the same subfolder.
--skipgroup -s Skip one or several (comma separated) group of tests.
--filter
since v0.3.2
-f Specify a filter (glob expression) as the file name format to use when loading the test files.

Test Groups

Nightwatch makes it possible to organize your test scripts into groups and run them as needed. To group tests together just place them in the same sub-folder. The folder name is the name of the group.

Example:

lib/
  ├── selenium-server-standalone.jar         
custom-commands/
  ├── loginUser.js         
  ├── attachPicture.js         
tests/
  ├── logingroup
  |   ├── login_test.js
  |   └── otherlogin_test.js
  ├── addressbook
  |   ├── addressbook_test.js
  |   └── contact_test.js
  ├── chat
  |   ├── chatwindow_test.js
  |   ├── chatmessage_test.js
  |   └── otherchat_test.js
  └── smoketests
      ├── smoke_test.js
      └── othersmoke_test.js

To run only the smoketests group you would do the following:

$ nightwatch --group smoketests

Also, if you would want to skip running the smoketests group you would do the following:

$ nightwatch --skipgroup smoketests

To skip multiple groups, just add them as comma-separated:

$ nightwatch --skipgroup addressbook,chat

setUp and tearDown methods

The setUp and tearDown methods are standard to any testing framework. As the name suggests, setUp runs before each test step and tearDown after each one respectively.

The setUp method is called with the Nightwatch instance passed as argument, while tearDown is called without any arguments as the asynchronous commands queue is finished by then and (presumably) the session is ended.

Example:


module.exports = {
  setUp : function(browser) {
    console.log("Setting up...");  
  },
  
  tearDown : function() {
    console.log("Closing down...");
  },
  
  "step one" : function (browser) {
    browser
     // ... 
  },
  
  "step two" : function (browser) {
    browser
    // ...
      .end();
  }
};

In the example abobe the sequence of method calles will be as follows: setUp(), "step one", tearDown(), setUp(), "step two", tearDown()

Asynchronous setUp/tearDown

The setUp and tearDown methods can also perform asynchronous operations, in which case they will require a callback argument.


The callback function must be called as the last step when the async operation completes. Not calling it will block the test runner.

Example with async setUp & tearDown:


module.exports = {
  setUp: function(browser, callback) {
    // performing an async operation
    setTimeout(function() {
      // finished async duties
      callback();
    }, 100);
  },
  
  tearDown: function(callback) {
    // performing an async operation
    setTimeout(function() {
      // finished async duties
      callback();
    }, 200);
  }
};

The test runner expects a configuration file to be passed, using by default a nightwatch.json file from the current directory, if present. So let's create one in the same folder as the previous nightwatch runner file.

The nightwatch.json file will look like this:

{
  "src_folders" : ["examples/tests"],
  "output_folder" : "examples/reports",
  "custom_commands_path" : "",
  "custom_assertions_path" : "",
  "globals_path" : "",
  
  "selenium" : {
    "start_process" : false,
    "server_path" : "",
    "log_path" : "",
    "host" : "127.0.0.1",
    "port" : 4444,
    "cli_args" : {
      "webdriver.chrome.driver" : "",
      "webdriver.ie.driver" : ""
    }  
  },
  
  "test_settings" : {
    "default" : {
      "launch_url" : "http://localhost",
      "selenium_port"  : 4444,
      "selenium_host"  : "localhost",
      "silent": true,
      "screenshots" : {
        "enabled" : false,
        "path" : ""
      },
      "desiredCapabilities": {
        "browserName": "firefox",
        "javascriptEnabled": true,
        "acceptSslCerts": true
      }
    },
    
    "chrome" : {
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true,
        "acceptSslCerts": true
      }
    }
  }
}

Basic settings:

Name type default description
src_folders string|array none An array of folders (excluding subfolders) where the tests are located.
output_folder
Optional
string tests_output The location where the JUnit XML report files will be saved.
custom_commands_path Optional string none Location where custom commands will be loaded from.
custom_assertions_path Optional string none Location where custom assertions will be loaded from.
globals_path
Optional since v0.4.8
string none Location of an external globals module which will be loaded and made available to the test as a property globals on the main client instance.

Globals can also be defined/overwritter inside a test_settings environment.
selenium
Optional
object An object containing Selenium Server related configuration options. See below for details.
test_settings object This object contains all the test related options. See below for details.
live_output
Optional
boolean false Whether or not to buffer the output in case of parallel running. See below for details.

Selenium settings:

Below are a number of options for the selenium server process. Nightwatch can start and stop the Selenium process automatically which is very convenient as you don't have to manage this yourself and focus only on the tests.

If you'd like to enable this, set start_process to true and specify the location of the jar file inside server_path.

Name type default description
start_process boolean false Whether or not to manage the selenium process automatically.
server_path string none The location of the selenium jar file. This needs to be specified if start_process is enabled.
E.g.: lib/selenium-server-standalone-2.39.0.jar
log_path string|boolean none The location where the selenium output.log file will be placed. Defaults to current directory.
To disable Selenium logging, set this to false
host string 127.0.0.1 Usually not required and only used if start_process is true. Specify the IP address you wish Selenium to listen on.
port integer 4444 The port number Selenium will listen on.
cli_args
since v0.5.1
object none List of cli arguments to be passed to the Selenium process. Here you can set various options for browser drivers, such as:

  • webdriver.firefox.profile: Selenium will be default create a new Firefox profile for each session. If you wish to use an existing Firefox profile you can specify its name here.
    Complete list of Firefox Driver arguments available here.
  • webdriver.chrome.driver: Nightwatch can run the tests using Chrome browser also. To enable this you have to download the ChromeDriver binary and specify it's location here. Also don't forget to specify chrome as the browser name in the desiredCapabilities object.
    More information can be found on the ChromeDriver website.
  • webdriver.ie.driver: Nightwatch has support for Internet Explorer also. To enable this you have to download the IE Driver binary and specify it's location here. Also don't forget to specify "internet explorer" as the browser name in the desiredCapabilities object.

Test settings:

Below are a number of options that will be passed to the Nightwatch instance. You can specify multiple groups of options so you could have different values per environment:

{
  ...        
  "test_settings" : {
    "default" : {
      ...
    },
    "integration" : {
      ...
    }
  }        
}

A "default" environment is required. All the other environments are derived from default and their settings can be overridden as needed.

The key of the settings group can be passed then to the runner as the --env argument to use the specified settings, like so:

$ nightwatch --env integration

This can be useful if you need to have different settings for your local machine and the Continous Integration server.

Name type default description
launch_url string none A url which can be used later in the tests as the main url to load. Can be useful if your tests will run on different environments, each one with a different url.
selenium_host string localhost The hostname/IP on which the selenium server is accepting connections.
selenium_port integer 4444 The port number on which the selenium server is accepting connections.
silent boolean true Whether to show extended Selenium command logs.
output boolean true Use to disable terminal output completely.
disable_colors
since v0.4.13
boolean false Use to disable colored output in the terminal.
firefox_profile
deprecated
string|boolean none This options has been deprecated in favor of the cli_args object on the selenium settings object.
chrome_driver
deprecated
string none This options has been deprecated in favor of the cli_args object on the selenium settings object.
ie_driver
deprecated
string none This options has been deprecated in favor of the cli_args object on the selenium settings object.
screenshots object none Selenium generates screenshots when command errors occur. These can be saved somewhere on the disk.
Example: "screenshots" : {"enabled" : true, "path" : ""}
username string none In case the selenium server requires credentials this username will be used to compute the Authorization header.

The value can be also an environment variable, in which case it will look like this:
"username" : "${SAUCE_USERNAME}"
access_key string none This field will be used together with username to compute the Authorization header.

Like username, the value can be also an environment variable:
"access_key" : "${SAUCE_ACCESS_KEY}"
desiredCapabilities object An object wich will be passed to the Selenium WebDriver when a new session will be created. You can specify browser name for instance along with other capabilities.
Example:

"desiredCapabilities" : {
  "browserName" : "firefox",
  "acceptSslCerts" : true
}

You can view the complete list of capabilities here.
globals
since v0.4.8
object An object wich will be made available within the test and can be overwritten per environment. Example:

"globals" : {
  "myGlobal" : "some_global"
}
exclude
since v0.4.9
array An array of folders or file patterns to be skipped.
Example:

"exclude" : ["excluded-folder"]
or:
"exclude" : ["tests/*-smoke.js"]
filter
since v0.5.1
string Folder or file pattern to be used when loading the tests. Files that don't match this patter will be ignored.
Example:

"filter" : "tests/*-smoke.js"
use_xpath
since v0.5.1
boolean false Use xpath as the default locator strategy

Parallel Running

Starting with v0.5 Nightwatch supports the tests to be run in parallel. This works by specifying multiple envionments in the command line, separated by comma. E.g.:

$ nightwatch -e default,chrome

The above will run two environments named default and chrome in parallel.


Terminal Output

Each environment will be run as a separate child_process and the output will be sent to the main process.


To make the output easier to read, Nightwatch by default buffers the output from each child process and displays everything at the end, groupped by environment.

If you'd like to disable the output buffering and see the output from each child process as it is sent to stdout, simply set the property "live_output" : true on the top level in your nightwatch.json (e.g.: after selenium).
You can create a separate environment per browser (by chaning desiredCapabilities) and then run them in parallel. In addition, using the filter and exclude options tests can be split per environment in order to be ran in parallel.

Writing Custom Commands

Most of the time you will need to extend the Nightwatch commands to suit your own application needs. Doing that is only a matter of creating a separate folder and defining your commands in that folder, each one inside its own file.

Then specify the path to that folder inside the nightwatch.json file, as the custom_commands_path property. The command name is the name of the file itself, and it needs to follow the following pattern:


exports.command = function(file, callback) {
  var self = this, imagedata, fs = require('fs');
  
  try {
    var originalData = fs.readFileSync(file);
    var base64Image = new Buffer(originalData, 'binary')
      .toString('base64');
    imageData = 'data:image/jpeg;base64,' + base64Image;
  } catch (err) {
    console.log(err);
    throw "Unable to open file: " + file;
  }
    
  this.execute(
    function(data) { // execute application specific code
      App.resizePicture(data);
      return true;
    }, 
    
    [imageData], // arguments array to be passed
    
    function(result) {
      if (typeof callback === "function") {
        callback.call(self, result);
      }
    }
  );
  
  return this; // allows the command to be chained.
};

The example below defines a command (e.g. resizePicture.js) which loads an image file as data-URI and calls a method named resizePicture defined inside the application.

With this command, the test will look something like:


module.exports = {
  "testing resize picture" : function (browser) {
    browser
      .url("http://app.host")
      .waitForElementVisible("body")
      .resizePicture("/var/www/pics/moon.jpg")
      .assert.element(".container .picture-large")
      .end();
  }
};

Writing Custom Assertions

Nightwatch allows you to even define your own assertions, extending the available .assert and .verify namespaces.

Beginning with v0.4 assertions have a very simple interface which is shared between built-in assertions and custom ones. Therefore, custom assertions must implement the following interface:


exports.assertion = function() {
  
  /**
   * The message which will be used in the test output and 
   * inside the XML reports
   * @type {string}
   */
  this.message;
  
  /**
   * A value to perform the assertion on. If a function is 
   * defined, its result will be used.
   * @type {function|*}
   */
  this.expected;
  
  /**
   * The method which performs the actual assertion. It is 
   * called with the result of the value method as the argument.
   * @type {function}
   */
  this.pass = function(value) {
    
  };

  /**
   * The method which returns the value to be used on the 
   * assertion. It is called with the result of the command's 
   * callback as argument.
   * @type {function}
   */
  this.value = function(result) {
    
  };
  
  /**
   * Performs a protocol command/action and its result is 
   * passed to the value method via the callback argument.
   * @type {function}
   */
  this.command = function(callback) {
     
    return this;
  };

};

Both custom assertions and custom commands inherit from EventEmitter if defined in the form above.
If, however, your command/assertion is defined in the constructor style, then you must inherit manually, similarly to the pause command.

To see some examples, check the assertions module source-code on Github:
/nightwatch/tree/master/lib/selenium/assertions