Overview
What is Nightwatch?
Nightwatch.js is an automated testing framework for web applications and websites, written in Node.js and using the W3C WebDriver API (formerly Selenium WebDriver).
It is a complete browser (End-to-End) testing solution which aims to simplify the process of setting up Continuous Integration and writing automated tests. Nightwatch can also be used for writing Node.js unit tests.
Nightwatch got its name from the famous painting The Night Watch by Dutch artist Rembrandt van Rijn. The masterpiece is prominently displayed in the Rijksmuseum, in Amsterdam - The Netherlands.
Overview of WebDriver
WebDriver is a general purpose library for automating web browsers. It was started as part of the Selenium project, which is a very popular and comprehensive set of tools for browser automation, initially written for Java but now with support for most programming languages.
Nightwatch uses the WebDriver API to perform the browser automation related tasks, like opening windows and clicking links for instance.
WebDriver is now a W3C specification, which aims to standardize browser automation. WebDriver is a remote control interface that enables introspection and control of user agents. It provides a platform and a restful HTTP api as a way for web browsers to be remotely controlled.
Theory of Operation
Nightwatch works by communicating over a restful HTTP api with a WebDriver server (typically the Selenium server). The restful API protocol is defined by the W3C WebDriver API. See below for an example workflow for browser initialization.
Most of the times, Nightwatch needs to send at least 2 requests to the WebDriver server in order to perform a command or assertion, the first one being the request to locate an element given a CSS selector (or Xpath expression) and the next to perform the actual command/assertion on the given element.
Installation
Install Node.js
From nodejs.org:
"Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices."
There are installation packages and instructions for most major Operating systems on its website nodejs.org. Remember to install also the npm tool, which is the node package manager and is distributed with the Node.js installer.
Install Nightwatch
To install the latest version using the npm command line tool, run the following:
$ npm install [-g] nightwatch
Add -g
option to make nightwatch
runner available globally in your system.
Selenium Server Setup
The most common WebDriver implementation is the Selenium Server. This allows you to manage multiple browser configurations in one place. However, you can also run the individual browser drivers directly, such as the ChromeDriver, more details are available in the Browser Drivers Setup section.
Selenium Server
Selenium Server is a Java application which Nightwatch uses to connect to the various browsers. It runs separately on the machine with the browser you want to test. You will need to have the Java Development Kit (JDK) installed, minimum required version is 7. You can check this by running java -version
from the command line.
Download Selenium
Download the latest version of the selenium-server-standalone-{VERSION}.jar
file from the Selenium downloads page and place it on the computer with the browser you want to test.
In most cases this will be on your local machine and typically inside your project's source folder.
A good practice is to create a separate subfolder (e.g. bin
) and place it there as you might have to download other driver binaries if you want to test multiple browsers.
Running Selenium Automatically
If the server is on the same machine where Nightwatch is running, it can be started/stopped directly by the Nightwatch Test Runner.
Running Selenium Manually
To run the Selenium Server manually, from the directory with the jar run the following:
$ java -jar selenium-server-standalone-{VERSION}.jar
Using Selenium
For viewing all the run-time options, run the previous command adding the -help
:
$ java -jar selenium-server-standalone-{VERSION}.jar -help
Starting with Selenium 3, FirefoxDriver is no longer included in the package. Also, starting with version 48, Firefox is no longer compatible with FirefoxDriver which is shipped with Selenium 2.x. Firefox users are advised to use GeckoDriver for their testing. For more info, refer to the browser setup section.
More info about running the Selenium Server can be found here: https://github.com/SeleniumHQ/selenium/wiki/RemoteWebDriverServer
Configuration
The test runner expects a configuration file to be passed, using by default a nightwatch.json
file from the current directory, if present. A nightwatch.conf.js
file will also be loaded by default, if found.
Let's create the nightwatch.json
in the project's root folder and add this inside:
{
"src_folders" : ["tests"],
"output_folder" : "reports",
"custom_commands_path" : "",
"custom_assertions_path" : "",
"page_objects_path" : "",
"globals_path" : "",
"selenium" : {
"start_process" : false,
"server_path" : "",
"log_path" : "",
"port" : 4444,
"cli_args" : {
"webdriver.chrome.driver" : "",
"webdriver.gecko.driver" : "",
"webdriver.edge.driver" : ""
}
},
"test_settings" : {
"default" : {
"launch_url" : "http://localhost",
"selenium_port" : 4444,
"selenium_host" : "localhost",
"silent": true,
"screenshots" : {
"enabled" : false,
"path" : ""
},
"desiredCapabilities": {
"browserName": "firefox",
"marionette": true
}
},
"chrome" : {
"desiredCapabilities": {
"browserName": "chrome"
}
},
"edge" : {
"desiredCapabilities": {
"browserName": "MicrosoftEdge"
}
}
}
}
Using both configuration files is also possible, with nightwatch.conf.js
always taking precedence if both are found.
Example
module.exports = (function(settings) {
settings.test_workers = false;
return settings;
})(require('./nightwatch.json'));
Basic settings
Name | type | default | description |
---|---|---|---|
src_folders | string|array | none | An array of folders (excluding subfolders) where the tests are located. |
output_folder Optional |
string | tests_output | The location where the JUnit XML report files will be saved. |
custom_commands_path Optional | string|array | none | Location(s) where custom commands will be loaded from. |
custom_assertions_path Optional | string|array | none | Location(s) where custom assertions will be loaded from. |
page_objects_path Optional |
string|array | none | Location(s) where page object files will be loaded from. |
globals_path Optional |
string | none | Location of an external globals module which will be loaded and made available to the test as a property globals on the main client instance. Globals can also be defined/overwritten inside a test_settings environment. |
selenium Optional |
object | An object containing Selenium Server related configuration options. See below for details. | |
test_settings | object | This object contains all the test related options. See below for details. | |
live_output Optional |
boolean | false | Whether or not to buffer the output in case of parallel running. See below for details. |
disable_colors Optional |
boolean | false | Controls whether or not to disable coloring of the cli output globally. |
parallel_process_delay Optional |
integer | 10 | Specifies the delay(in milliseconds) between starting the child processes when running in parallel mode. |
test_workers Optional |
boolean|object | false | Whether or not to run individual test files in parallel. If set to true , runs the tests in parallel and determines the number of workers automatically. If set to an object, can specify specify the number of workers as "auto" or a number .
Example: "test_workers" : {"enabled" : true, "workers" : "auto"} |
test_runner Optional since v0.8.0 |
string|object | "default" | Specifies which test runner to use when running the tests. Values can be either default (built in nightwatch runner) or mocha .Example: "test_runner" : {"type" : "mocha", "options" : {"ui" : "tdd"}} |
Selenium settings
Below are a number of options for the selenium server process. Nightwatch can start and stop the Selenium process automatically which is very convenient as you don't have to manage this yourself and focus only on the tests.
If you'd like to enable this, set start_process
to true
and specify the location of the jar
file inside server_path
.
Name | type | default | description |
---|---|---|---|
start_process | boolean | false | Whether or not to manage the selenium process automatically. |
start_session | boolean | true | Whether or not to automatically start the Selenium session. This will typically be set to false when running unit/integration tests that don't interact with the Selenium server. |
server_path | string | none | The location of the selenium jar file. This needs to be specified if start_process is enabled.E.g.: bin/selenium-server-standalone-2.43.0.jar |
log_path | string|boolean | none | The location where the selenium output.log file will be placed. Defaults to current directory.To disable Selenium logging, set this to false |
port | integer | 4444 | The port number Selenium will listen on. |
cli_args | object | none | List of cli arguments to be passed to the Selenium process. Here you can set various options for browser drivers, such as:
|
Test settings
Below are a number of settings that will be passed to the Nightwatch instance. You can define multiple sections (environments) of test settings so you could overwrite specific values per environment.
A "default" environment is required. All the other environments are inheriting from default and can overwrite settings as needed.
{
...
"test_settings" : {
"default" : {
"launch_url" : "http://localhost",
"globals" : {
"myGlobalVar" : "some value",
"otherGlobal" : "some other value"
}
},
"integration" : {
"launch_url" : "http://staging.host",
"globals" : {
"myGlobalVar" : "other value"
}
}
}
}
The key of the settings group can be passed then to the runner as the --env
argument to use the specified settings, like so:
$ nightwatch --env integration
This can be useful if you need to have different settings for your local machine and the Continuous Integration server.
The launch_url
property
This property will be made available to the main Nightwatch api which is used in the tests. Its value depends on which environment is used.
If you run your tests as in the example above (with --env integration
) launch_url
will be set to http://staging.host
, as per the configuration. Otherwise it will have the value defined in the default
environment (i.e. http://localhost
).
module.exports = {
'Demo test' : function (browser) {
browser
.url(browser.launchUrl)
// ...
.end();
}
};
Test globals
A very useful concept that Nightwatch provides is test globals. In its most simple form, this is a dictionary of name-value pairs which is defined in your nightwatch.json
configuration file.
Like the launch_url
property, this is made available directly on the Nightwatch api which is passed to the tests. It is also dependent on the environment used, having the ability to overwrite specific globals per environment.
If we still pass the --env integration
option to the runner, then our globals object will look like below:
module.exports = {
'Demo test' : function (browser) {
console.log(browser.globals);
// {
// "myGlobalVar" : "some value",
// "otherGlobal" : "some other value"
// }
}
};
By default, a deep object copy will be created for each test suite run. If you'd like to maintain the same object throughout the entire tests run, set the persist_globals
option to true
, as detailed below.
Full list of settings
Name | type | default | description |
---|---|---|---|
launch_url | string | none | A url which can be used later in the tests as the main url to load. Can be useful if your tests will run on different environments, each one with a different url. |
selenium_host | string | localhost | The hostname/IP on which the selenium server is accepting connections. |
selenium_port | integer | 4444 | The port number on which the selenium server is accepting connections. |
request_timeout_options since v0.9.11 |
object | 60000 / 0 | Defines the number of milliseconds an HTTP request to the Selenium server will be kept open before a timeout is reached. After a timeout, the request can be automatically retried a specified number of times, defined by the retry_attempts property.
Example: "request_timeout_options": {
|
silent | boolean | true | Whether to show extended Selenium command logs. |
output | boolean | true | Use to disable terminal output completely. |
disable_colors | boolean | false | Use to disable colored output in the terminal. |
firefox_profile deprecated |
string|boolean | none |
This options has been deprecated in favor of the cli_args object on the selenium settings object.
|
chrome_driver deprecated |
string | none |
This options has been deprecated in favor of the cli_args object on the selenium settings object.
|
ie_driver deprecated |
string | none |
This options has been deprecated in favor of the cli_args object on the selenium settings object.
|
screenshots | object | none | Selenium generates screenshots when command errors occur. With on_failure set to true, also generates screenshots for failing or erroring tests. These are saved on the disk. Since v0.7.5 you can disable screenshots for command errors by setting "on_error" to false .
Example: "screenshots" : { |
username | string | none | In case the selenium server requires credentials this username will be used to compute the Authorization header. The value can be also an environment variable, in which case it will look like this: "username" : "${SAUCE_USERNAME}"
|
access_key | string | none | This field will be used together with username to compute the Authorization header. Like username , the value can be also an environment variable:"access_key" : "${SAUCE_ACCESS_KEY}"
|
proxy since v0.8.6 |
string | none | Proxy requests to the selenium server. http, https, socks(v5), socks5, sock4, and pac are accepted. Uses node-proxy-agent. Example: http://user:pass@host:port |
desiredCapabilities | object | An object which will be passed to the Selenium WebDriver when a new session will be created. You can specify browser name for instance along with other capabilities.
Example: "desiredCapabilities" : { You can view the complete list of capabilities here. |
|
globals | object | An object which will be made available within the test and can be overwritten per environment. Example:"globals" : {
|
|
exclude | array | An array of folders or file patterns to be skipped (relative to the main source folder). Example: "exclude" : ["excluded-folder"] or: "exclude" : ["test-folder/*-smoke.js"] |
|
filter | string | Folder or file pattern to be used when loading the tests. Files that don't match this pattern will be ignored. Example: "filter" : "tests/*-smoke.js" |
|
log_screenshot_data | boolean | false | Do not show the Base64 image data in the (verbose) log when taking screenshots. |
use_xpath | boolean | false | Use xpath as the default locator strategy |
cli_args | object | none | Same as Selenium settings cli_args . You can override the global cli_args on a per-environment basis. |
end_session_on_fail | boolean | true | End the session automatically when the test is being terminated, usually after a failed assertion. |
skip_testcases_on_fail | boolean | true | Skip the remaining testcases (or test steps) from the same test suite (i.e. test file), when one testcase fails. |
output_folder since v0.8.18 |
string|boolean | Define the location where the JUnit XML report files will be saved. This will overwrite any value defined in the Basic Settings section. If you'd like to disable the reports completely inside a specific environment, set this to false . |
|
persist_globals since v0.8.18 |
boolean | false | Set this to true if you'd like to persist the same globals object between testsuite runs or have a (deep) copy of it per each testsuite. |
compatible_testcase_support since v0.9.0 |
boolean | false | Applies to unit tests. When set to true this allows for tests to be written in the standard Exports interface which is interchangeable with the Mocha framework. Prior unit tests interface support is deprecated and this will become the default in future releases. |
detailed_output since v0.9.0 |
boolean | true | By default detailed assertion output is displayed while the test is running. Set this to false if you'd like to only see the test case name displayed and pass/fail status. This is especially useful when running tests in parallel. |
Browser Drivers Setup
This section contains guides for getting started with most of the major browsers and setup instructions on how to configure the individual webdriver implementations to work with Nightwatch.
The individual drivers described here are usually standalone applications which are used to interact with the browsers via the WebDriver HTTP API. You can run them either directly, or through the Selenium Server.
GeckoDriver
Overview
GeckoDriver is a standalone application used to interact with Gecko-based browsers, such as Firefox. It is written in Rust and maintained by Mozilla.
Starting with Firefox 48, GeckoDriver is the only way to automate Firefox, the legacy FirefoxDriver which used to be part of Selenium is no longer supported. Internally it translates the HTTP calls into Marionette, Mozilla's automation protocol built into Firefox.
Download
Binaries are available for download on the GeckoDriver Releases page on GitHub, for various platforms.
Selenium 2.x users are advised to use version v0.9, whereas Selenium 3 users should use the latest version.
Usage
If you're using GeckoDriver through Selenium Server, simply set the cli argument "webdriver.gecko.driver"
to point to the location of the binary file. E.g.:
{
"selenium" : {
"start_process" : true,
"server_path" : "./bin/selenium-server-standalone-3.{VERSION}.jar",
"log_path" : "",
"port" : 4444,
"cli_args" : {
"webdriver.gecko.driver" : "./bin/geckodriver"
}
}
}
GeckoDriver can also be used as a standalone application. Usage steps are documented on GitHub: https://github.com/mozilla/geckodriver#usage.
Command line usage
$ ./bin/geckodriver-0.10 -help
geckodriver 0.10.0
USAGE:
geckodriver-0.10 [FLAGS] [OPTIONS]
FLAGS:
--connect-existing Connect to an existing Firefox instance
-h, --help Prints help information
--no-e10s Start Firefox without multiprocess support (e10s) enabled
-V, --version Prints version information
-v Set the level of verbosity. Pass once for debug level logging and twice for trace level logging
OPTIONS:
-b, --binary Path to the Firefox binary, if no binary capability provided
--log Set Gecko log level [values: fatal, error, warn, info, config, debug, trace]
--marionette-port Port to use to connect to gecko (default: random free port)
--host Host ip to use for WebDriver server (default: 127.0.0.1)
-p, --port Port to use for WebDriver server (default: 4444)
Firefox Capabilities
GeckoDriver supports a capability named firefoxOptions
which takes Firefox-specific preference values. Details are available on the GeckoDriver GitHub page: https://github.com/mozilla/geckodriver#firefox-capabilities.
Firefox Profile
Specifying the firefox profile can be done by setting the profile
property in the firefoxOptions
dictionary, as detailed above. This can be the base64-encoded zip of a profile directory and it may be used to install extensions or custom certificates.
Implementation Status
GeckoDriver is not yet feature complete, which means it does not yet offer full conformance with the WebDriver standard or complete compatibility with Selenium. Implementation status can be tracked on the Marionette MDN page.
ChromeDriver
Overview
ChromeDriver is a standalone server which implements the W3C WebDriver wire protocol for Chromium. ChromeDriver is available for Chrome on Android and Chrome on Desktop (Mac, Linux, Windows and ChromeOS).
Download
Binaries are available for download on the ChromeDriver Downloads page, for various platforms.
Selenium Server Usage
If you're using ChromeDriver through Selenium Server, simply set the cli argument "webdriver.chrome.driver"
to point to the location of the binary file. E.g.:
{
"selenium" : {
"start_process" : true,
"server_path" : "./bin/selenium-server-standalone-3.{VERSION}.jar",
"log_path" : "",
"port" : 4444,
"cli_args" : {
"webdriver.chrome.driver" : "./bin/chromedriver"
}
}
}
Standalone Usage
If you're only running your tests against Chrome, running ChromeDriver standalone is easier and slightly faster. Also there is no dependency on Java.
This requires a bit more configuration:
1) First, disable Selenium Server, if applicable:
{
"selenium" : {
"start_process" : false
}
}
2) Configure the port and default path prefix.
ChromeDriver runs by default on port 9515. We also need to clear the default_path_prefix
, as it is set by default to /wd/hub
, which is what selenium is using.
{
"test_settings" : {
"default" : {
"selenium_port" : 9515,
"selenium_host" : "localhost",
"default_path_prefix" : "",
"desiredCapabilities": {
"browserName": "chrome",
"chromeOptions" : {
"args" : ["--no-sandbox"]
},
"acceptSslCerts": true
}
}
}
}
3) Start the ChromeDriver server
The easiest way to manage the ChromeDriver process is by using the chromedriver
NPM package, which is a third-party wrapper against the binary. This will abstract the downloading of the chromedriver binary and will make it easy to manage starting and stopping of the process.
You can add this to your external globals
file, like so:
var chromedriver = require('chromedriver');
module.exports = {
before : function(done) {
chromedriver.start();
done();
},
after : function(done) {
chromedriver.stop();
done();
}
};
Using a fixed ChromeDriver version
In some situations you may need to use a specific version of ChromeDriver. For instance, the CI server runs an older version of Chrome. Then you will need an older version of ChromeDriver.
Here's what your globals file might look like:
var chromedriver = require('chromedriver');
var path = require('path');
var driverInstanceCI;
function isRunningInCI() {
return this.test_settings.globals.integration;
}
function startChromeDriver() {
if (isRunningInCI.call(this)) {
var location = path.join(__dirname, '../bin/chromedriver-linux64-2.17');
driverInstanceCI = require('child_process').execFile(location, []);
return;
}
chromedriver.start();
}
function stopChromeDriver() {
if (isRunningInCI.call(this)) {
driverInstanceCI && driverInstanceCI.kill();
return;
}
chromedriver.stop();
}
module.exports = {
'ci-server' : {
integration : true
},
before : function(done) {
startChromeDriver.call(this);
done();
},
after : function(done) {
stopChromeDriver.call(this);
done();
}
};
Run your tests then with (on the CI server):
$ ./node_modules/.bin/nightwatch --env ci-server
ChromeOptions
You can specify Chrome options or switches using the chromeOptions
dictionary, under the desiredCapabilities
. Refer to the ChromeDriver website for a fill list of supported capabilities and options.
Command line usage
$ ./bin/chromedriver -h
Usage: ./bin/chromedriver [OPTIONS]
Options
--port=PORT port to listen on
--adb-port=PORT adb server port
--log-path=FILE write server log to file instead of stderr, increases log level to INFO
--verbose log verbosely
--version print the version number and exit
--silent log nothing
--url-base base URL path prefix for commands, e.g. wd/url
--port-server address of server to contact for reserving a port
--whitelisted-ips comma-separated whitelist of remote IPv4 addresses which are allowed to connect to ChromeDriver
Microsoft WebDriver
Overview
Microsoft WebDriver is a standalone server which implements the W3C WebDriver wire protocol for the Edge browser. It is supported by Windows 10 and onwards.
Download
Binaries are available for download on the Microsoft WebDriver homepage.
Selenium Server Usage
If you're using Microsoft WebDriver through Selenium Server, simply set the cli argument "webdriver.edge.driver"
to point to the location of the binary file. E.g.:
{
"selenium" : {
"start_process" : true,
"server_path" : "bin/selenium-server-standalone-3.{VERSION}.jar",
"log_path" : "",
"port" : 4444,
"cli_args" : {
"webdriver.edge.driver" : "bin/MicrosoftWebDriver.exe"
}
},
"test_settings" : {
"default" : {
"selenium_port" : 4444,
"selenium_host" : "localhost",
"desiredCapabilities": {
"browserName": "MicrosoftEdge",
"acceptSslCerts": true
}
}
}
}
Standalone Usage
If you're only running your tests against Edge, running the EdgeDriver standalone can be slightly faster. Also there is no dependency on Java.
This requires a bit more configuration and you will need to start/stop the EdgeDriver:
1) First, disable Selenium Server, if applicable:
{
"selenium" : {
"start_process" : false
}
}
2) Configure the port and default path prefix.
Microsoft WebDriver runs by default on port 9515. We also need to clear the default_path_prefix
, as it is set by default to /wd/hub
, which is what selenium is using.
{
"test_settings" : {
"default" : {
"selenium_port" : 17556,
"selenium_host" : "localhost",
"default_path_prefix" : "",
"desiredCapabilities": {
"browserName": "MicrosoftEdge",
"acceptSslCerts": true
}
}
}
}
3) Start the MicrosoftWebDriver server
From the Windows CMD prompt, simply CD to the folder where the MicrosoftWebDriver.exe
binary is located and run:
C:\nightwatch\bin>MicrosoftWebDriver.exe
[13:44:49.515] - Listening on http://localhost:17556/
Full command line usage:
C:\nightwatch\bin>MicrosoftWebDriver.exe -h
Usage:
MicrosoftWebDriver.exe --host= --port= --package= --verbose
Implementation Status
EdgeDriver is not yet feature complete, which means it does not yet offer full conformance with the WebDriver standard or complete compatibility with Selenium. Implementation status can be tracked on the Microsoft WebDriver homepage.