Introducing Component Testing in Nightwatch

Last year in November we've began working on adding component testing to Nightwatch and now I am delighted to present the first version which is ready for public use. So far, it supports Vue and React components but our plans is to support all kinds of web components in the near future.

Component testing in Nightwatch is the next logical step since we cover the browser integration and we have built-in assertion libraries and reporters. We only needed a fast and reliable way of rendering the componets in isolation and for that we have chosen Vite – the new frontend build tool created by the team behind the Vue.js project.

Below I will attempt to explain what component testing actually is and how Nightwatch (v2.0) tackles the various problems that arise with testing web components.

Component Testing Overview

Unit testing single page front-end applications usually presents an entire set of issues to deal with over regular approaches of unit testing. In order to reliably test a web component in isolation, we need to be able to render it first.

When testing web components, the crucial aspect is how the component is rendered. In many cases, rendering using a real browser might seem like an overkill and something clumsy that will slow everything down considerably. For instance, a large project might have a dedicated QA team who takes care of the end-to-end testing and in such a case, it might seem unnecessary to write component tests using real browsers, especially if it’s overly complicated or slow.

Rendering Using JSDom

JSDom is tool which offers the possibility to render a web component using a Node.js virtual renderer, without the need of a real browser. Everything happens in the CLI and it’s usually pretty fast.

Here’s how it normally looks like:

Rendering using Karma TestRunner/Puppeteer

The other popular approach is to use something like the Karma Runner, which is a frontend testing tool created about 10 years ago at Google. There are also variations on this theme where Puppetter and browser-based Mocha is used.

In this approach all files needed are loaded on the test renderer page where it all happens: the component is loaded together with dependencies and any testing tools needed, then the component is rendered and the test is executed in the same browser context. It looks a bit like this:

Rendering in Nightwatch

Either one of the above approaches are fine in terms of running tests and debugging, however both of them come with their one limitations, which are easy to spot:

Using JSDom rendering the limitation is clear: no real browser. However, advantages to consider are speed and access to OS-level APIs, which would make tasks like loading files or generating advanced reports more straightforward.

On the other hand, when using the Karma Runner or similar approach the advantage is clear: everything is happening in the browser and thus the test is more reliable. However, the disadvantage is that working with external files is not straightforward and also reporting is limited. I also found configuration to be quite difficult, as normally you have to combine several tools together in the setup just to get it working.

What Nightwatch aims to provide is a combination of both approaches by extending its cross-browser CLI test runner and built-in assertion library and provide an integrated and easy to use solution. Nightwatch already has support for end-to-end testing in all major browsers and comes with built-in reporting and support for parallelism out of the box.

Here’s what Nightwatch does to run a component test:

the CLI test runner launches a real browser and navigates to a basic HTML page (the test renderer)
inside the test renderer, it injects the Vue or React test utils and then mounts the component which needs to be tested, optionally specifying a list of plugins–in case of Vue, such as a store or a router– or props in case of React
once the component has been successfully rendered a reference to the DOM element will be sent back to the Nightwatch CLI runner
the CLI runner continues running the test in the same way it does for end-to-end testing; the assertions are run in the Node.js context

Rendering a Vue component using Nightwatch

What is Vite?

Vite is an extremely fast build tool for modern web applications, initially created for Vue.js apps but now with support for React and other UI frameworks as well. Vite is the French word for fast, which is appropriate because among the available front-end build tools, Vite is the fastest and also one of the easiest build tools to use.

If you have used tools like Babel or Webpack you may be familiar with the problems that arise from the complexity of the build setup and the slow startup times. Vite has somehow managed to eliminate all these issues by providing a tool that it's already configured out of the box and which leverages the new capabilities of modern browsers to handle ES Modules directly, so there's no need of using tools like Babel.

In addition, Vite is using ESBuild under the hood for bundling the Javascript code and related assets, which appears to be the fastest among the bunch by a great deal.

ESBuild performance metrics; source: https://esbuild.github.io

How Does it Work?

At the moment, component testing in Nightwatch is available via our new vite-plugin-nightwatch plugin in projects that are using Vite. Our initial aim is to support React and Vue components so we can gather some initial feedback from the community.

Once Vue and React support is firmly in place, we will expand to add other frameworks like Svelte and Lit.

Installation

For now, the new Vite plugin can be installed from NPM with:

npm install vite-plugin-nightwatch

Configuration

Update your Vite configuration:

import { defineConfig } from 'vite'
import nightwatchPlugin from 'vite-plugin-nightwatch'

export default defineConfig({
  plugins: [
    // ... other plugins, such as vue() or react()
    nightwatchPlugin()
  ]
})

vite.config.jsc

Update your Nightwatch config and add the plugin to the list:

module.exports = {
  plugins: ['vite-plugin-nightwatch'],
  // ... other nightwatch settings
}

nightwatch.conf.js

For more usage details, examples, and API docs, head over to the Github page:

Complete Examples

We've also put together example projects for both React and Vue which show how the plugin can be used.

Both of these projects are under constant development so make to keep an eye on them for updates.

Vue Component Example Test

Here's a basic example of a Vue component test. It's testing a basic Form component written in Vue which is available here, as part of the plugin:

describe('Render Vue Component test', function() {
  let formComponent;

  before(async function() {
    formComponent = await browser.mountVueComponent('/test/components/vue/Form.vue', {});
  });

  it('checks the vue component', function(browser) {
    browser.expect.element(formComponent).to.be.present;
    browser.setValue('#movie-input', 'Fargo');

    const inputEl = formComponent.find('input[type="radio"][value="3"]');

    browser.expect(inputEl).to.be.present;

    browser.click(inputEl);

    browser.expect(formComponent.property('rating')).to.equal('3');
    browser.expect(formComponent.property('title')).to.be.a('string')
        .and.equal('Fargo');
  });

});

Example Vue component test available here: https://github.com/nightwatchjs/vite-plugin-nightwatch/blob/main/test/specs/vue/testVueComponent.js

You can run this as a any other Nightwatch test or just clone/fork the project and run the included tests:

git clone git@github.com:nightwatchjs/vite-plugin-nightwatch.git
npm install
npx nightwatch test/specs/vue/testVueComponent.js --env vue

Make sure the Vite dev server is up and running, with:

npm run dev

Running the Vite dev-server programmatically from Nightwatch

It is also possible to start the Vite dev server from the Nightwatch global before hook and close it in the after hook.

Our plugin does just that in order to run its own tests and details are provided in the Readme. Future Nightwatch plugin releases will provide this functionality out of the box and you will be able run component tests wheather or not your project is based on Vite.

Debugging Component Tests

Debugging component tests in Nightwatch isn't as straightforward as debugging a regular Node.js application or service, since Nightwatch needs to inject the code to render to component into the browser.

However, for when running the tests in Chrome, you can use the DevTools to do debugging directly in the browser. For this purpose, Nightwatch provide 2 CLI flags:

--devtools - when this is on, the Chrome DevTools will open automatically
--debug - this will cause the test execution to pause right after the component is rendered

Advantages of using Nightwatch for Component Testing

The main advantage and motivation for using Nightwatch for component testing is ease of use and consistency with how end-to-end testing is done. With Nightwatch you have everything installed so there is nothing more to configure in terms of testing.

And since the testing is done by the CLI test runner, we have also access to OS-level APIs and we also have access to the Vite runner APIs, so we could do more advanced integration directly between Vite and Nightwatch, but we’ll leave that one for a future update.

The only disadvantage would be that it’s going to be a bit slower than JSDom renderer and possibly also the Karma Runner. However, I believe it compensates by offering a more reliable and easy to use solution, better reporting and overall better experience. Once you enable parallelism, then I think the speed will not be a problem any longer.