Running Spectron Tests in Visual Studio Team Services Build

Having end-to-end tests and running them locally can definitely be beneficial, but one of the main reasons to have them is to run them automatically during a nightly build. Getting these tests to run in Visual Studio Team Services isn’t trivial. Here’s some tips on how to get them going.

Setting up build definitions

When first setting up a build for your Spectron tests all the build definitions need to be manually entered. However, the first step is to actually specify what repository you want to use. This gets set when you start your first build definition.

Select the “Empty” template when the dialog pops up.

From there you can then choose what repository you want to from VSTS or even choose a repository from GitHub.

Here we can set up our steps to have our tests run. If you recall from our [first post on Spectron with our demo project I set up an npm run script to run our tests. In this case, we only have two tasks to create for our build definition:

• npm install
• npm run

Setting up the build agent

So the easy part is done. I only say that, though, because this next part was a bit tricky and wasn’t very well documented on what to do to get these types of tests to run. Because of that, I actually had to ask on StackOverflow for help. Luckily, some Microsoft folks were kind enough to share their knowledge.

The best thing to do is to follow these steps that is already documented by Microsoft to set up a build agent, if you don’t already have one. These tests can’t be run on a hosted build agent since they require a machine to run the application. I was able to set this up locally without any issues.

Make note here that the build agent must be run in interactive mode instead of as a service. The tests won’t run if the agent is running as a service. If you already have a build agent on a remote machine, login to it and stop the service (it will start with ‘VSO Agent’) and run the run.cmd file to have it run interactively. Once this is going, you’ll see something similar to the below on your console window.

Also, in VSTS, make sure that the build definition is set to the correct build agent that you set up when going through Microsoft’s documentation.

Running the build

Now all that’s needed is to queue up a new build! The interactive should be running and listening for jobs and you’ll get some nice success messages showing that everything is working perfectly. If you set your build agent locally, the application will start up and the tests will be executed.

Depending on your project you may have to make some additional steps in your build definition, but hopefully this post gives you a good way to get up and run your tests in Visual Studio Team Services.