Recently I’ve been working on a website that is a single page AJAX driven application that’s essentially just a frontend to an API, rather than doing any of it’s own processing.
For testing it I wanted to be able to test without hitting the API but since the application is so javascript heavy and makes use of various complex workarounds for various browser issues, testing it using the usual methods never worked and only “real” browsers cut the mustard.
I’ve tried a number of headless browsers such as zombie.js but they’ve all had issues so I’ve settled on using selenium for testing in a real browser. The other issue is that most web application testing systems assume at least some kind of backend system that you can then mock the http calls of. However in my case the entire app is client side and has no server at all, so doing that wasn’t an option.
The solution I came up with was a cobbled together selection of rake tasks, shell scripts, a custom node.js server and some hooks in cucumber to set up fixtures. I run my tests against a live server through a recording proxy to get the results of the API calls, then format the results into little fixture files to get loaded by the fixtures server based on tags on my cucumber scenarios. It’s quite a flexible setup now it’s running, the only chore is writing the fixtures, though hopefully I can knock up a quick script to generate the fixtures automatically from my proxy recordings.
I wont go into the details of running cucumber with capybara and selenium, since those are already well documented elsewhere, such as on the capybara github page. However, I’ll run through my scripts.
First off, I have my bash script to actually setup my server and run all my tests, it goes a little something like this:
UPDATE: Made some amendments thanks to suggestions from Ralph Corderoy in the comments.
UPDATE 2: Removed set -eu line since breaking out on errors means not shutting down the fixture server when tests fail.
|
|
#!/bin/bash
echo "Starting fixtures server..."
node ./fixture-server.js &
PID=$!
sleep 1
echo "Running tests..."
rake features
echo "Shutting down fixtures server..."
curl http://127.0.0.1:7357/_fixtures/shutdown &> /dev/null
sleep 1
echo "Force shutdown of fixtures server..."
kill $PID &> /dev/null || true |
What is happening in the above should be fairly obvious, but essentially I’m running my fixtures server and capture it’s PID so I can shut it down later. I wait for a bit, then execute my tests via a Rakefile, then I send a command to the fixture server to shutdown, wait for a bit, then force a shutdown in case it’s not done it already. Simples!
The fixture server can be found in my random scripts repository on github. It’s hard-coded to run on port 7357 but that’s easy to change. I’ve tried to document how it works in the code but a gist of it is that the fixture server will server static assets but fall through on all other requests to look at whatever fixtures it has loaded. By default, it will have loaded no fixtures so will 404. However, it has some special URLs for loading in fixtures. On /_fixtures/load/:fixture the server will load a json fixture file with the name :fixture.json. Multiple fixtures can be loaded for the same path and the server will work it’s way through them, serving each one once until it has no more where it will then 404 again. Fixtures can be cleared from memory with a call to /_fixtures/clear and you can inspect the currently loaded fixtures by calling /_fixtures/inspect. A call to /_fixtures/shutdown will instruct the server to shut itself down. At the moment the fixture server only supports loading fixtures for GET and POST requests as that’s all I’ve needed, but it’s easy to extend that.
The fixture files themselves are very simple. Here is an example one:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
[
{
"method": "GET",
"path": "/test",
"status": 403,
"headers": {
"Content-Type": "application/json; charset=utf-8"
},
"content": "{\"success\":false,\"error\":true,\"error_code\":403,\"error_message\":\"Unauthorized\"}"
},
{
"method": "POST",
"path": "/test/post",
"status": 302,
"headers": {
"Location": "/"
},
"content":""
},
{
"method": "GET",
"path": "/test",
"status": 200,
"headers": {
"Content-Type": "application/json; charset=utf-8"
},
"content": "{\"success\":true,\"message\":\"Authorized\"}"
}
] |
In the above fixture file, the server is setup to respond with a failure message the first time it is accessed with a GET on /test, a redirect the first time it is accessed with a POST on /test/post and a success message the second time it is accessed with a GET on /test.
In cucumber, using the stuff I’ve mentioned above, we can now do cool stuff like configuring fixtures for each scenario. I use something like this to setup my fixtures:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
require 'net/http'
module FixtureHelpers
def load_fixture(fixture)
Net::HTTP.get(::TEST_HOST, "/_fixtures/load/#{fixture}", ::TEST_PORT)
end
def clear_fixtures
Net::HTTP.get(::TEST_HOST, '/_fixtures/clear', ::TEST_PORT)
end
end
World(FixtureHelpers)
Before('@notauthed') do
load_fixture(:notauthed)
end
Before('@authed') do
load_fixture(:authed)
end
After do
clear_fixtures
end |
I slap that in a .rb file in my features/support directory and I can now use the @authed and @notauthed tags to load the appropriate fixtures into the server, like in the following feature:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
Feature: Login page
In order to access the site
As a customer
I want to be able to login
@notauthed
Scenario: Should get redirected to the login page when not logged in
Given I am on the home page
Then I should be on the login page
@notauthed
Scenario: Login without any details
Given I am on the login page
When I press "Login »"
Then the "Username" field should have errors
And the "Password" field should have errors
And I should see "Whoops!"
And I should see "Your login details could not be authenticated"
@notauthed
Scenario: Login with bad details
Given I am on the login page
And I fill in "Username" with "invalid-user"
And I fill in "Password" with "invalid-pass"
When I press "Login »"
Then the "Username" field should have errors
And the "Password" field should have errors
And I should see "Whoops!"
And I should see "Your login details could not be authenticated"
@notauthed @authed
Scenario: Login with valid details
Given I am on the login page
And I fill in "Username" with "valid-user"
And I fill in "Password" with "valid-pass"
When I press "Login »"
Then I should be on the home page |
You’ll notice that I can do cool things like load multiple fixtures by specifying multiple tags. The fixtures get loaded in the order that that tags appear so I can create fairly complex fixture scenarios from simple individual fixtures.
All in all I’ve found it a reasonably nice simple way of being able to test client-side only apps that drive APIs without having to hit the actual API server. Obviously if you are developing your own API server as well, you should be writing tests for the API server in addition to the client, but this way to can keep them nicely de-coupled in your tests so you don’t have to hit live data, which based on your project might be impossible to test against anyway.
Lorcian is discussing. 
). There are many stories I want to finish and that a few people on Protagonize want to see the conclusions of, so I need to get back on the saddle. I had said to myself last year I would try and seriously pursue publication, but that hasn’t happened. I’m hoping this year, now I have more time due to driving to work, things will be different, but if I’m entirely honest with myself, I doubt I’ll put the required time or effort into making a serious attempt. Not only is it a lot of work, it’s quite intimidating. I do want to finish at least one novel though by the end of the year and hopefully get into the meat of editing it with an eye to publish it independently, for kicks, since I’m not sure I’m even interested in mainstream publication or actually making a living from writing.
Nice write-up, though I stumbled where you nouned apply! My brain is wired to know that word has one meaning and sound. Perhaps ‘appli’, if not ‘app’? On the bash you may like to know &> does the job of > followed by 2>&1, e.g. &>/dev/null, and ‘set -eu’ is always handy at the start of a bash script, e.g. curl(1) fails, though you’d expect the kill(1) to fail; kill $PID &>/dev/null || true.