Visual Regression Testing and Environment Comparison with Backstop JS

 As a Front End Web Developer, experimenting with new tools to add to your workflow (and going down the rabbit hole with them!) is all part and parcel of refining your craft. Chris Eccles, Technical Manager at Building Blocks, has been doing just this and has some invaluable insight into CSS Visual Regression using Backstop.JS.

CSS Visual Regression testing is the process of running automated visual test comparisons on pages or elements in your projects. Using Backstop.JS, Chris has discovered that this tool is intuitive, allowing quick configuration to allow you to get up and rolling quickly.

Backstop.JS serves your tests via a webpage which gives you the visual feedback needed for targeting bugs caused from CSS related issues. These comparisons can uncover bugs you’d otherwise not learn about until it’s too late. A very useful tool to have in your Front End arsenal, wouldn’t you agree?

Chris has been sharing his insights with the BB team and wanted to share with our blog readers also. So, sit back and enjoy the ride through the wonderful world of Backstop.JS.

Visual Regression Testing and Environment Comparison with Backstop.JS.png

What is Backstop.JS exactly?

Backstop.JS is a open source project to run visual tests using headless browsers to capture screenshots.

Originally ran using either PhantomJS or SlimerJS headless browser libraries. These were implementations of WebKit (Chrome) or Gecko (Firefox).

From release v3.0.19 this now supports Chrome headless.

The latest release has a lot of stability improvements v3.0.27

Git Hub URL : github.com/garris/BackstopJS

What Features Does Backstop.JS Offer?

  • Allows capture and comparison of screenshots from headless browser
  • Excellent comparison tool to identify and highlight differences
  • Allows setup of multiple breakpoints to test responsive sites
  • Uses simple CSS selectors to identify what to capture
  • Pre capture scripts can be enabled to be ran

Visual Regression Testing

Visual regression testing is the process of creating reference images for each component as they are created. This allows a comparison over time to monitor changes.

Ran by the developer on their local development environment after changes made to ensure no regressions issues have occurred due to their changes.

Why is that useful?

  • This allows the developer greater test coverage than they could do manually when ensuring their changes have not had regression impact on other components.
  • Allows greater detail comparison than possible reviewing the site manually. E.g. If the font size is increased by 1px then it will be caught which could easily be missed.
  • Allows developers without a full understanding of the project have a safety net about what the site should look like before their changes.

Environment Comparison

Using visual testing to compare different environments involves comparing one site against another. The reference images are not stored beyond the test.

A reference image set is generated from one site and then another site is compared against them

Why is that useful?

  • To ensure that the HTML created has been integrated within the website correctly and is rendering out in the same way. This highlights mistakes with the integration or conflicting integrations that are not present on the HTML Demo site.
  • Running regression tests against the Staging and Live instances of a site to see where the changes have occurred.
  • When setting up comparison between a demo and dev site powered by a CMS this can make the tests brittle so the tests pages need to be isolated from changes in content as possible.

Here is a slideshare that Chris has put together to get down to the nitty gritty of how to implement this too:

 
 

If you have any questions for Chris or if you would like to find out a bit more about technical expertise here at Building Blocks, get in touch and let's start a conversation about it! 

 

Any questions?

If you need more information or have any questions just get in touch and we'd be happy to answer them for you.