Getting started with SpecFlow, WatiN, ATDD and BDD

After completing my Scrum Master course I felt it was high time I review some of our existing engineering practices and consider ways to improve.

Whilst we practice strict test-first TDD and have 100% code coverage as part of our “definition of done” the one area that we’ve been missing is automated UI testing and acceptance testing. After all, high code coverage doesn’t mean a thing if the requirements are not met.

ATDD & BDD

This motivated me to rediscover what is known as Acceptance Test Driven Development or ATDD which is about defining automatable tests from a customer perspective that closely reflect the requirements described in the User Stories. This is also associated to Behaviour-Driven Development (BDD).

The cool kids in the Ruby world caught on to this way back when we were still doing web forms development.

Some of the Tools available in this space are:

RSpec (Ruby)
Cucumber (Ruby)
Robot Framework (Java & Python)  
FITNESSE
WatiR Web UI Testing Framework for Ruby

Of all these tools & frameworks Cucumber is the most interesting in that it allows to describe behavioural tests in plain (non-technical) language allowing to the tests to be authored by the customers or customer proxies.

But What About .NET?

So I’ve mentioned a couple of the leading tools and frameworks in this area which are available in the Java and dynamic language world. In the .NET world there are a number of frameworks around which are either inspired by or ports of their counterparts.

NSpec (Inspired by RSpec)
SpecFlow (Inspired by Cucumber and conforms to Gherkin syntax)
WatiN (Inspired by WatiR)

NSpec and RSpec focus on BDD style Unit-Testing where as Cucumber and SpecFlow are more suited to acceptance tests.

For that reason I’m going to be looking at SpecFlow and WatiN for the remained of this post.

SpecFlow

For those of you without any knowledge of Cucumber or SpecFlow essentially how it works is that you specify Features which are files in your project.
Features firstly define the requirement from a customers perspective using non-techie vocabulary and then specify the scenarios or steps using the Given, When, Then words.

Here is a simple example of a how a feature can be written (I’m not an expert on this language yet).

Feature: Authentication
    In order to allow site personalization
    As a user
    I want to be able to login 

Scenario: Navigation to Home 
    When I navigate to /Home/Index 
    Then I should be on the home page

Scenario: Logging in
    Given I am on the home page
    And I have entered will into the username field
    And I have entered test into the password field 
    When I press submit
    Then the result should be a Logged In title on the screen 

 

Relating this back to Agile you can think of a feature as a user story. 

It should be noted that developers should not be writing these, they should be written by either the customer or a customer proxy such as a tester or business analyst. If these are written by non-technical people then you can ensure that they are focusing on the requirement and not the technical implementation.

Setting up the Test Project

After downloading and installing SpecFlow (make sure to close Visual Studio when doing so) you’ll need to create a standard Unit Test project. e.g. SpecFlowDemo.AcceptanceTests.

Add a reference to TechTalk.SpecFlow.dll which you can find in C:\Program Files\TechTalk\SpecFlow.

If you’re using MSTest then you’ll need to add the following to your App.Config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
  </configSections>
  <specFlow>
    <unitTestProvider name="MsTest" />
  </specFlow>
</configuration>

 

I like to create a separate folder for the Features and Steps.

At this point your project should be looking something like.

 

Defining a Feature

As I mentioned earlier a feature should try and map to a single user story.

For this example I’m going to create a Search feature which searches google for BDD related articles.

Add New Item > SpecFlow Feature File

This will add a Feature file which comprises of the .feature part and the .cs code-behind. Make sure you don’t touch the .cs file as this contains designer generated code.

The Feature

For this fictitious example I’m going to use a Search feature. This is described below using the Gherkin language.

Feature: Search
    In order to find articles on BDD
    As a BDD fanatic 
    I want to enter a keyword into a search engine and be shown a list of related websites


Scenario: Navigate to Search Engine
    When I enter http://www.google.com in the address bar
    Then I should be on the home page


Scenario: Perform search
    Given I am on the home page
    And I have entered BDD into the keyword textbox
    When I press the btnG button
    Then I should see a list of articles related to BDD 

Adding the Steps

Now we are going to add a Step definition for each of the scenarios described in the Feature file.

Right Click on Project > Add New Item > SpecFlow Step Defintion

Once you’ve added the Step you have to add the methods for each of the Given, When, Then lines in your scenario and annotate them with the correct attribute.

The key thing here is that the class needs to be annotated with the Binding attribute.

You can use the (.*) expression to represent variables coming from your Feature file. 

NavigateToSearchEngine Step

[Binding]
 public class NavigateToSearchEngine
 {
     [When("I enter (.*) in the address bar")]
     public void when_i_enter_the_url(string url)
     {
     }

     [Then("I should be on the home page")]
     public void then_i_should_be_on_the_home_page()
     {
     }
 }

 

PerformSearch Step

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
           
        }

        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string button)
       {

       }
    }

 

WatiN

Now that we have our Feature and Steps defined in order to execute the UI tests we’re going to need WatiN. For more info on WatiN check out the documentation.

Download WatiN

Add references to Waitn.Core.dll & Interop.SHDocVw.dll which can be found in C:\Program Files\WatiN Test Recorder

Ensure that Interop.SHDocVw has Embed Interop Types set to False

Static Browser Class

This class is a static helper class for accessing the browser. WatiN has support for Internet Explorer & Firefox but I haven’t been able to get it to work with Firefox 4 I just tend to use the IE implementation.

#region

using TechTalk.SpecFlow;
using WatiN.Core;

#endregion

namespace SpecFlowDemo.AcceptanceTests
{
    [Binding]
    public class WebBrowser
    {
        public static Browser Current
        {
            get
            {
                if (!ScenarioContext.Current.ContainsKey("browser"))
                {
                    ScenarioContext.Current["browser"] = new IE();
                }
                return (Browser) ScenarioContext.Current["browser"];
            }
        }

        [AfterScenario]
        public static void Close()
        {
            if (ScenarioContext.Current.ContainsKey("browser"))
            {
                Current.Close();
            }
        }
    }
}

 

Implementing the Steps

 

NavigateToSearchEngine

    [Binding]
    public class NavigateToSearchEngine
    {
        [When("I enter (.*) in the address bar")]
        public void when_i_enter_the_url(string url)
        {
            WebBrowser.Current.GoTo(url);
        }

        [Then("I should be on the home page")]
        public void then_i_should_be_on_the_home_page()
        {
            Assert.AreEqual(WebBrowser.Current.Title, "Google");
            Assert.IsTrue(WebBrowser.Current.TextFields.Exists(tf => tf.Name == "q"));
        }
    }

 

PerformSearch

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
            WebBrowser.Current.GoTo("http://www.google.com");
        }

       
        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {
            var field = WebBrowser.Current.TextField(tf => tf.Name == "q");
            
            field.TypeText(keyword);

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string buttonName)
       {
           var button = WebBrowser.Current.Button(b => b.Name == buttonName);

           button.Click();
           
           WebBrowser.Current.WaitForComplete(); 

       }

       [Then("I should see a list of articles related to (.*)")]
       public void then_i_should_see_a_list_articles_related_to(string keyword)
       {
           WebBrowser.Current.WaitUntilContainsText("Advanced search");
           
           var searchDiv = WebBrowser.Current.Div("res");

           Assert.IsTrue(searchDiv.Children().Count == 3); 
           
       }
    }

 

Running the Tests

Now you’re ready to run the tests and with a bit of luck they should both pass. You will notice during the tests that Internet Explorer will open and you can actually watch the UI test happen.

 

Console Output

You get a nice descriptive console output as well which is nice.

When I enter http://www.google.com in the address bar
-> done: NavigateToSearchEngine.when_i_enter_the_url("http://www.google...") (15.5s)
Then I should be on the home page
-> done: NavigateToSearchEngine.then_i_should_be_on_the_home_page() (2.9s)

Given I am on the home page
-> done: PerformSearch.given_i_am_on_the_home_page() (6.4s)
And I have entered BDD into the keyword textbox
-> done: PerformSearch.and_i_have_entered("BDD") (3.2s)
When I press the btnG button
-> done: PerformSearch.when_i_press_the_button("btnG") (0.3s)
Then I should see a list of articles related to BDD
-> done: PerformSearch.then_i_should_see_a_list_articles_related_to("BDD") (1.4s)

 

The Code

You can download the code here.

Conclusion

If you’re not doing ATDD or Automated UI Testing then I strongly urge you to consider SpecFlow and WaitN which can add huge value to your testing process. I’m really looking forward to expanding our definition of done to include these.

There is also a WatiN Test Recorder which can assist in automating some of your tests or at the least show you the capability of WatiN. At last look though it doesn’t seem to be an active project anymore.