Skip to main content

Getting started with SpecFlow, WatiN, ATDD and BDD

After completing my Scrum Master course I felt it was high time I review some of our existing engineering practices and consider ways to improve.

Whilst we practice strict test-first TDD and have 100% code coverage as part of our “definition of done” the one area that we’ve been missing is automated UI testing and acceptance testing. After all, high code coverage doesn’t mean a thing if the requirements are not met.

ATDD & BDD

This motivated me to rediscover what is known as Acceptance Test Driven Development or ATDD which is about defining automatable tests from a customer perspective that closely reflect the requirements described in the User Stories. This is also associated to Behaviour-Driven Development (BDD).

The cool kids in the Ruby world caught on to this way back when we were still doing web forms development.

Some of the Tools available in this space are:

RSpec (Ruby)
Cucumber (Ruby)
Robot Framework (Java & Python)  
FITNESSE
WatiR Web UI Testing Framework for Ruby

Of all these tools & frameworks Cucumber is the most interesting in that it allows to describe behavioural tests in plain (non-technical) language allowing to the tests to be authored by the customers or customer proxies.

But What About .NET?

So I’ve mentioned a couple of the leading tools and frameworks in this area which are available in the Java and dynamic language world. In the .NET world there are a number of frameworks around which are either inspired by or ports of their counterparts.

NSpec (Inspired by RSpec)
SpecFlow (Inspired by Cucumber and conforms to Gherkin syntax)
WatiN (Inspired by WatiR)

NSpec and RSpec focus on BDD style Unit-Testing where as Cucumber and SpecFlow are more suited to acceptance tests.

For that reason I’m going to be looking at SpecFlow and WatiN for the remained of this post.

SpecFlow

For those of you without any knowledge of Cucumber or SpecFlow essentially how it works is that you specify Features which are files in your project.
Features firstly define the requirement from a customers perspective using non-techie vocabulary and then specify the scenarios or steps using the Given, When, Then words.

Here is a simple example of a how a feature can be written (I’m not an expert on this language yet).

Feature: Authentication
    In order to allow site personalization
    As a user
    I want to be able to login 

Scenario: Navigation to Home 
    When I navigate to /Home/Index 
    Then I should be on the home page

Scenario: Logging in
    Given I am on the home page
    And I have entered will into the username field
    And I have entered test into the password field 
    When I press submit
    Then the result should be a Logged In title on the screen 

 

Relating this back to Agile you can think of a feature as a user story. 

It should be noted that developers should not be writing these, they should be written by either the customer or a customer proxy such as a tester or business analyst. If these are written by non-technical people then you can ensure that they are focusing on the requirement and not the technical implementation.

Setting up the Test Project

After downloading and installing SpecFlow (make sure to close Visual Studio when doing so) you’ll need to create a standard Unit Test project. e.g. SpecFlowDemo.AcceptanceTests.

Add a reference to TechTalk.SpecFlow.dll which you can find in C:\Program Files\TechTalk\SpecFlow.

If you’re using MSTest then you’ll need to add the following to your App.Config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
  </configSections>
  <specFlow>
    <unitTestProvider name="MsTest" />
  </specFlow>
</configuration>

 

I like to create a separate folder for the Features and Steps.

At this point your project should be looking something like.

project

 

Defining a Feature

As I mentioned earlier a feature should try and map to a single user story.

For this example I’m going to create a Search feature which searches google for BDD related articles.

Add New Item > SpecFlow Feature File

addnewitem_search_feature

This will add a Feature file which comprises of the .feature part and the .cs code-behind. Make sure you don’t touch the .cs file as this contains designer generated code.

The Feature

For this fictitious example I’m going to use a Search feature. This is described below using the Gherkin language.

Feature: Search
    In order to find articles on BDD
    As a BDD fanatic 
    I want to enter a keyword into a search engine and be shown a list of related websites


Scenario: Navigate to Search Engine
    When I enter http://www.google.com in the address bar
    Then I should be on the home page


Scenario: Perform search
    Given I am on the home page
    And I have entered BDD into the keyword textbox
    When I press the btnG button
    Then I should see a list of articles related to BDD 

Adding the Steps

Now we are going to add a Step definition for each of the scenarios described in the Feature file.

Right Click on Project > Add New Item > SpecFlow Step Defintion

addnewitem_step_definition

Once you’ve added the Step you have to add the methods for each of the Given, When, Then lines in your scenario and annotate them with the correct attribute.

The key thing here is that the class needs to be annotated with the Binding attribute.

You can use the (.*) expression to represent variables coming from your Feature file. 

NavigateToSearchEngine Step

[Binding]
 public class NavigateToSearchEngine
 {
     [When("I enter (.*) in the address bar")]
     public void when_i_enter_the_url(string url)
     {
     }

     [Then("I should be on the home page")]
     public void then_i_should_be_on_the_home_page()
     {
     }
 }

 

PerformSearch Step

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
           
        }

        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string button)
       {

       }
    }

 

WatiN

Now that we have our Feature and Steps defined in order to execute the UI tests we’re going to need WatiN. For more info on WatiN check out the documentation.

Download WatiN

Add references to Waitn.Core.dll & Interop.SHDocVw.dll which can be found in C:\Program Files\WatiN Test Recorder

Ensure that Interop.SHDocVw has Embed Interop Types set to False

Static Browser Class

This class is a static helper class for accessing the browser. WatiN has support for Internet Explorer & Firefox but I haven’t been able to get it to work with Firefox 4 I just tend to use the IE implementation.

#region

using TechTalk.SpecFlow;
using WatiN.Core;

#endregion

namespace SpecFlowDemo.AcceptanceTests
{
    [Binding]
    public class WebBrowser
    {
        public static Browser Current
        {
            get
            {
                if (!ScenarioContext.Current.ContainsKey("browser"))
                {
                    ScenarioContext.Current["browser"] = new IE();
                }
                return (Browser) ScenarioContext.Current["browser"];
            }
        }

        [AfterScenario]
        public static void Close()
        {
            if (ScenarioContext.Current.ContainsKey("browser"))
            {
                Current.Close();
            }
        }
    }
}

 

Implementing the Steps

 

NavigateToSearchEngine

    [Binding]
    public class NavigateToSearchEngine
    {
        [When("I enter (.*) in the address bar")]
        public void when_i_enter_the_url(string url)
        {
            WebBrowser.Current.GoTo(url);
        }

        [Then("I should be on the home page")]
        public void then_i_should_be_on_the_home_page()
        {
            Assert.AreEqual(WebBrowser.Current.Title, "Google");
            Assert.IsTrue(WebBrowser.Current.TextFields.Exists(tf => tf.Name == "q"));
        }
    }

 

PerformSearch

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
            WebBrowser.Current.GoTo("http://www.google.com");
        }

       
        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {
            var field = WebBrowser.Current.TextField(tf => tf.Name == "q");
            
            field.TypeText(keyword);

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string buttonName)
       {
           var button = WebBrowser.Current.Button(b => b.Name == buttonName);

           button.Click();
           
           WebBrowser.Current.WaitForComplete(); 

       }

       [Then("I should see a list of articles related to (.*)")]
       public void then_i_should_see_a_list_articles_related_to(string keyword)
       {
           WebBrowser.Current.WaitUntilContainsText("Advanced search");
           
           var searchDiv = WebBrowser.Current.Div("res");

           Assert.IsTrue(searchDiv.Children().Count == 3); 
           
       }
    }

 

Running the Tests

Now you’re ready to run the tests and with a bit of luck they should both pass. You will notice during the tests that Internet Explorer will open and you can actually watch the UI test happen.

test_results

 

Console Output

You get a nice descriptive console output as well which is nice.

When I enter http://www.google.com in the address bar
-> done: NavigateToSearchEngine.when_i_enter_the_url("http://www.google...") (15.5s)
Then I should be on the home page
-> done: NavigateToSearchEngine.then_i_should_be_on_the_home_page() (2.9s)

Given I am on the home page
-> done: PerformSearch.given_i_am_on_the_home_page() (6.4s)
And I have entered BDD into the keyword textbox
-> done: PerformSearch.and_i_have_entered("BDD") (3.2s)
When I press the btnG button
-> done: PerformSearch.when_i_press_the_button("btnG") (0.3s)
Then I should see a list of articles related to BDD
-> done: PerformSearch.then_i_should_see_a_list_articles_related_to("BDD") (1.4s)

 

The Code

You can download the code here.

Conclusion

If you’re not doing ATDD or Automated UI Testing then I strongly urge you to consider SpecFlow and WaitN which can add huge value to your testing process. I’m really looking forward to expanding our definition of done to include these.

There is also a WatiN Test Recorder which can assist in automating some of your tests or at the least show you the capability of WatiN. At last look though it doesn’t seem to be an active project anymore.

Popular posts from this blog

ASP.NET MVC Release Candidate - Upgrade issues - Spec#

First of all, great news that the ASP.NET MVC Release Candidate has finally been released.  Full credit to the team for the hard work on this.  You can get the download here  However this is the first time I have had upgrade issues.  Phil Haack has noted some of the issues here   If like me you have lot's of CTP's and Add-Ins then you might experience some pain in Uninstalling MVC Beta on Vista SP1  This is the list of Add-Ins / CTP's I had to uninstall to get it to work  Spec# PEX Resharper 4.1  Sourcelinks ANTS Profiler 4   Can't say I'm too impressed as it wasted over an hour of my time.  As it turned out Spec# turned out to be the offending culprit, it's forgiveable to have issues with a third party product but a Microsoft one? Guess no-one on the ASP.NET team has Spec# installed. 

Freeing Disk Space on C:\ Windows Server 2008

  I just spent the last little while trying to clear space on our servers in order to install .NET 4.5 . Decided to post so my future self can find the information when I next have to do this. I performed all the usual tasks: Deleting any files/folders from C:\windows\temp and C:\Users\%UserName%\AppData\Local\Temp Delete all EventViewer logs Save to another Disk if you want to keep them Remove any unused programs, e.g. Firefox Remove anything in C:\inetpub\logs Remove any file/folders C:\Windows\System32\LogFiles Remove any file/folders from C:\Users\%UserName%\Downloads Remove any file/folders able to be removed from C:\Users\%UserName%\Desktop Remove any file/folders able to be removed from C:\Users\%UserName%\My Documents Stop Windows Update service and remove all files/folders from C:\Windows\SoftwareDistribution Deleting an Event Logs Run COMPCLN.exe Move the Virtual Memory file to another disk However this wasn’t enough & I found the most space was

Consuming the SSRS ReportExecutionService from a .NET Client

  I’ve just finished writing a nice wrapper which internally calls the SSRS ReportExecutionService to generate reports. Whilst it was fairly simple to implement there has been some major changes between 2005 and 2008 and the majority of online and documentation is based on the 2005 implementation. The most important change is that the Report Server and Report Manager are no longer hosted in IIS which will be a welcomed change to Sys Admins but makes the security model and hosting model vastly different. So far I’ve yet to figure out how to allow Anonymous Access, if anyone knows how to do this leave a comment and it will be most appreciated. Getting Started To get started you’ll want to add a service reference to http://localhost/ReportServer_SQL2008/ReportExecution2005.asmx where ReportServer_SQL2008 is the name you configure in the Reporting Services Configuration Manager. The Web Application files are located in C:\Program Files\Microsoft SQL Server\MSRS10.SQL2008\R