Skip to main content

Getting started with SpecFlow, WatiN, ATDD and BDD

After completing my Scrum Master course I felt it was high time I review some of our existing engineering practices and consider ways to improve.

Whilst we practice strict test-first TDD and have 100% code coverage as part of our “definition of done” the one area that we’ve been missing is automated UI testing and acceptance testing. After all, high code coverage doesn’t mean a thing if the requirements are not met.

ATDD & BDD

This motivated me to rediscover what is known as Acceptance Test Driven Development or ATDD which is about defining automatable tests from a customer perspective that closely reflect the requirements described in the User Stories. This is also associated to Behaviour-Driven Development (BDD).

The cool kids in the Ruby world caught on to this way back when we were still doing web forms development.

Some of the Tools available in this space are:

RSpec (Ruby)
Cucumber (Ruby)
Robot Framework (Java & Python)  
FITNESSE
WatiR Web UI Testing Framework for Ruby

Of all these tools & frameworks Cucumber is the most interesting in that it allows to describe behavioural tests in plain (non-technical) language allowing to the tests to be authored by the customers or customer proxies.

But What About .NET?

So I’ve mentioned a couple of the leading tools and frameworks in this area which are available in the Java and dynamic language world. In the .NET world there are a number of frameworks around which are either inspired by or ports of their counterparts.

NSpec (Inspired by RSpec)
SpecFlow (Inspired by Cucumber and conforms to Gherkin syntax)
WatiN (Inspired by WatiR)

NSpec and RSpec focus on BDD style Unit-Testing where as Cucumber and SpecFlow are more suited to acceptance tests.

For that reason I’m going to be looking at SpecFlow and WatiN for the remained of this post.

SpecFlow

For those of you without any knowledge of Cucumber or SpecFlow essentially how it works is that you specify Features which are files in your project.
Features firstly define the requirement from a customers perspective using non-techie vocabulary and then specify the scenarios or steps using the Given, When, Then words.

Here is a simple example of a how a feature can be written (I’m not an expert on this language yet).

Feature: Authentication
    In order to allow site personalization
    As a user
    I want to be able to login 

Scenario: Navigation to Home 
    When I navigate to /Home/Index 
    Then I should be on the home page

Scenario: Logging in
    Given I am on the home page
    And I have entered will into the username field
    And I have entered test into the password field 
    When I press submit
    Then the result should be a Logged In title on the screen 

 

Relating this back to Agile you can think of a feature as a user story. 

It should be noted that developers should not be writing these, they should be written by either the customer or a customer proxy such as a tester or business analyst. If these are written by non-technical people then you can ensure that they are focusing on the requirement and not the technical implementation.

Setting up the Test Project

After downloading and installing SpecFlow (make sure to close Visual Studio when doing so) you’ll need to create a standard Unit Test project. e.g. SpecFlowDemo.AcceptanceTests.

Add a reference to TechTalk.SpecFlow.dll which you can find in C:\Program Files\TechTalk\SpecFlow.

If you’re using MSTest then you’ll need to add the following to your App.Config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
  </configSections>
  <specFlow>
    <unitTestProvider name="MsTest" />
  </specFlow>
</configuration>

 

I like to create a separate folder for the Features and Steps.

At this point your project should be looking something like.

project

 

Defining a Feature

As I mentioned earlier a feature should try and map to a single user story.

For this example I’m going to create a Search feature which searches google for BDD related articles.

Add New Item > SpecFlow Feature File

addnewitem_search_feature

This will add a Feature file which comprises of the .feature part and the .cs code-behind. Make sure you don’t touch the .cs file as this contains designer generated code.

The Feature

For this fictitious example I’m going to use a Search feature. This is described below using the Gherkin language.

Feature: Search
    In order to find articles on BDD
    As a BDD fanatic 
    I want to enter a keyword into a search engine and be shown a list of related websites


Scenario: Navigate to Search Engine
    When I enter http://www.google.com in the address bar
    Then I should be on the home page


Scenario: Perform search
    Given I am on the home page
    And I have entered BDD into the keyword textbox
    When I press the btnG button
    Then I should see a list of articles related to BDD 

Adding the Steps

Now we are going to add a Step definition for each of the scenarios described in the Feature file.

Right Click on Project > Add New Item > SpecFlow Step Defintion

addnewitem_step_definition

Once you’ve added the Step you have to add the methods for each of the Given, When, Then lines in your scenario and annotate them with the correct attribute.

The key thing here is that the class needs to be annotated with the Binding attribute.

You can use the (.*) expression to represent variables coming from your Feature file. 

NavigateToSearchEngine Step

[Binding]
 public class NavigateToSearchEngine
 {
     [When("I enter (.*) in the address bar")]
     public void when_i_enter_the_url(string url)
     {
     }

     [Then("I should be on the home page")]
     public void then_i_should_be_on_the_home_page()
     {
     }
 }

 

PerformSearch Step

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
           
        }

        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string button)
       {

       }
    }

 

WatiN

Now that we have our Feature and Steps defined in order to execute the UI tests we’re going to need WatiN. For more info on WatiN check out the documentation.

Download WatiN

Add references to Waitn.Core.dll & Interop.SHDocVw.dll which can be found in C:\Program Files\WatiN Test Recorder

Ensure that Interop.SHDocVw has Embed Interop Types set to False

Static Browser Class

This class is a static helper class for accessing the browser. WatiN has support for Internet Explorer & Firefox but I haven’t been able to get it to work with Firefox 4 I just tend to use the IE implementation.

#region

using TechTalk.SpecFlow;
using WatiN.Core;

#endregion

namespace SpecFlowDemo.AcceptanceTests
{
    [Binding]
    public class WebBrowser
    {
        public static Browser Current
        {
            get
            {
                if (!ScenarioContext.Current.ContainsKey("browser"))
                {
                    ScenarioContext.Current["browser"] = new IE();
                }
                return (Browser) ScenarioContext.Current["browser"];
            }
        }

        [AfterScenario]
        public static void Close()
        {
            if (ScenarioContext.Current.ContainsKey("browser"))
            {
                Current.Close();
            }
        }
    }
}

 

Implementing the Steps

 

NavigateToSearchEngine

    [Binding]
    public class NavigateToSearchEngine
    {
        [When("I enter (.*) in the address bar")]
        public void when_i_enter_the_url(string url)
        {
            WebBrowser.Current.GoTo(url);
        }

        [Then("I should be on the home page")]
        public void then_i_should_be_on_the_home_page()
        {
            Assert.AreEqual(WebBrowser.Current.Title, "Google");
            Assert.IsTrue(WebBrowser.Current.TextFields.Exists(tf => tf.Name == "q"));
        }
    }

 

PerformSearch

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
            WebBrowser.Current.GoTo("http://www.google.com");
        }

       
        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {
            var field = WebBrowser.Current.TextField(tf => tf.Name == "q");
            
            field.TypeText(keyword);

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string buttonName)
       {
           var button = WebBrowser.Current.Button(b => b.Name == buttonName);

           button.Click();
           
           WebBrowser.Current.WaitForComplete(); 

       }

       [Then("I should see a list of articles related to (.*)")]
       public void then_i_should_see_a_list_articles_related_to(string keyword)
       {
           WebBrowser.Current.WaitUntilContainsText("Advanced search");
           
           var searchDiv = WebBrowser.Current.Div("res");

           Assert.IsTrue(searchDiv.Children().Count == 3); 
           
       }
    }

 

Running the Tests

Now you’re ready to run the tests and with a bit of luck they should both pass. You will notice during the tests that Internet Explorer will open and you can actually watch the UI test happen.

test_results

 

Console Output

You get a nice descriptive console output as well which is nice.

When I enter http://www.google.com in the address bar
-> done: NavigateToSearchEngine.when_i_enter_the_url("http://www.google...") (15.5s)
Then I should be on the home page
-> done: NavigateToSearchEngine.then_i_should_be_on_the_home_page() (2.9s)

Given I am on the home page
-> done: PerformSearch.given_i_am_on_the_home_page() (6.4s)
And I have entered BDD into the keyword textbox
-> done: PerformSearch.and_i_have_entered("BDD") (3.2s)
When I press the btnG button
-> done: PerformSearch.when_i_press_the_button("btnG") (0.3s)
Then I should see a list of articles related to BDD
-> done: PerformSearch.then_i_should_see_a_list_articles_related_to("BDD") (1.4s)

 

The Code

You can download the code here.

Conclusion

If you’re not doing ATDD or Automated UI Testing then I strongly urge you to consider SpecFlow and WaitN which can add huge value to your testing process. I’m really looking forward to expanding our definition of done to include these.

There is also a WatiN Test Recorder which can assist in automating some of your tests or at the least show you the capability of WatiN. At last look though it doesn’t seem to be an active project anymore.

Comments

  1. Very useful post! Thank you!
    I also found some more information here
    http://volaresystems.com/Blog/post/2013/01/06/SpecFlow-and-WatiN-Worst-Practices-What-NOT-to-do

    Hope it helps somebody.

    ReplyDelete

Post a Comment

Popular posts from this blog

Freeing Disk Space on C:\ Windows Server 2008

I just spent the last little while trying to clear space on our servers in order to install .NET 4.5. Decided to post so my future self can find the information when I next have to do this. I performed all the usual tasks: Deleting any files/folders from C:\windows\temp and C:\Users\%UserName%\AppData\Local\TempDelete all EventViewer logs Save to another Disk if you want to keep themRemove any unused programs, e.g. FirefoxRemove anything in C:\inetpub\logsRemove any file/folders C:\Windows\System32\LogFilesRemove any file/folders from C:\Users\%UserName%\DownloadsRemove any file/folders able to be removed from C:\Users\%UserName%\DesktopRemove any file/folders able to be removed from C:\Users\%UserName%\My DocumentsStop Windows Update service and remove all files/folders from C:\Windows\SoftwareDistributionDeleting an Event Logs Run COMPCLN.exe Move the Virtual Memory file to another disk However this wasn’t enough & I found the most space was cleared by using the Disk Cleanup to…

Consuming the SSRS ReportExecutionService from a .NET Client

I’ve just finished writing a nice wrapper which internally calls the SSRS ReportExecutionService to generate reports.
Whilst it was fairly simple to implement there has been some major changes between 2005 and 2008 and the majority of online and documentation is based on the 2005 implementation. The most important change is that the Report Server and Report Manager are no longer hosted in IIS which will be a welcomed change to Sys Admins but makes the security model and hosting model vastly different. So far I’ve yet to figure out how to allow Anonymous Access, if anyone knows how to do this leave a comment and it will be most appreciated. Getting StartedTo get started you’ll want to add a service reference to http://localhost/ReportServer_SQL2008/ReportExecution2005.asmx where ReportServer_SQL2008 is the name you configure in the Reporting Services Configuration Manager. The Web Application files are located in C:\Program Files\Microsoft SQL Server\MSRS10.SQL2008\Reporting Servic…

Log Shipping in SQL Server Express 2008

When you entrust your Applications and Data to the Cloud Based Service Providers such as GoGrid and Amazon EC2 it becomes absolutely critical to employ a strict Disaster Recovery strategy. As part of of our strategy it was imperative that we have a failover for our SQL Server. The only problem was that SQL Server Express 2008 doesn’t support Log Shipping.
Well technically it does but SQL Server Express does not have SQL Server Agent. Now I’m a big fan of SQL Server, but when it comes to doing something a bit more complicated the Express editions limitations become a bit of a problem, that’s why it’s time to find creative solutions. To set up Log Shipping the first step is to Backup your database and then Restore it on another instance making sure set the Recovery state as “RESTORE WITH STANDBY”. See belowTo get this going you’ll need three stored procedures, one in the Master database and two in the slave database. Now when you are trying to do this across Networks and Hosting Pro…