After completing my Scrum Master course I felt it was high time I review some of our existing engineering practices and consider ways to improve.
Whilst we practice strict test-first TDD and have 100% code coverage as part of our “definition of done” the one area that we’ve been missing is automated UI testing and acceptance testing. After all, high code coverage doesn’t mean a thing if the requirements are not met.
ATDD & BDD
This motivated me to rediscover what is known as Acceptance Test Driven Development or ATDD which is about defining automatable tests from a customer perspective that closely reflect the requirements described in the User Stories. This is also associated to Behaviour-Driven Development (BDD).
The cool kids in the Ruby world caught on to this way back when we were still doing web forms development.
Some of the Tools available in this space are:
RSpec (Ruby)
Cucumber (Ruby)
Robot Framework (Java & Python)
FITNESSE
WatiR Web UI Testing Framework for Ruby
Of all these tools & frameworks Cucumber is the most interesting in that it allows to describe behavioural tests in plain (non-technical) language allowing to the tests to be authored by the customers or customer proxies.
But What About .NET?
So I’ve mentioned a couple of the leading tools and frameworks in this area which are available in the Java and dynamic language world. In the .NET world there are a number of frameworks around which are either inspired by or ports of their counterparts.
NSpec (Inspired by RSpec)
SpecFlow (Inspired by Cucumber and conforms to Gherkin syntax)
WatiN (Inspired by WatiR)
NSpec and RSpec focus on BDD style Unit-Testing where as Cucumber and SpecFlow are more suited to acceptance tests.
For that reason I’m going to be looking at SpecFlow and WatiN for the remained of this post.
SpecFlow
For those of you without any knowledge of Cucumber or SpecFlow essentially how it works is that you specify Features which are files in your project.
Features firstly define the requirement from a customers perspective using non-techie vocabulary and then specify the scenarios or steps using the Given, When, Then words.
Here is a simple example of a how a feature can be written (I’m not an expert on this language yet).
Feature: Authentication In order to allow site personalization As a user I want to be able to login Scenario: Navigation to Home When I navigate to /Home/Index Then I should be on the home page Scenario: Logging in Given I am on the home page And I have entered will into the username field And I have entered test into the password field When I press submit Then the result should be a Logged In title on the screen
Relating this back to Agile you can think of a feature as a user story.
It should be noted that developers should not be writing these, they should be written by either the customer or a customer proxy such as a tester or business analyst. If these are written by non-technical people then you can ensure that they are focusing on the requirement and not the technical implementation.
Setting up the Test Project
After downloading and installing SpecFlow (make sure to close Visual Studio when doing so) you’ll need to create a standard Unit Test project. e.g. SpecFlowDemo.AcceptanceTests.
Add a reference to TechTalk.SpecFlow.dll which you can find in C:\Program Files\TechTalk\SpecFlow.
If you’re using MSTest then you’ll need to add the following to your App.Config
<?xml version="1.0" encoding="utf-8" ?> <configuration> <configSections> <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/> </configSections> <specFlow> <unitTestProvider name="MsTest" /> </specFlow> </configuration>
I like to create a separate folder for the Features and Steps.
At this point your project should be looking something like.
Defining a Feature
As I mentioned earlier a feature should try and map to a single user story.
For this example I’m going to create a Search feature which searches google for BDD related articles.
Add New Item > SpecFlow Feature File
This will add a Feature file which comprises of the .feature part and the .cs code-behind. Make sure you don’t touch the .cs file as this contains designer generated code.
The Feature
For this fictitious example I’m going to use a Search feature. This is described below using the Gherkin language.
Feature: Search In order to find articles on BDD As a BDD fanatic I want to enter a keyword into a search engine and be shown a list of related websites Scenario: Navigate to Search Engine When I enter http://www.google.com in the address bar Then I should be on the home page Scenario: Perform search Given I am on the home page And I have entered BDD into the keyword textbox When I press the btnG button Then I should see a list of articles related to BDD
Adding the Steps
Now we are going to add a Step definition for each of the scenarios described in the Feature file.
Right Click on Project > Add New Item > SpecFlow Step Defintion
Once you’ve added the Step you have to add the methods for each of the Given, When, Then lines in your scenario and annotate them with the correct attribute.
The key thing here is that the class needs to be annotated with the Binding attribute.
You can use the (.*) expression to represent variables coming from your Feature file.
NavigateToSearchEngine Step
[Binding] public class NavigateToSearchEngine { [When("I enter (.*) in the address bar")] public void when_i_enter_the_url(string url) { } [Then("I should be on the home page")] public void then_i_should_be_on_the_home_page() { } }
PerformSearch Step
[Binding] public class PerformSearch { [Given("I am on the home page")] public void given_i_am_on_the_home_page() { } [Given("I have entered (.*) into the keyword textbox")] public void and_i_have_entered(string keyword) { } [When("I press the (.*) button")] public void when_i_press_the_button(string button) { } }
WatiN
Now that we have our Feature and Steps defined in order to execute the UI tests we’re going to need WatiN. For more info on WatiN check out the documentation.
Download WatiN
Add references to Waitn.Core.dll & Interop.SHDocVw.dll which can be found in C:\Program Files\WatiN Test Recorder
Ensure that Interop.SHDocVw has Embed Interop Types set to False
Static Browser Class
This class is a static helper class for accessing the browser. WatiN has support for Internet Explorer & Firefox but I haven’t been able to get it to work with Firefox 4 I just tend to use the IE implementation.
#region using TechTalk.SpecFlow; using WatiN.Core; #endregion namespace SpecFlowDemo.AcceptanceTests { [Binding] public class WebBrowser { public static Browser Current { get { if (!ScenarioContext.Current.ContainsKey("browser")) { ScenarioContext.Current["browser"] = new IE(); } return (Browser) ScenarioContext.Current["browser"]; } } [AfterScenario] public static void Close() { if (ScenarioContext.Current.ContainsKey("browser")) { Current.Close(); } } } }
Implementing the Steps
NavigateToSearchEngine
[Binding] public class NavigateToSearchEngine { [When("I enter (.*) in the address bar")] public void when_i_enter_the_url(string url) { WebBrowser.Current.GoTo(url); } [Then("I should be on the home page")] public void then_i_should_be_on_the_home_page() { Assert.AreEqual(WebBrowser.Current.Title, "Google"); Assert.IsTrue(WebBrowser.Current.TextFields.Exists(tf => tf.Name == "q")); } }
PerformSearch
[Binding] public class PerformSearch { [Given("I am on the home page")] public void given_i_am_on_the_home_page() { WebBrowser.Current.GoTo("http://www.google.com"); } [Given("I have entered (.*) into the keyword textbox")] public void and_i_have_entered(string keyword) { var field = WebBrowser.Current.TextField(tf => tf.Name == "q"); field.TypeText(keyword); } [When("I press the (.*) button")] public void when_i_press_the_button(string buttonName) { var button = WebBrowser.Current.Button(b => b.Name == buttonName); button.Click(); WebBrowser.Current.WaitForComplete(); } [Then("I should see a list of articles related to (.*)")] public void then_i_should_see_a_list_articles_related_to(string keyword) { WebBrowser.Current.WaitUntilContainsText("Advanced search"); var searchDiv = WebBrowser.Current.Div("res"); Assert.IsTrue(searchDiv.Children().Count == 3); } }
Running the Tests
Now you’re ready to run the tests and with a bit of luck they should both pass. You will notice during the tests that Internet Explorer will open and you can actually watch the UI test happen.
Console Output
You get a nice descriptive console output as well which is nice.
When I enter http://www.google.com in the address bar
-> done: NavigateToSearchEngine.when_i_enter_the_url("http://www.google...") (15.5s)
Then I should be on the home page
-> done: NavigateToSearchEngine.then_i_should_be_on_the_home_page() (2.9s)
Given I am on the home page
-> done: PerformSearch.given_i_am_on_the_home_page() (6.4s)
And I have entered BDD into the keyword textbox
-> done: PerformSearch.and_i_have_entered("BDD") (3.2s)
When I press the btnG button
-> done: PerformSearch.when_i_press_the_button("btnG") (0.3s)
Then I should see a list of articles related to BDD
-> done: PerformSearch.then_i_should_see_a_list_articles_related_to("BDD") (1.4s)
The Code
You can download the code here.
Conclusion
If you’re not doing ATDD or Automated UI Testing then I strongly urge you to consider SpecFlow and WaitN which can add huge value to your testing process. I’m really looking forward to expanding our definition of done to include these.
There is also a WatiN Test Recorder which can assist in automating some of your tests or at the least show you the capability of WatiN. At last look though it doesn’t seem to be an active project anymore.
Very useful post! Thank you!
ReplyDeleteI also found some more information here
http://volaresystems.com/Blog/post/2013/01/06/SpecFlow-and-WatiN-Worst-Practices-What-NOT-to-do
Hope it helps somebody.
I am happy to find this post very useful for me, as it contains a lot of information. I always prefer to read the quality content I found in you post. Thanks for sharing.
ReplyDeletesoftware testing selenium training
selenium testing training in chennai
Hadoop concepts, Applying modelling through R programming using Machine learning algorithms and illustrate impeccable Data Visualization by leveraging on 'R' capabilities.With companies across industries striving to bring their research and analysis (R&A) departments up to speed, the demand for qualified data scientists is rising.
ReplyDeletedata science training in bangalore
Big Data and Hadoop training Unlike traditional systems, Big Data and Hadoop enables multiple types of analytic workloads to run on the same data, at the same time, at massive scale on industry-standard hardware.myTectra Big Data and Hadoop training is designed to help you become a expert Hadoop developer. myTectra offers Big Data Hadoop Training in Bangalore using Class Room.
hadoop training in bangalore
Looking for best Machine Learning Training in Bangalore then join myTectra the leader in Machine Learning Training in Bangalore. Classroom & Online Training
machine learning training in bangalore
Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work!
ReplyDeletemachine learning course in bangalore
Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites!
ReplyDeletemachine learning course bangalore
Your post is really awesome .it is very helpful for me to develop my skills in a right way
ReplyDeleteSelenium Training in Chennai
Selenium Training in Bangalore
Selenium Training in Coimbatore
Selenium course in Chennai
Selenium Course in Bangalore
Selenium Course in Coimbatore
Software Testing Course in Chennai
Hacking Course in Bangalore
Thanks for posting keep updating it.
ReplyDeleteInformatica MDM Training in Chennai
Informatica mdm training
french courses in chennai
pearson vue
Blockchain Training in Chennai
Spoken English Course in Chennai
spanish institute in chennai
content writing training in chennai
Informatica MDM Training in Velachery
Informatica MDM Training in Tambaram
This Blog is really informative!! keep update more about this...
ReplyDeleteAviation Academy in Chennai
Air Hostess Training in Chennai
Airport Management Courses in Chennai
Best Aviation Academy in Chennai
Ground Staff Training in Chennai
Air Hostess Academy in Chennai
Airport Management Training in Chennai
Airport Ground Staff Training Courses in Chennai
Best Digital Marketing Course Online with over 10+ Certifications, Instructor-led live classes, Case Studies, 20+ advanced modules and a lot more. Please visit our website to know more information.
ReplyDeletehttps://onlineidealab.com/digital-marketing-course-online/
I should thank you for posting this blog on the grounds that the subject is particularly sought after today and everybody needs to find out about specflow. Oracle Fusion Cloud Training
ReplyDeleteAmazing Article, Really useful information to all So, I hope you will share more information to be check and share here.
ReplyDeletePygame Tutorial
Pygame Download
Pygame Install
Matplotlib Python
Matplotlib Tutorial
Matplotlib install
PouchDB Tutorial
What is PouchDB
PouchDB Installation
pouchdb server