Jun 27
2009

Natural Language Acceptance Testing with Cucumber

Posted by Nick Plante in TestingRuby

nap

Cucumber is a testing framework for writing high-level natural language descriptions of software functionality. Translated to programmer speak this means that it's a compelling tool with which to write integration tests, and can be a big part of your Behavior Driven Development (BDD) strategy...

We all know that good integration / acceptance tests are written much like storycarded user stories are, and in Cucumber they take the following form. You declare a particular feature, followed by a number of scenarios that demonstrate particular behavior of that feature. Here's a sample:

    Feature: User Registration
      In order to keep track of my bookmarks
      As a user
      I want to have an account

      Scenario: Sign up with invite code
        Given That I do not have a user account
        And I have a valid invite code "abc123"
        When I go to the signup page
        And I fill in "Username" with "MyName"
        And I fill in "Password" with "Password"
        And I fill in "Password Confirmation" with "Password"
        And I fill in "Invitation Code" with "abc123"
        And I press "Signup"
        Then I should see "Thank you for signing up"

Note the use of the Given / When / Then template here. It's used repeatedly in Cucumber tests, asserting what we know about what the user wants / has, what actions they take, and what the expected outcome is. How these English-language steps are taken and converted into runnable test cases is the magic of Cucumber and its Gherkin domain specific language (Gherkin supports many langauges in addition to English too). Cucumber contains a number of step definitions, and when combined with the powerful WebRat web integration testing package, you'll find that it natively understands many commands used to visit particular pages, traverse the DOM, and fill in and submit forms.

Anyway, if Cucumber isn't sure how to interpret a particular step, the runner process will tell you that the step is undefined and prompt you to define those steps yourself. For instance, in the example above, Cucumber would have no way of knowing what a "valid invite code" is, so you would need to define that. The step definition might look something like this

    Given /^I have a valid invite code \"(.+)\"$/ do |code|
      Invitation.create!(:code => code)
    end

In true BDD fashion, you're encouraged to implement the behavior specifications first, and continue to implement behavior (and the appropriate step definitions) until the criteria established with your stakeholders is satisfied.

The difference between using Cucumber for integration tests and using something like WebRat straight-up is largely syntactical. But since the ultimate goal of Cucumber (and of BDD in general) is to communicate the behavior of your system to everyone involved in a project, this is a pretty important distinction. Sometimes your stakeholders won't be folks with a technical skillset, and in those cases Cucumber's natural language stories present an easy way for everyone to make sure they're on the same page regarding overall system functionality

Do Cucumber acceptance tests take extra time to write? Absolutely. But the reward that you get in terms of lowering the communication barrier with your customers could definitely be worth the effort. Next time you start a project, start with writing proper integration tests, with Cucumber, in English. Review them with your clients. Make sure that the stories make sense to them. Maybe even get them to participate in the spec writing process itself.

Then, simply implement the steps and iteratively develop your software to satisfy them, piece by piece, until all the known specifications pass. It sounds simple, and it is -- but it's also a very powerful approach. I know I've been tackling software projects in a vein similar to this for quite some time, but, toying with Cucumber just now I can see how this extra layer of abstraction can make the functionality specification process even easier, while at the same time helping to ensure that there aren't any communication problems with the clients or stakeholders.

Make note that Cucumber is _not_ a replacement for a unit testing framework, but rather a higher-level tool for constructing and testing scenarios, including customer acceptance tests. It works hand-in-hand with RSpec, Test/Unit, or whatever your unit testing framework of choice is. I should also probably say that, although Cucumber is implemented in Ruby, it can certainly be used to test (at the web integration level) web-based software written in other languages!

Anyway, that's it for me. For more information, make sure to check out the official Cucumber website which is packed with helpful examples. Ryan Bates of Railscasts fame has also put together a couple short screencasts (155, 159) that were really helpful to me when researching the topic.



Comments (1)Add Comment
Thanks for this
written by Mike Swaine, June 28, 2009
I had a chance to get somewhat familiar with Cucumber recently, and I think you hit the thumb on the nail, er, nail on the head. Cucumber is about communicating unambiguously among all stakeholders in a project. I can't speak with any authority about its effectiveness, but it looks promising.

Write comment
You must be logged in to a comment. Please register if you do not have an account yet.

busy

Get your FREE Subscription to Dr. Dobb’s Digest today!

Dobbs Code Talk Quick Poll

This time next year, your most important operating system (host and/or target) will be:

Look Who's Code Talking


John Towell III
City: Brookfield

Arunachalam Muthu
City: Hosur

Ron Lambkin
City: Kansas City

Bil Lewis
City: Cambridge

Ville Ingman
City: Turku

Richard Bullington-McGuire
City: Arlington

Dobbs Code Talk Tags

.NET abstraction Ada Adobe Agile Ajax algorithm Algorithmic complexity ALM Analogical reasoning Android Anecdotes Apple Application Development AppStore Architecture and Design ARM Artificial Intelligence Artificial Life Assembler Programming Audio files AVX AWK Banking Bazaar Best Practices Blender Books Brain computer interfacing Build C C Programming C Sharp Cartoon Category theory Cellular automata Clojure Cloud Computing Cobol Cocoa Coder Of The Month Cognition as compression Collaboration Common Process/Frameworks Compilers Computational humour Computational narrative Computational politics Computer Science Computers in art computing pioneers concurrency Conferences Consciousness research Contest Contest140 contests CPlusPlus crime CSharp D Programming Data Centers Databases Debugging Delphi Deployment design Design Patterns Digital Signal Processing Distributed Django Documentation DSL dynamic language Eclipse EDA education Emacs Embedded Systems Encryption engineering Erlang Etymology Excel exception handling Facebook Financial computing Five Questions Flash Flash Lite Flex Forth Fortran Fraud FreeBSD Fun Functional Programming gadgets Games Gender Git gnuplot Go Google Graphics GUI hardware Heron High School High-Performance Computing History Holographic reduced representations HTML5 Humanity Humour Hungarian Notation Identity Inkscape Innovation Intel Interview iPhone J2EE Java JavaFX JavaOne JavaScript language engineering Legal lex LINQ Linux Lisp Literate Programming Logic Programming m4 Mainframes Make Mathematica Mercurial Mesh messaging Metaprogramming Microsoft MID Miscellaneous Musings ML Mobile Software Mobility modeling modular programming multicore Music MVC myblog Natural Language Processing Networking Neural networks newspeak Nokia numerical computing Object Rexx ObjectiveC Office Office 2007 Online spreadsheets OOP Open Source Openaccess publishing OpenBSD OpenSolaris Operating Systems Optimization Oracle Pair Programming Parallelism Concurrency Parsing Pascal Patents Patterns Performance Perl PHP Podcast Pop11 Poplog Privacy Processing Productivity Programming Language Implementation Programming Language One Programming language semantics Programming Languages Programming Style Project Management Prolog Psychology Public understanding of science puzzle Python QA Quantum Computing Quotes Rails Realtime recls Requirements Research practice REST Review RIA rich internet applications Robotics Ruby SaaS Software as a service Scala Schadenfreude Science fiction Screencast Scripting SD Best Practices Search Security Semantic Web Silverlight Snobol SOA social Social Networks Society for the Study of Artificial Intelligence a Software Development Methodology and Management Songs and poems Spending Priorities Spreadsheets SQL Startups Statistics Storage String pattern matching Survey Teaching Testing The Business of Programming The Dobbs Challenge The Future Theory Topology Transhumanism Travel on the Job Twitter Types Unix Upgrade Usability Use Cases USENET User Experience User Interface Design Version Control video virtual machines Virtualization Visual Studio Visual Studio Sponsored Post WCF Web Development Windows Windows 7 Windows Live Wireless WOA WPF X Window System yacc

Subscribe to Dr. Dobbs Newsletter

Email:
Dr. Dobb's Update
Delivered twice a week, Dr. Dobb's Update provides unbiased and objective news, commentary and technical features spanning the entire software development marketplace.

Latest Comments

Jonathan's Last Day at Sun
For the 8 years I worked there, it was fantastic. I worked there under McNealy and I have undying admiration for the guy. I only knew Jonathan periphe...
Implementing Thread Local Storage on OS ...
Back in the day, I did a fair amount of work with PThreads. Wonderful design. Some quirks, but basically really, really nice. Although I wrote a lot ...
More Technonecrophilia with Snobol One-L...
Yeah, It's probably identical except for the (embedded) copy number, I would think. Once it became freely distributable, the copy I've been distribut...
More Technonecrophilia with Snobol One-L...
There's a spitbol-3.7-win.exe at http://code.google.com/p/spitbol/downloads/list . I found it via Dave Shield's blog page http://daveshields.wordpress...
Jonathan's Last Day at Sun
Sadness.

The Latest From Our Member Blogs

How To Select Trainees
Written by Joel Wiesen   
01/27/10
Hiring the right trainee can be harder than hiring a trained programmer.  One approach is described at my website: http://www.aprtestingservices.com/business/lpat/
 
Technical Job Interviews
Written by Keith Kerlan   
01/20/10
What is the best way to interview for software developer positions?  I've been on both sides of the job interviewing table, but have been on the interviewee side of some not too  great inter
 
Timers/timeouts in multi-threaded event-loops
Written by Christof Meerwald   
01/03/10
The traditional way to integrate timeout handling (or timers) in (single-threaded) event loops was to just pass the appropriate timeout value to the select/poll/epoll syscall. While this works fine
 
C vs C++
Written by Issam Lahlali   
12/04/09
I think that the debate "C vs C++" will end when the two langages died, and each one have its advantages and inconvenients, the choice of one instead of another depend on the application c
 
Great Jobs at CISCO
Written by Brent Rogers   
11/30/09
Hello! I am a recruiter at CISCO. We have a number of great jobopportunities at CISCO right now. Please take a look at the job links listedbelow and please send me an updated resume if you are interes
 
OK Labs, ST-Ericsson, and the Mobile/Wireless Ecosystem
Written by Steve Subar   
11/17/09
Two weeks ago, OK Labs and ST-Ericsson announced the selection of OK Labs as ST-Ericsson's mobile virtualization partner. To earn this coveted position, OK Labs prevailed in a rigorous evaluation
 
C++ Ninjas Needed in Santa Clara, California
Written by Brent Rogers   
09/30/09
Hello! I am a recruiter at CISCO. Our PostPath teamin Santa Clara is building a new Email SaaS business at CISCO. We are looking forsenior developers with Zimbra expertise to help us accomplish this t
 
Fighting Fragmentation with Mobile Virtualization
Written by Steve Subar   
09/21/09
Last week Motorola and T-Mobile announced the launch of a new and innovative Android-based smartphone, the Cliq. This attractive, feature-rich slider handset happens to build on a chipset and firmware
 
Insights into Router Design: Unit Testing of Networking Protocols
Written by Rajesh Kumar Venkateswaran   
09/07/09
  Unit testing is a software validation methodology through which a programmer tests individual modules or units of source code. If the programmer has been responsible for developing a networ
 
Insights into Router Design: Implementation of Networking Protocols
Written by Rajesh Kumar Venkateswaran   
09/06/09
  Modern data networking consists of a large number of networking protocols, each of which has its own domain of applicability. Some run on end stations (also called hosts), some on enterp
 
Insights and Innovations in Networking
Written by Rajesh Kumar Venkateswaran   
09/05/09
Networking devices such as routers and switches have evolved quite a bit over the past years, both in the service provider network and in the enterprise. It is a challenge to build these devices, bo
 
reddit threads community
Written by Christof Meerwald   
08/30/09
I have just started a threads community over at reddit to cover topics such as multithreading, concurrency and parallel programming. Feel free to join if you are interested. -- cmeerw.org 
 

The Latest From Dr. Dobbs

DDJ