Using Cucumber to Build Console Applications Test-First

Cucumber

Paul Rayner

@thepaulrayner

Company Logo

Our Goal

Create a simple Ruby command line utility for changing the extension for a set of files in a specified folder

Pair Exercise

Come up with a realistic example of how our rename program should work

Specification by Example

Why Examples?

Focus on understanding the business domain
Improve collaboration with non-technical people
Become tests

Examples

From Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing by Gojko Adzic

Exploring Scenarios

First Example

Replace every file in a folder that ends with the extension jpeg with the new extension jpg

Be Concrete

Folder name: photos
Existing extension: jpeg
New extension: jpg

No, Even More Concrete!

If the following files are in the photos folder:

a.jpeg
b.jpeg
c.jpeg

Then after the bulkrename command runs we should see these three files renamed to:

a.jpg
b.jpg
c.jpg

Another Example

Update the extension to md for Markdown text files with TXT and txt extensions

What About the Future?

(Possible) Future Scenarios

Will need to handle more sophisticated file renaming schemes and options
Since we will need these capabilities, shouldn’t we build them now?

NO! Don’t Get Distracted!

Focus for now on the simple case: renaming extensions

CLI Usage

    bulkrename photos jpeg jpg

Good CLIs

Characteristics

Have a clear and concise purpose
Are easy to use
Are helpful
Play well with others
Delight casual users
Make configuration easy
Distribute painlessly
Are easy to maintain
Are well tested

From Build Awesome Command-Line Applications in Ruby: Control Your Computer, Simplify Your Life by David Bryant Copeland

Cucumber

How it Works

TODO: Diagram of Cucumber assets: feature file and step def files

Feature File

Create features/bulkrename.feature

Feature: Bulk Rename Command Line Utility
  In order to perform bulk renames of files
  As a newcomer to Cucumber
  I want to be able to rename multiple files in a folder by file extension

  Scenario: Rename files in specified folder by extension
   Given an empty file named "photos/a.jpeg"
   And an empty file named "photos/b.jpeg"
   And an empty file named "photos/c.jpeg"
   When I run `bulkrename photos jpeg jpg`
   Then the following files should exist:
    | photos/a.jpg |
    | photos/b.jpg |
    | photos/c.jpg |

Run Cucumber

cucumber features/bulkrename.feature

Run Cucumber

Cucumber (correctly) reports that none of the steps are yet implemented as step definitions yet

Aruba

What is Aruba?

Ruby gem that enables you to use Cucumber to test-drive CLIs.
Provides a set of predefined step definitions to use in our scenarios.

Installing Aruba

gem install aruba

Add Aruba Library

Add to env.rb:

require 'aruba/cucumber'

Using Aruba

Feature: Bulk Rename Command Line Utility
  In order to perform bulk renames of files
  As a newcomer to Cucumber
  I want to be able to rename multiple files in a folder by file extension

  Scenario: Rename files in specified folder by extension # features/bulkrename.feature:6
    Given an empty file named "photos/a.jpeg"             # aruba-0.5.1/lib/aruba/cucumber.rb:27
    And an empty file named "photos/b.jpeg"               # aruba-0.5.1/lib/aruba/cucumber.rb:27
    And an empty file named "photos/c.jpeg"               # aruba-0.5.1/lib/aruba/cucumber.rb:27

Using Aruba (continued)

    When I run `bulkrename photos jpeg jpg`               # aruba-0.5.1/lib/aruba/cucumber.rb:60
      No such file or directory - bulkrename (Aruba::LaunchError)
      features/bulkrename.feature:10:in `When I run `bulkrename photos jpeg jpg`'
    Then the following files should exist:                # aruba-0.5.1/lib/aruba/cucumber.rb:256
      | photos/a.jpg |
      | photos/b.jpg |
      | photos/c.jpg |

Failing Scenarios:
cucumber features/bulkrename.feature:6 # Scenario: Rename files in specified folder by extension

1 scenario (1 failed)
5 steps (1 failed, 1 skipped, 3 passed)
0m0.050s

Our New Script

Create Bulkrename

#!/usr/bin/env ruby
$: << File.expand_path("../lib/", __FILE__)

Make it Executable

`chmod +x bulkrename`

Aruba looks for scripts in `./bin`

mkdir bin
mv bulkrename bin/

cucumber features/bulkrename.feature

Run it

Scenario Fails

Feature: Bulk Rename Command Line Utility
  In order to perform bulk renames of files
  As a newcomer to Cucumber
  I want to be able to rename multiple files in a folder by file extension

  Scenario: Rename files in specified folder by extension # features/bulkrename.feature:6
    Given an empty file named "photos/a.jpeg"             # aruba-0.5.1/lib/aruba/cucumber.rb:27
    And an empty file named "photos/b.jpeg"               # aruba-0.5.1/lib/aruba/cucumber.rb:27
    And an empty file named "photos/c.jpeg"               # aruba-0.5.1/lib/aruba/cucumber.rb:27
    When I run `bulkrename photos jpeg jpg`               # aruba-0.5.1/lib/aruba/cucumber.rb:60

Scenario Fails (Continued)

    Then the following files should exist:                # aruba-0.5.1/lib/aruba/cucumber.rb:256
      | photos/a.jpg |
      | photos/b.jpg |
      | photos/c.jpg |
      expected file?("photos/a.jpg") to return true, got false (RSpec::Expectations::ExpectationNotMetError)
      features/bulkrename.feature:11:in `Then the following files should exist:'

Failing Scenarios:
cucumber features/bulkrename.feature:6 # Scenario: Rename files in specified folder by extension

1 scenario (1 failed)
5 steps (1 failed, 4 passed)
0m0.327s

Minimalist implementation

How Minimalist?

Hard code the command arguments for now. Yes, hard code them!
Refactor later to pull folder name, file extension and replacement file extension from the arguments (once we are sure the rename is working correctly)

Update Bulkrename

#!/usr/bin/env ruby
$: << File.expand_path("../lib/", __FILE__)

Dir.foreach("photos") do |file_name|
 type = File.extname(file_name).gsub(/^\./, '')
 if type == "jpeg"
   old_name = "photos/" + file_name
   new_name = "photos/" + File.basename(file_name, ".#{type}") + ".jpg"
   File.rename(old_name, new_name)
 end
end

Success!

Feature: Bulk Rename Command Line Utility
  In order to perform bulk renames of files
  As a newcomer to Cucumber
  I want to be able to rename multiple files in a folder by file extension

  Scenario: Rename files in specified folder by extension # features/bulkrename.feature:6
    Given an empty file named "photos/a.jpeg"             # aruba-0.5.1/lib/aruba/cucumber.rb:27
    And an empty file named "photos/b.jpeg"               # aruba-0.5.1/lib/aruba/cucumber.rb:27
    And an empty file named "photos/c.jpeg"               # aruba-0.5.1/lib/aruba/cucumber.rb:27

Success! (Continued)

    When I run `bulkrename photos jpeg jpg`               # aruba-0.5.1/lib/aruba/cucumber.rb:60
    Then the following files should exist:                # aruba-0.5.1/lib/aruba/cucumber.rb:256
      | photos/a.jpg |
      | photos/b.jpg |
      | photos/c.jpg |

1 scenario (1 passed)
5 steps (5 passed)
0m0.331s

Next Scenario

Examples So Far

Replace every file in photos folder that ends with the extension jpeg with the new extension jpg
Handling Markdown files in textfiles folder ←

Scenario

Handling Markdown files in textfiles folder
- txt to md
- textfiles folder

Add to Feature File

  Scenario: Rename files in specified folder with extension to completely new file type
    Given an empty file named "textfiles/doc1.txt"
    And an empty file named "textfiles/doc2.txt"
    And an empty file named "textfiles/doc3.txt"
    When I run `bulkrename textfiles txt md`
    Then the following files should exist:
    | textfiles/doc1.md |
    | textfiles/doc2.md |
    | textfiles/doc3.md |

Test Fails

Expected…now let’s implement!

Refactor: Remove hardcoding

Extract folder, find_type and replace_type values from the three arguments to the bulkrename command.
Check whether the file extension is the same as find_type for each file in the specified folder
If it is, rename it with the replace_type extension.

Refactor

#!/usr/bin/env ruby
$: << File.expand_path("../lib/", __FILE__)

folder = ARGV[0]
find_type = ARGV[1]
replace_type = ARGV[2]

Dir.foreach(folder) do |file_name|
 type = File.extname(file_name).gsub(/^\./, '')
 if type == find_type
   old_name = folder + "/" + file_name
   new_name = folder + "/" + File.basename(file_name, ".#{type}") + "." + replace_type
   File.rename(old_name, new_name)
 end
end

Next Example

Examples So Far

Replace every file in photos folder that ends with the extension jpeg with the new extension jpg
Handling Markdown files in textfiles folder
Case-insensitive replacement ←

Ignoring Case

Replacing TXT and txt with csv in a manner that ignores case

Add to Feature file

  Scenario: Rename files in specified folder by extension ignoring case
    Given an empty file named "textfiles/May-financials.txt"
    And an empty file named "textfiles/June-financials.TXT"
    And an empty file named "textfiles/July-financials.TXT"
    When I run `bulkrename textfiles txt csv`
    Then the following files should exist:
    | textfiles/May-financials.csv |
    | textfiles/June-financials.csv |
    | textfiles/July-financials.csv |

Ignore Case of File Extensions

#!/usr/bin/env ruby
$: << File.expand_path("../lib/", __FILE__)

folder = ARGV[0]
find_type = ARGV[1]
replace_type = ARGV[2]

Dir.foreach(folder) do |file_name|
 type = File.extname(file_name).gsub(/^\./, '')
 if type.downcase == find_type.downcase
   old_name = "#{folder}/" + file_name
   new_name = "#{folder}/" + File.basename(file_name, ".#{type}") + ".#{replace_type}"
   File.rename(old_name, new_name)
 end
end

Adding New Scenarios and Refactoring

Examples So Far

Replace every file in photos folder that ends with the extension jpeg with the new extension jpg
Handling Markdown files in textfiles folder
Case-insensitive replacement
Handle default command output ←

Default Command Output

Ensure is helpful when no arguments are specified:

  @wip
  Scenario: Default script output is correct
    When I run `bulkrename`
    Then the exit status should be 0
    And the output should contain:
    """
    USAGE: bulkrename <folder name> <find extension> <replace extension>
    """

Using Tags

cucumber --tags @wip features/bulkrename.feature

Make it Pass

Add to bulkrename before we pull the values out of the arguments:

if ARGV.size == 0 then
  puts 'USAGE: bulkrename <folder name> <find extension> <replace extension>'
  exit 0
end

New Example

Examples So Far

Replace every file in photos folder that ends with the extension jpeg with the new extension jpg
Handling Markdown files in textfiles folder
Case-insensitive replacement
Handle default command output
Don’t overwrite files ←

New Scenario

  @wip
  Scenario: Do not overwrite existing file(s)
    Given an empty file named "photos/d.jpeg"
    And an empty file named "photos/d.jpg"
    When I run `bulkrename photos jpeg jpg`
    Then the following files should exist:
      | photos/d.jpeg |
      | photos/d.jpg  |

Update Bulkrename

   if not File.exists?(new_name)
     File.rename(old_name, new_name)
   end

Never Overwrite a File?

Examples So Far

Replace every file in photos folder that ends with the extension jpeg with the new extension jpg
Handling Markdown files in textfiles folder
Case-insensitive replacement
Handle default command output
Don’t overwrite files
Prompt to overwrite files ←

Add New Switch

  @wip
  Scenario: Detect an existing file
    Given an empty file named "photos/d.jpeg"
    And an empty file named "photos/d.jpg"
    When I run `bulkrename photos jpeg jpg --askoverwrite`
    Then the output should contain:
    """
    File 'photos/d.jpg' already exists, do you want to overwrite it (y/n)?
    """

Check for Overwrites

    if File.exists?(new_name) && ARGV[3] == "--askoverwrite"
      print "File '#{new_name}' already exists, do you want to overwrite it (y/n)?"
    end
   if not File.exists?(new_name)
     File.rename(old_name, new_name)
   end

Interactive Aruba

  Scenario: Choose to overwrite an existing file
    Given an empty file named "photos/d.jpeg"
      And an empty file named "photos/d.jpg"
      When I run `bulkrename photos jpeg jpg --askoverwrite` interactively
      And I type "yes"
      Then the following files should exist:
        | photos/d.jpg |
      And the output should contain:
      """
      File 'photos/d.jpg' already exists, do you want to overwrite it (y/n)?
      """
      And the output should contain:
      """
      Overwriting file 'photos/d.jpg'
      """

Incomplete!?!?

Don’t worry if the only step you can think to write and implement feels incomplete
You can always refactor it or extend it once you get it working.

Goal-Driven Approach

Narrows the implementation focus for each scenario to accomplishing one distinct, well-defined goal.
Helps us deal with uncertainty, where perhaps we can’t see a clear way forward, but can at least see the next step.

Goal-Driven Approach

Constraining and then subsequently refactoring scenarios in this way:
- Gives us the courage to proceed when the final solution seems vague and perhaps even in doubt
- Minimizes the time spent with broken tests
- Helps us to go faster overall
- Guides us towards a right solution

Make it Pass

And Refactor…

Extract out some new well-named methods and variables
Use File.Join to build the file names instead of conventional string operations

Final Code

#!/usr/bin/env ruby
$: << File.expand_path("../lib/", __FILE__)

def do_overwrite(ask_overwrite, new_name)
  overwrite = false
  if ask_overwrite
    print "File '#{new_name}' already exists, do you want to overwrite it (y/n)?"
    input = STDIN.gets.strip
    overwrite = true if input[0].downcase == 'y'
  end

  puts "Overwriting file '#{new_name}'" if overwrite == true
  overwrite
end

def do_rename(ask_overwrite, new_name)
  return true if not File.exists?(new_name)
  return true if File.exists?(new_name) && do_overwrite(ask_overwrite, new_name)
end

if ARGV.size == 0
  puts 'USAGE: bulkrename <folder name> <find extension> <replace extension>'
  exit 0
end

Continued…

folder = ARGV[0]
find_type = ARGV[1]
replace_type = ARGV[2]

if ARGV[3]
  ask_overwrite = ARGV[3] == "--askoverwrite"
end

Dir.foreach(folder) do |file_name|
  type = File.extname(file_name).gsub(/^\./, '')
  if type.downcase == find_type.downcase
    old_name = File.join(folder, file_name)
    new_name = File.join(folder, File.basename(file_name, ".#{type}") + ".#{replace_type}")
    if do_rename(ask_overwrite, new_name)
      File.rename(old_name, new_name)
    end
  end
end

When Not Overwriting…

  Scenario: Choose not to overwrite an existing file
    Given an empty file named "photos/d.jpeg"
    And an empty file named "photos/d.jpg"
    When I run `bulkrename photos jpeg jpg --askoverwrite` interactively
    And I type "no"
    Then the following files should exist:
      | photos/d.jpeg |
      | photos/d.jpg  |
    And the output should contain:
    """
    File 'photos/d.jpg' already exists, do you want to overwrite it (y/n)?
    """
    But the output should not contain:
    """
    Overwriting file 'photos/d.jpg'
    """

Done! (for now)

Nicely factored, easy-to-understand script
Feature file that covers all the behavior we currently care about

Adding Your Own Steps

Why Your Own?

Treat the steps included by Aruba as examples rather than templates.

Note

There are some improvements we can make to our feature file. Remember, we want to be refactoring our feature file, incorporating our learning and ubiquitous language. As we’ve already mentioned, we should endeavor to write scenarios centered around the goals that we want to accomplish and the immediate tasks that support those goals. We should avoid writing steps that are too finely- grained because they concentrate on the actions that are required to accomplish the tasks. For this reason you should regard

The steps provided out-of-the-box by Aruba will help you get started, but if you only ever use the Aruba steps provided it is likely that you will end up writing Cucumber scenarios that look like a series of imperative actions, instead of capturing in the ubiquitous language what is important in terms of the user’s goals and the tasks required to accomplish these goals.

Improve the Language

  @wip
  Scenario: Default script output is correct
    When I run `bulkrename`
    Then the exit status should be 0
    And the output should contain:
    """
    USAGE: bulkrename <folder name> <find extension> <replace extension>
    """

Can this be improved?

Improve the Language

  @wip
  Scenario: Default script output is correct
    When I run `bulkrename`
    Then the exit status should be 0
    And the output should contain:
    """
    USAGE: bulkrename <folder name> <find extension> <replace extension>
    """

Check for the correct usage message, rather than just looking for certain text:

  @wip
  Scenario: Default script output is correct
    When I run `bulkrename`
    Then the correct usage message should be displayed:
    """
    USAGE: bulkrename <folder name> <find extension> <replace extension> [--askoverwrite]
    """

Our Own Aruba Step Definiton

Then /^the correct usage message should be displayed:$/ do |usage_message|
  assert_partial_output(usage_message, all_output)
  last_exit_status.should == 0
end

Create bulkrename_steps.rb in feature/step_definitions

What About Collaborating?

Consulting with QA

After discussing the script with the tester on our team, we identified other error scenarios which should also be handled in the same basic manner…

New Scenarios

<find extension> is missing (eg. bulkrename photos)
<replace extension> is missing (eg. bulkrename photos txt)
Optional argument/switch has incorrect format (eg. bulkrename photos txt option)
Optional argument has correct format but is invalid (eg. bulkrename photos txt —verbose)

See a Pattern?

Many, many scenarios where only the arguments and error message change
In each case, the arguments and error message are changing together
Can’t we write scenarios to avoid repetitious scenarios?

Scenario Outline

  @wip
  Scenario Outline: Parameters should be present and valid
    Given an empty file named "photos/doc1.txt"
    When I run `bulkrename <arguments>`
    Then the program should exit by displaying the error:
    """
    Error: <message>
    """

  Examples:
    | arguments                            | message                           |
    | photos jpeg jpg --askoverwrite extra | Too many arguments                |
    | documents txt md                     | Folder 'documents' does not exist |

Adding More Examples is Easy Now

  @wip
  Scenario Outline: Parameters should be present and valid
    Given an empty file named "photos/doc1.txt"
    When I run `bulkrename <arguments>`
    Then the program should exit by displaying the error:
    """
    Error: <message>
    """

Adding More Examples is Easy Now (Continued)

  Examples:
    | arguments                            | message                           |
    | photos jpeg jpg --askoverwrite extra | Too many arguments                |
    | documents txt md                     | Folder 'documents' does not exist |
    | photos                               | <find extension> is required      |
    | photos txt                           | <replace extension> is required   |
    | photos txt md extra                  | Invalid option format 'extra'     |
    | photos jpeg jpg --verbose            | Invalid option name '--verbose'   |

Argument Processing Code

if ARGV.size == 0
  puts 'USAGE: bulkrename <folder name> <find extension> <replace extension> [--askoverwrite]'
  exit 0
end
abort 'Error: Too many arguments' if ARGV.size > 4
abort("Error: <find extension> is required") if ARGV.size == 1
abort("Error: <replace extension> is required") if ARGV.size == 2

folder = ARGV[0]
find_type = ARGV[1]
replace_type = ARGV[2]

if ARGV[3]
  abort "Error: Invalid option format '#{ARGV[3]}'" unless ARGV[3][0..1] == "--"
  ask_overwrite = ARGV[3] == "--askoverwrite"
  abort "Error: Invalid option name '#{ARGV[3]}'" unless ask_overwrite
end

abort("Error: Folder '#{folder}' does not exist") unless File.exists?(folder)

Conclusion

Crafting our steps in this way allows us to focus on the important details and abstract away the inconsequential ones
It takes practice to know how much detail to put in a scenario step versus what goes into the step definition
With collaborative practice and refactoring your scenarios as you learn more about the business domain a good balance can be achieved.

Resources

One Book, Three Classes

BDD with Cucumber, published by Addison Wesley
- Coming Soon! Early Access available this month.
Ignite BDD - One-Day Class
BDD Fuel - 2 Hour Workshop
BDD Blaze - 2.5-Day Workshop

BDD with Cucumber

My new book for coders and testers
Covers Cucumber in Ruby, Java and .NET
Coming soon, from Addison Wesley
Sign up for Early Access review

Ignite BDD - One-Day Class

New to BDD and Cucumber?
Get your whole team and management fired up about doing BDD
Talk to me about scheduling one for your team

BDD Fuel - 2 Hour Workshop

Started using Cucumber and want to improve?
I’ll sit with your team, review your work, answer your questions and coach your team on doing BDD with Cucumber effectively
Talk to me about scheduling one for your team

BDD Blaze - 2.5-Day Workshop

Your team will take a huge step towards mastery of BDD
Intensive training, coaching and leadership mentoring for your team in BDD with Cucumber
Talk to me about scheduling one for your team

Company Logo

Business Card