Requirements and tests

Awesome Requirements And Tests

I write requirements and tests using CoffeeScript. The CoffeeScript gets compiled and run by a testing tool called mocha.

Simple Requirements

I start by writing requirements, in Coffee:

describe "Examples", ->
  it "should succeed because 1 is not equal to 1"
  it "should fail because 1 is not equal to 2"

describe and it are the names of functions in the mocha library. ()-> or -> indicate that what follows is a function block. So this translates to:

1. Call the describe function, passing two arguments, a string, and a function block. The function block is the series of indented lines that follow.
2. Within the function block, call the it function, passing a descriptive screen.

If I had provided a function block for the it statements, even an empty block, mocha would consider this a test. Without a function block, it’s a “pending” test: in other words, a requirement.

If I invoke mocha on this file, I get this output:

 Examples
  - should succeed because 1 is not equal to 1 
   - should fail because 1 is not equal to 2

0 passing (5 ms)
2 pending

Converting A Requirement To A Test

I turn the requirements into tests by adding a blocks of code, also written in CoffeeScript. To pass in the block I must first change a line like:

it "should succeed because 1 is not equal to 1"

to

 it "should succeed because 1 is equal to 1", -> #Note the -> at the end

Now whatever is indented underneath the it statement is the test.

mocha uses another library called chai that supports several different styles for text expression. The one I prefer is theshould format. Writing a test that passes and one that fails in should format gives me this:

describe "Examples", ->
  it "should succeed because 1 is equal to 1", ->
       1.should.equal 1
  it "should fail because 1 is not equal to 2", ->
       1.should.equal 2

For the first test I could also write:

  1.should.equal(1)

or I could use expect format:

  expect(1).to.equal(1)

or the assert format. The last parameter is the message to display if the test fails, which this one cannot:

   assert( 1 === 1, "one is not equal to one")
   #or 
   assert.equal( 1, 1, "one is not equal to one")

Whatever format I use, the result will look like this:

Examples
    ✓ should succeed because 1 is not equal to 1 
    1) should fail because 1 is not equal to 2

 1 passing (28ms)
 0 pending
 1 failing

  1) Examples should fail because 1 is not equal to 2:

      AssertionError: expected 1 to equal 2
      + expected - actual

      +2
      -1

A Real Example

I am working on the specs for my build system. I use a tool called gulp to run my builds. As make uses makefiles, gulp uses gulpfiles. So here’s how I describe my gulpfile:

describe "gulpfile", ->
  it "uses a javascript file that uses 'require' to load a utility file"
  it "uses a utility to read the task files from the tasks directory"
  it "prints the number of files loaded"
  it "uses 'gulp-load-plugins' to load plugins"
  describe "Tasks", ->
    it "has a 'links' task that creates links in the components directory so that Brackets can run on the gulp server"
    it "has a 'mocha:gulpfile' task that runs mocha on this file"
    it "continues running mocha tests even when a test fails"
    it "has a 'mocha' task that runs mocha on all specs in the test directory and sub-directories"
    it "has a less task that runs the less compiler"
    it "has a jade task that converts jade to HTML"
    it "has a coffee task that converts client side coffee code to js"
    it "has a lint task that runs jshint on all coffeescript"
    it "has an html task that runs useref on all html files"
    it "has a clean task that cleans the dist directory before a build"
    it "has a build task"
    it "has a connect task"
    it "has a watch task that rebuilds anything that changes"
    it "has a default task"

When I ask mocha to run the gulpfile, my output looks like this:

gulpfile
    - uses a javascript file that uses 'require' to load a utilty file
    - uses a utilty to read the task files from the tasks directory
    - prints the number of files loaded
    - uses 'gulp-load-plugins' to load plugins
    Tasks
      - has a 'links' task that creates links in the components directory so that Brackets can run on the gulp server
      - has a 'mocha:gulpfile' task that runs mocha on this file
      - continues running mocha tests even when a test fails
      - has a 'mocha' task that runs mocha on all specs in the test directory and sub-directories
      - has a less task that runs the less compiler
      - has a jade task that converts jade to HTML
      - has a coffee task that converts client side coffee code to js
      - has a lint task that runs jshint on all coffeescript
      - has an html task that runs useref on all html files
      - has a clean task that cleans the dist directory before a build
      - has a build task
      - has a connect task
      - has a watch task that rebuilds anything that changes
      - has a default task

  0 passing (6ms)
  18 pending

I have written no tests, so 0 are passing and 18 are pending.

Once I convert the first requirements to tests they look like this:

  it "uses a javascript file that uses 'require' to load a utilty file", ->
    require('fs').statSync('gulpfile.js')

  it "uses a utilty to read the task files from the tasks directory", ->
    exec('gulp utils:register')
      .then( (result)->outputShouldMatch( result, /\n\[gulp] registered\n/       ))

The second test has two parts that run asynchronously. A process needs to be spawned with exec, run to completion, and on completion the output needs to be matched for a string whose presence indicates that the job ran properly.

The exec function that is used is not the one that is part of the node library.  Rather it is a derivative function that uses Promises. Promises is an increasingly popular abstraction that makes writing asynchronous code as easy as writing synchronous. When a function like exec runs it returns promise object which has a then method. The argument to the then must be a function, which does not run until the promise is fulfilled, in this case when the exec'd process completes.

The output of my tests now becomes:

gulpfile
    ✓ uses a javascript file that uses 'require' to load a utility file 
    ✓ uses a utility to read the task files from the tasks directory (779ms)

    <other tests>

2 passing (787ms)
  16 pending

The test shows the time the second test took because it was significant.

More requirements

I’m continuing to convert more requirements to tests, and right now the output looks like this:

gulpfile
    ✓ uses a javascript file that uses 'require' to load a utilty file 
    ✓ uses a utilty to read the task files from the tasks directory (847ms)
    ✓ prints the number of files loaded (810ms)
    ✓ uses 'gulp-load-plugins' to load plugins 
    Tasks
      ✓ has a 'links' task that creates links in the components directory so that Brackets can run on the gulp server (822ms)
      ✓ has a 'mocha:gulpfile' task that runs mocha on this file 
      ✓ continues running mocha tests even when a test fails (1033ms)


      <tests I have not converted>

  7 passing (4s)
  11 pending
==========================