JavascriptXUnitKata - XpUtah Wiki

Introduction

We want to create an xUnit testing framework for javascript.

The framework will have tests, results, fixtures and suites.

The following is an example of using our not yet built testing framework.

var arrayTests = createFixture('ArrayTests', {
    testFirst : function() {
        this.assertEquals('foo', ['foo', 'bar'].first());
        this.assertEquals(undefined, [].first());
    },

    testLast : function() {
        this.assertEquals('bar', ['foo', 'bar'].last());
        this.assertEquals(undefined, [].last());
    }
});

var suite = createSuite();
suite.add(arrayTests.createTest('testFirst'));
suite.add(arrayTests.createTest('testLast'));

var result = createResult();
suite.run(result);
document.writeln(result.summary());

Initial Setup

We create four files:

runtest.html
object-maker.js
jsunit.js
jsunit-test.js

<!-- runtest.html -->
<html><body><pre>
<script src="object-maker.js"></script>
<script src="jsunit.js"></script>
<script src="jsunit-test.js"></script>
</pre></body></html>

// object-maker.js

// For explanation of Object.create and new_constructor (and good javascript practices), see the following:
//   Douglas Crockford's book: "JavaScript: The Good Parts" 
//   http://www.slideshare.net/douglascrockford/crockford-on-javascript-act-iii-function-the-ultimate
//   http://www.crockford.com/ 

if (typeof Object.create !== 'function') {
    Object.create = function (o) {
        function F() {}
        F.prototype = o;
        return new F();
    };
}

function new_constructor(extend, initializer, methods) {
    var prototype = Object.create(extend && extend.prototype);

    if (methods) {
        for (key in methods) {
            prototype[key] = methods[key];
        }
    }

    var func = function () {
        var that = Object.create(prototype);
        if (typeof initializer === 'function') {
            initializer.apply(that, arguments);
        }
        return that;
    };

    func.prototype = prototype;
    prototype.constructor = func;
    return func;
}

// jsunit.js

// jsunit-test.js

We only need jsunit.js and jsunit-test.js active in our editor.

Invoke test method

TODO List

Invoke test method
Run multiple tests
Report collected result

Test 0: Bootstraping

// jsunit-test.js
try {

    document.writeln(test.wasRun);
    test.testMethod();
    document.writeln(test.wasRun);

} catch(e) {
    document.writeln(e.message);
    document.writeln(e.stack);
}

I want to see the errors (browsers fail silently), so I am surrounding everything with try/catch.

Obviously this won't even run.

ReferenceError: test is not defined

I make it run in the test file.

// jsunit-test.js
try {

    var test = {
        wasRun : false,
        testMethod : function() {
        }
    };
...

Now I see it fail.

false
false

Now I make it pass.

// jsunit-test.js
...
      testMethod : function() {
        this.wasRun = true;
      }
...

Yeah, green bar!

false
true

Test 1: Run Runs (Test 0 -> Test 1)

// jsunit-test.js
...
    document.writeln(test.wasRun);
    test.run();
    document.writeln(test.wasRun);
...

We can make it pass by adding a run method to the object and calling our testMethod. This is an example of FakeItTillYouMakeIt.

// jsunit-test.js
...
    var test = {
        run : function() {
            this.testMethod();
        },
...

Of course it passes.

false
true

We want our test object to have the name of the test method and run will execute the named method.

// jsunit-test.js
...
    var test = {
        name : 'testMethod',
        run : function() {
            this[this.name]();
        }
        ...
    }
...

We are still passing. Yeah!

false
true

Our test object is doing two things:

It is defining the tests
It is running the tests

We extract the running of the tests into the beginings of a test framework. We leave defining the test in jsunit-test.js

We replace var test = ... with a call to our new test framework.

// jsunit-test.js
...
    var test = createTest('testMethod', {
        wasRun : false,
        testMethod : function() {
            this.wasRun = true;
        }
     });
...

// jsunit.js

// Make a new 'function' called createTest
// that each time we call it, will make a single test
createTest = new_constructor(
    // extends
    null,

    // initializer
    function(name, methods) {
        this.name = name;
        for (var key in methods) {
            this[key] = methods[key];
        }
    },

    // methods
    {
        run : function() {
            this[this.name]();
        }
    }
);

Our test still passes, excellent.

false
true

We're finally ready to create a test which tests that our test framework can run a test. (Got it?)

(Looking ahead a little, we make sure all our test methods start with the word test. Later, we will enhance our framework to run *all* methods that begin with test. We aren't there yet.)

// jsunit-test.js
try {

    var fmwkRunsATest = createTest('testFrameworkRunsATest', {

        aTest : createTest('testMethod', {
            wasRun : false,
            testMethod : function() {
                this.wasRun = true;
            }
         }),

        testFrameworkRunsATest : function() {
            if (this.aTest.wasRun) {
                throw new Error('aTest shouldn\'t have run');
            }
            this.aTest.run();
            if (! this.aTest.wasRun) {
                throw new Error('aTest should have run');
            }
        }
    });

    fmwkRunsATest.run();

} catch(e) {
...

And the test passes (showing nothing)

Because I am a nervous Nellie, I comment out this.wasRun = true; just to see it fail.

// jsunit-test.js
...
//                this.wasRun = true;
...

aTest should have run
Error: aTest should have run
    at [object Object].testRunning (file:///E:/users/xunit/javascript/jsunit-test.js:20:23)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:50:27)
    at file:///E:/users/xunit/javascript/jsunit-test.js:25:10

// jsunit-test.js
...
                this.wasRun = true;
...

Now I feel much better with seeing nothing for a passing test.

It is time to check-in and cross an item off our TODO list.

TODO List

Invoke test method
Run multiple tests
Report collected result

Report collected result

Test 2: Collect and Organize Result

It bugs me that tests succeed silently. I want output! Let's work on Report collected result.

Naturally, we will first write a new test (keeping in the other test in place).

// jsunit-test.js
...
    var fmwkReportsResults = createTest('testFrameworkReportsResults', {

        aTest : createTest('testMethod', {
            wasRun : false,
            testMethod : function() {
                this.wasRun = true;
            }
         }),

        testFrameworkReportsResults : function() {
            var result = this.aTest.run();
            var expected = "1 run, 0 failed";
            var actual = result.summary();

            if (expected !== actual) {
                throw new Error(expected + ' !== ' + actual);
            }
        }
    });

    fmwkRunsATest.run();
    fmwkReportsResults.run();
...

(Hopefully) The new test fails.

Cannot call method 'summary' of undefined
TypeError: Cannot call method 'summary' of undefined
    at [object Object].testResult (file:///E:/users/xunit/javascript/jsunit-test.js:37:33)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:50:27)
    at file:///E:/users/xunit/javascript/jsunit-test.js:46:11

We make it pass with the time honored TDD technique "fake it till you make it."

// jsunit.js
...
createTest = new_constructor(
    ...
    // methods
    {
        run : function() {
            var result = {
                summary : function() {
                    return "1 run, 0 failed";
                }
            };

            this[this.name]();
            return result;
        }
    }
...

Yep passing!

Taking small steps makes elephant eating pleasant. We replace the object literal, which faked our result summary, with a function.

// jsunit.js
...
        run : function() {
            var result = createResult();
            ...
        }
...

createResult = new_constructor(
    // extends
    null,

    // initializer,
    function() {
    },

    // methods
    {
        summary : function() {
            return "1 run, 0 failed";
        }
    }
);

Still passing.

Another baby step is adding runCount.

// jsunit.js
...
createResult = new_constructor(
    ...
    // initializer,
    function() {
        this.runCount = 1;
    },

    // methods
    {
        summary : function() {
            return this.runCount + " run, 0 failed";
        }
    }
);

Still passing

I can no longer fake it.

// jsunit.js
...
        run : function() {
            var result = createResult();
            result.testStarted();
...
    // initializer,
    function() {
        this.runCount = 0;
    },

    // methods
    {
        summary : function() {
            return this.runCount + " run, 0 failed";
        },

        testStarted : function() {
            this.runCount += 1;
        }
...

It would be silly not to use the wonderful code we just wrote.

// jsunit-test.js
...
    var output = '';
    output += fmwkRunsATest.run().summary();
    output += '\n';
    output += fmwkReportsResults.run().summary();
    output += '\n';

    document.writeln(output);
...

Much nicer output.

1 run, 0 failed
1 run, 0 failed

Tests are passing, yeah. I still have a problem with the hard coded zero. It is not obvious to me how to move forward, so I will triangulate (fancy word for writing another test to force me to remove hard coded solution).

Additionally, the ugly duplication between the tests is bothering me.

But first, I add my concerns to the TODO list, thus allowing me to check in.

TODO List

Invoke test method
Run multiple tests
Report collected result
Report failed result
Introduce fixture

Introduce Fixture

Refactor/Redesign interface

Duplication and ugly code are the enemy of all good coders everywhere.

We have shared data between the two tests (obvious by the duplication). Lets push the shared data and the tests up into a Fixture.

// jsunit-test.js

try {

    var fixture = createFixture('Framework Tests', {
        aFixture : createFixture('A Fixture', {
            wasRun : false,
            testMethod : function() {
                this.wasRun = true;
            }
        }),

        testFrameworkRunsATest : function() {
            var aTest = this.aFixture.createTest('testMethod');
            if (aTest.wasRun) {
                throw new Error('aTest shouldn\'t have run');
            }
            aTest.run();
            if (! aTest.wasRun) {
                throw new Error('aTest should have run');
            }
        },

        testFrameworkReportsResults : function() {
            var aTest = this.aFixture.createTest('testMethod');
            var result = aTest.run();
            var expected = "1 run, 0 failed";
            var actual = result.summary();

            if (expected !== actual) {
                throw new Error(expected + ' !== ' + actual);
            }
        }
    });

    var fmwkRunsATest = fixture.createTest('testFrameworkRunsATest');
    var fmwkReportsResults = fixture.createTest('testFrameworkReportsResults');

    var output = '';
    output += fmwkRunsATest.run().summary();
    output += '\n';
    output += fmwkReportsResults.run().summary();
    output += '\n';

    document.writeln(output);
...

Test fails with a nice error message depending on your browser.

createFixture is not defined
@file:///Users/kayjohansen/projects/javascript/xunit/jsunit-test.js:4

We implement createFixture.

// jsunit.js
...
// replace createTest (and this comment) with the following code:
createFixture = new_constructor(
    // extends
    null,

    // initializer
    function(fixtureName, methods) {
        this.fixtureName = fixtureName;
        for (var key in methods) {
            this[key] = methods[key];
        }
    },

    // methods
    {
        run : function() {
            var result = createResult();
            result.testStarted();
            this[this.name]();
            return result;
        },

        createTest : function(testName) {
            var that = Object.create(this);
            that.name = testName;
            return that;
        }
    }
);
...

Chocolate Cake and Beer for everyone whose tests pass.

1 run, 0 failed
1 run, 0 failed

If we're still sober, we'd better remember to update our TODO list and check in.

TODO List

Invoke test method
Run multiple tests
Report collected result
Report failed result
Introduce fixture

Report failed result

Test 3: Failed Result

Moving on, we know we want to fix the hard-coded failure count. Remember we are triangulating - adding another failing test - to help direct our implementation.

// jsunit-test.js
...
    var fixture = createFixture('Framework Tests', {
        ...
        testFrameworkReportsErrorCount : function() {
            var aTest = this.aFixture.createTest('testBrokenMethod');
            var result = aTest.run();
            var expected = "1 run, 1 failed";
            var actual = result.summary();

            if (expected !== actual) {
                throw new Error(expected + ' !== ' + testFrameworkReportsErrorCount);
            }
        }
...
    var fmwkReportsFailureCounts = fixture.createTest('testFrameworkReportsErrorCount');
...
    output += fmwkReportsFailureCounts.run().summary();
    output += '\n';
...

When I run the test, I get the following error in Chrome:

undefined is not a function
TypeError: undefined is not a function
    at CALL_NON_FUNCTION (native)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:17:19)
    at [object Object].testFailedResults (file:///E:/users/xunit/javascript/jsunit-test.js:44:31)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:17:19)
    at file:///E:/users/xunit/javascript/jsunit-test.js:59:50

This is telling me I need to add testBrokenMethod to aFixture (not to fixture).

// jsunit-test.js
...
        aFixture : createFixture('A Fixture', {
            ...
            testBrokenMethod : function() {
                throw new Error('Broken test');
            }
...

We see the test fail.

Broken test
Error: Broken test
    at [object Object].testBrokenMethod (file:///E:/users/xunit/javascript/jsunit-test.js:13:23)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:17:19)
    at [object Object].testFailedResults (file:///E:/users/xunit/javascript/jsunit-test.js:42:31)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:17:19)
    at file:///E:/users/xunit/javascript/jsunit-test.js:57:50

In order to get this test running I have to do two things. I don't want to do that. So I comment out this test.

// jsunit-test.js
...
    //output += fmwkReportsFailureCounts.run().summary();
    //output += '\n';
...

Test 4: Create '''test failed result''' on a result fixture.

Lets create a new fixture to test result in isolation. That will get me back to doing one test one solution.

// jsunit-test.js
...
    var resultFixture = createFixture('Result Tests', {

       testResultsIncludeFailedCount : function() {
            var result = createResult();
            result.testStarted();
            result.testFailed();
            var expected = "1 run, 1 failed"
            var actual = result.summary();

            if (expected !== actual) {
                throw new Error(expected + ' !== ' + actual);
            }
        }
    });
...
    var resultFailedResult = resultFixture.createTest('testResultsIncludeFailedCount');
    ...
    //output += fmwkReportsFailureCounts.run().summary();
    //output += '\n';
    output += resultFailedResult.run().summary();
    output += '\n';
...

The tests fail with an error:

Object # has no method 'testFailed'
TypeError: Object # has no method 'testFailed'
    at [object Object].testFailedResults (file:///JavaScript/jsunit-test.js:57:21)
    at [object Object].run (file:///JavaScript/jsunit.js:53:27)
    at file:///JavaScript/jsunit-test.js:79:31

After seeing the test fail, I realize this is an easy fix (obvious implementation);

// jsunit.js
...
createResult = new_constructor(
    ...
    // initializer
    function() {
        ...
        this.failedCount = 0;
    },

    // methods
    {
        summary : function() {
            return this.runCount + " run, " + this.failedCount + " failed";
        },
        ...
        testFailed : function() {
            this.failedCount += 1;
        }
...

1 run, 0 failed
1 run, 0 failed
1 run, 0 failed

Now I am bothered by my multiple line assert. I won't think about it, I will just write it on my TODO list.

TODO List

Invoke test method
Run multiple tests
Report collected result
Report failed result
Introduce fixture
Introduce asserts

Test 3: Failed Results on FixtureTests (revisited)

Before I cross out my "report failed tests" item, I better put my test back.

// jsunit-test.js
...
    output += fmwkReportsFailureCounts.run().summary();
    output += '\n';
...

Of course it fails.

Broken test
Error: Broken test
    at [object Object].testBrokenMethod (file:///E:/users/xunit/javascript/jsunit-test.js:13:23)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:53:27)
    at [object Object].testFailedResults (file:///E:/users/xunit/javascript/jsunit-test.js:41:31)
    at [object Object].run (file:///E:/users/xunit/javascript/jsunit.js:53:27)
    at file:///E:/users/xunit/javascript/jsunit-test.js:71:54

We are going to need a try/catch.

// jsunit.js
...
        run : function() {
            ....
            try {
                this[this.name]();
            } catch(e) {
                result.testFailed();
            }
            return result;
...

And of course we run our tests and see them pass.

1 run, 0 failed
1 run, 0 failed
1 run, 0 failed
1 run, 0 failed

Time to check in and cross off another item on our TODO list. Progress feels great. On the other hand, losing the failure details might cause me pain. I'll put it on the list.

TODO List

Invoke test method
Run multiple tests
Report collected results
Report failed tests
Introduce Asserts
Report failed details

Run multiple tests

Test 5: Collect to Suite

Lets tackle running multiple tests. That should get rid of some of that ugly duplication. We add a new test called testSuiteRunsAllTests.

// jsunit-test.js
...
    var fixture = createFixture('Framework Tests', {
        ...
        testSuiteRunsAllTests : function() {
            var suite = createSuite();
            var result = createResult();
            suite.add(this.aFixture.createTest('testMethod'));
            suite.add(this.aFixture.createTest('testBrokenMethod'));

            suite.run(result);

            var expected = "2 run, 1 failed";
            var actual = result.summary();

            if (expected !== actual) {
                throw new Error(expected + ' !== ' + actual);
            }
        }
...
    var fmwkSuite = fixture.createTest('testSuiteRunsAllTests');
    ...
    output += fmwkSuite.run().summary();
    output += '\n';
...

Seeing it fail is is something we do.

1 run, 0 failed
1 run, 0 failed
1 run, 0 failed
1 run, 0 failed
1 run, 1 failed

Now we make it pass.

// jsunit.js
...
createSuite = new_constructor(
    // extends
    null,

    // initializer
    function() {
        this.tests = [];
    },

    // methods
    {
        add : function(test) {
            this.tests.push(test);
        },

        run : function(result) {
            for (var i=0; i < this.tests.length; i+=1) {
                this.tests[i].run(result);
            }
        }
    }
);

Oops, I forgot to change the createFixture#run method. I don't want to break my other tests so default argument time.

// jsunit.js
...
createFixture = new_constructor(
    ...
        run : function(result) {
            // Notice we removed var result = createResult();
            if (! result) {
                result = createResult();
            }
...

Now that everything is passing, he said idly rerunning the tests,

1 run, 0 failed
1 run, 0 failed
1 run, 0 failed
1 run, 0 failed
1 run, 0 failed

we want to use our newly created test suite.

// jsunit-test.js
...
    var suite = createSuite();

    suite.add(fixture.createTest('testFrameworkRunsATest'));
    suite.add(fixture.createTest('testFrameworkReportsResults'));
    suite.add(fixture.createTest('testFrameworkReportsErrorCount'));
    suite.add(resultFixture.createTest('testResultsIncludeFailedCount'));
    suite.add(fixture.createTest('testSuiteRunsAllTests'));

    var result = createResult();
    suite.run(result);

    document.writeln(result.summary());
...

Our passing tests look different. Cool, less noise.

5 run, 0 failed

Before I check in I want to remove run's default parameter.

// jsunit-test.js

...
        testFrameworkRunsATest : function() {
            ...
            aTest.run(createResult());
            ...
         },

        testFrameworkReportsResults : function() {
            ...
            var result = createResult();
            aTest.run(result);
            ...
        },

        testFrameworkReportsErrorCount : function() {
            ...
            var result = createResult();
            aTest.run(result);
            ...
        },
...

With the tests still running, we can remove the default parameter.

// jsunit.js
...
        run : function(result) {
            result.testStarted(); // notice result = createResult(); has been deleted
            try {
                this[this.name]();
            } catch(e) {
                result.testFailed();
            }
            // notice we no longer need to return result;
        },
...

After seeing the tests pass, I am ready to check in. With all that mucking around, I noticed we are polluting the global space. I will add it to my TODO list and check in.

TODO List

Invoke test method
Run multiple tests
Report collected result
Report failed tests
Introduce asserts
Report failed details
Remove globals

If you are here, you can count yourself as a top notch programmer. As you can see, software is never done. You can continue until you have a testing framework you like or you can use one of these: JavaScriptUnitTesting

I like http://code.google.com/p/js-test-driver/ and http://github.com/tobie/Evidence/

Thank you for playing along.