Test scripts

One of the most valuable uses we have found for Tcl Blend and Java is for writing test suites for our Java development. Because running tests is somewhat ad-hoc, Tcl is a perfect match for both initial unit testing by hand, for writing test scripts, and for gluing together suites of test scripts for regression testing and code coverage analysis. He we'll describe the essence of building test suites for Java using Tcl. (This approach is based on the approach used in the test suites supplied with the Tcl source distribution.)

The Tcl code we will use is in the file javatute/test.tcl. The main part of this is the test procedure:

set TEST_PASSED 0
set TEST_FAILED 0
proc test {name description script answer} {
    global TEST_PASSED TEST_FAILED

    # Evaluate the script
    puts "Test $name"
    set code [catch {uplevel $script} result]

    # Print info if it failed
    if {$code != 0} {
	puts "Description:"
	puts "   $description"
	puts "Stack trace:"
	jdkStackTrace
	incr TEST_FAILED
    } elseif {[string compare $result $answer] != 0} then { 
	puts "Description:\n    $description"
	puts "Expected result:\n   $answer"
	puts "Received result:\n   $result"
	incr TEST_FAILED
    } else {
	incr TEST_PASSED
    }
    update
}

After evaluating the script argument, this procedure prints simple error diagnostics if the test threw an exception or produced the wrong result.

To illustrate the use of the test procedure, here is a simple Java class, tutorial.tcltk98.BadPath, written (supposedly) for the purpose of constructing a filename path incrementally. It has two methods: append, which adds another element to the filename and getPath, which returns the path:

package tutorial.tcltk98;
public class BadPath {
    StringBuffer path;
    public void append (String element) {
        if (path == null) {
            path = new StringBuffer();
        } else {
            path.append("/");
        }
        path.append(element);
    }
    public String getPath () {
        return path.toString();
    }
}

We were careful to avoid putting a slash at the start of the path, since we're assuming this object stored relative paths. If we compile this file and try it, it appears to work:

  set p [java::new tutorial.tcltk98.BadPath]
  $p append foo
  $p append bar
  $p getPath

which prints "foo/bar". To test it a little more thoroughly, we write a test suite, javatute/testBadPath.tcl, that calls the test procedure:

test path-1 "Construct a path" {
  set p [java::new tutorial.tcltk98.BadPath]
  $p getPath
} ""

test path-2 "Single-level path" {
  set p [java::new tutorial.tcltk98.BadPath]
  $p append foo
  $p getPath
} foo

test path-3 "Two-level path" {
  set p [java::new tutorial.tcltk98.BadPath]
  $p append foo
  $p append bar
  $p getPath
} foo/bar

test path-4 "Ignore blank elements" {
    set p [java::new tutorial.tcltk98.BadPath]
    $p append foo
    $p append ""
  $p getPath
} foo

testDone

When we source this test suite, however, two of the tests fail: path-1 fails because we get a java.lang.NullPointerException, and path-4 fails because we forgot to account for blank elements in the argument to append.

The corrected code is in the class tutorial.tcltk98.GoodPath. In this version, we have also taken the trouble to properly catch erroneous input in append and throw a new exception that we defined, tutorial.tcltk98.PathException.

package tutorial.tcltk98;
public class GoodPath {
    StringBuffer path;
    public void append (String element) throws PathException {
        if (element.indexOf('/') >= 0 ) {
            throw new PathException("Malformed path");
        }
        if (!element.equals("")) {
            if (path == null) {
                path = new StringBuffer();
            } else {
                path.append("/");
            }
            path.append(element);
        }
    }
    public String getPath () {
        if (path == null) {
            return "./";
        } else {
            return path.toString();
        }
    }
}

In addition to testing for valid return results, we want to test that erroneous input conditions throw the "right" exception. To support this, test.tcl also contains a procedure called testException, that expects a Java exception to be thrown. Here is its declaration:

  proc testException {name description script exception {message {}}}

As weell as the test name, description, and script, this procedure is passed the name of a Java exception class, and, optionally, the string contained in the thrown exception. The test fails if no exception is thrown or if the wrong one is thrown.

The new test suite is in javatute/testGoodPath.tcl. It contains six tests, the four from the previous version (modified slightly), and an additional two that check that test for exceptions:

testException path-5 "Throw exception on null" {
    set p [java::new tutorial.tcltk98.GoodPath]
    $p append [java::null]
} java.lang.NullPointerException

testException path-6 "Throw exception if element has slash" {
    set p [java::new tutorial.tcltk98.GoodPath]
    $p append "foo/bar"
} tutorial.tcltk98.PathException "Malformed path"

If you source this file, all the tests will pass.

We have shown only the process of running a single test suite. Because we are using Tcl, however, it is easy to construct test procedures for multiple tests, and to produce ad-hoc test runs as needed. For example, to run all test suites in a directory, we might run

foreach f [glob test*.tcl] {
    source $f
}

This kind of thing -- gluing together small scripts into larger ones and producing ad-hoc test runs as needed -- is much, much easier and faster in Tcl than it would be to write and compile new Java code each time.