Using the CS3 testing library

The CS3 testing library consists of some simple procedures to help you create and manage tests for you projects. Your tests will consist of a set(s) of test cases, where each test case contains:

• a scheme expression -- this is a call to the procedure that you want to test, with arguments that will check a specific aspect of its functionality.
• an expected return value -- this is what the scheme expression should return if it is operating correctly.
• a name -- so you can identify the particular test case when it is listed in a report.

You can create sets of test cases and easily select individual sets of tests to run on your program. This approach is called unit testing (where a unit is a set of tests). Unit testing supports a modular approach to programming -- remember the notion of abstraction -- letting you write and run tests for individual modules.

Using the testing library

There are three functions you need to know:

(add-test-case test-name expected-return-value scheme-expression)


(run-test-cases optional-name-or-procedure)


(clear-test-cases)

add-test-case

add-test-case takes in a quoted name, the return value you expect expect to get when evaluating the testing expression, and the scheme expression itself. Each test case you add needs to have a unique name; add-test-case will warn you if the name is not unique.

The first argument must be quoted -- it is simply a name. The second argument may or may not be quoted: if you want to give a specific word or sentence as an expected value, you need to quote this. If you want evaluate some expression to give the expected value, then it shouldn't be quoted. The third argument should never be quoted in practice -- this is the scheme expression that you want evaluated. Remember, quote things that are names or specific words/sentences; don't quote things that need to be evaluated.

       STk> (add-test-case 'single-digit-test '(nine) (number-name 9))
       okay
       STk> (add-test-case 'show-hundreds '(one hundred) (number-name 100))
       STk> (add-test-case 'comparing-my-appearances (appearances 'a 'abracadabra)
                                                     (my-appearances 'a 'abracadabra))
       okay

The third test case above is, basically, ensuring that my-appearances works as does the built-in appearances. Note that the expected value does not have a quote in front of it because we want to test our code again what the built-in appearances returns.

run-test-cases

run-test-cases will run some or all the test cases you added using add-test-cases. If you call run-test-cases without any arguments, all test cases will be run.

STk> (run-test-cases)
(----- Running 3 tests... Failure output below. -----)
(----- Done. 0 failed test(s). 0 error(s). -----)
okay

You can, optionally give it a single argument, either a name (e.g., a word) or a function. If that argument evaluates to a function, then only the test cases for which the scheme expression calls that function (as the first element) will be run. Note: don't quote the function, so that run-test-cases knows you are referring to a function rather than a name.

STk> (run-test-cases number-name)
(----- Running 2 tests... Failure output below. -----)
(----- Done. 0 failed test(s). 0 error(s). -----)
okay

If that argument is a name -- that is, a quoted word -- then all the test cases that have a name that starts with the word will be run.

STk> (run-test-cases 'compar)
(----- Running 1 tests... Failure output below. -----)
(----- Done. 0 failed test(s). 0 error(s). -----)
okay

If your test cases contain cases that generate errors or return values other than the expected value, they will be listed in the report:

STk> (run-test-cases)
(----- Running 3 tests... Failure output below. -----)
nine
ERROR: on show-hundreds
(----- Done. 1 failed test(s). 1 error(s). -----)
okay

In the above report, the test case named nine didn't return its expected value, and the test case named show-hundreds generated an error when it was run (and, therefore didn't return any value!). The test for my-appearances worked correctly, and wasn't reported.

clear-test-cases

clear-test-cases erases all the test cases that you have currently added.

STk> (clear-test-cases)
okay

Setting up and running your tests

The best way to use this library is to create a separate testing file, named the same as the file you are trying to test except for an added -tests. For instance, if you are testing the code inside number-name.scm, create a file number-name-tests.scm to hold all of the test cases that you generate.

Put a comment at the top of your testing file describing it, and immediately after include the expression (clear-test-cases). This ensures that, every time the file is loaded, you start with an empty set of test cases.

Next, include groups of test cases for each procedure that you want to test. Put a comment at the top of the group stating what procedure it is for and anything special you need to remember. Below the comment, add your test cases with appropriate calls to add-test-case.

For example, a testing file might look like:

;;number-name-tests.scm

(clear-test-cases) ;;MAKE SURE THIS IS INCLUDED AT THE TOP OF YOUR FILE


;;number-name: testing one-digit-numbers
(add-test-case 'one '(one) (number-name 1))
(add-test-case 'five '(five) (number-name 5))
(add-test-case 'nine '(nine) (number-name 9))

;;number-name: testing two-digit-numbers
(add-test-case 'thirteen '(thirteen) (number-name 13))
(add-test-case 'twenties '(twenty one) (number-name 21))


Once you have written your test file, send all the definitions to STk by using the Send Buffer command in emacs. Finally, you can use (run-test-cases) from within STk to test your program. As you make changes to the code, use (run-test-cases), either with the optional argument or not, to check that you haven't broken anything that used to work! If you add more test cases, simply reload the Emacs buffer containing your test cases.