From a testing viewpoint, hardware designs in VHDL are similar to conventional software programs. When testing a VHDL design, the focus is on errors in the VHDL, not in the fabrication process to come. When testing software, the focus is on errors in the code, not in the compiler. In both cases the main source of problems is coding errors. This similarity suggests that techniques that have been successfully used to test software may also be effective when applied to testing hardware designs in VHDL. Because VHDL designs are for ICs or cells, it makes sense to focus on techniques used for testing software components, as opposed to large software systems.
For software components, thorough testing requires a large number of test cases; thousands of test cases are not uncommon. These test cases must be run many times: after each version due to bug fixes or enhancements, and after each change in the environment including the compiler and support libraries. With many tests that must each be run many times, it is critical that the cost of each test run be low. Ideally there would be no cost except for machine time. Only highly automated testing can achieve these low costs. In particular, the checking of the correctness of the outputs must be automated.
Many programmers test their code interactively, using a debugger or a driver program that prompts them for inputs and displays outputs. The test inputs are entered with the keyboard and mouse and the outputs are checked visually. This approach is good for debugging, where the next test input is not known until the results of the current input have been seen. However, interactive testing is expensive and unreliable when many test cases must be repeated many times.
With algorithmic testing, the test inputs are generated by a program and the outputs are checked by a program. Algorithmic testing provides cheap and reliable regression testing. The cost of developing the test programs must be considered, but once developed, the cost of repeating the tests is negligible.
Modern capture/playback tools support a hybrid testing approach. Inputs are entered by hand but ``captured'' for later playback. In some cases, outputs may also be recorded. This approach does allow tests to be repeated cheaply and reliably. However, capture/playback tests are expensive to develop if many tests are required. Further, the scripts are very sensitive to change and are thus expensive to maintain [7].
PGMGEN is a test driver generator originally developed to test C modules [2]. Other versions have been developed and used to test Fortran, assembler, and C++ [1,6]. ClassBench is a framework specifically designed for testing C++ class libraries [3].
VHDLGEN was adapted from the PGMGEN tool to test VHDL designs. With the current efforts to add OO features to VHDL [8], we believe that the ClassBench approach may soon be applicable to VHDL.