Writing test files

The test file defines one or more test instances to be run by vvtest. It is both a set of specifications as well as the script that executes the test.

Test specifications are also called "header directives" because they must appear at the top of the test file (the header), and they direct the test harness for when and how to run the test. Consider a file called atest.vvt having this content:

# these are the header directives; they all start with #VVT:

#VVT: parameterize : np = 1 4
#VVT: keywords : unit fracture 2D
#VVT: link : input.yml
#VVT: enable (platform="not ATS*")

# now comes the test execution scripting

import subprocess
import vvtest_util as vvt

if int(vvt.np) == 1:
    subprocess.check_call( 'myapp_serial input.yml', shell=True )
else:
    subprocess.check_call( 'mpiexec -n '+vvt.np+' myapp input.yml', shell=True )

This test file would expand into two test instances, one with the parameter np=1 and one with np=4. The test scripting uses the parameter values to adjust what it actually executes (in this case, it runs a serial version of the application or an MPI parallel version).

The keywords are arbitrary and allow the test to be selected using keyword filtering (using the -k command line option). The "enable" directive tells vvtest to only run the test on platforms whose name does not start with "ATS".

Every test will have the file vvtest_util.py written to the execution directory, and is used to access the test parameters and other test-specific and runtime-specific information.

Thus, writing a test consists of deciding what information to provide to vvtest prior to execution via the header directives, and writing the scripting to execute the test (most likely using the vvtest_util.py for information).

Test directory structure

Test files can be placed anywhere; it is up to the project to design a good layout. Since vvtest recursively searches for tests in subdirectories, they are typically organized by a directory hierarchy.

• Multiple test files and input files can live in the same directory
• The source directory structure is directly mimicked in the test execution area