Configuration overview

This section is an overview of the primary ways to configure and control vvtest for execution on specialized platforms and to allow for project specific behaviors. More detail is provided in separate wiki sections.

The three primary categories of configuration and control are

1. Platform configuration: Resource management is challenging for testing MPI distributed, computationally intensive applications on diverse platforms, including batch managed systems. vvtest has simple probes for basic workstations, but not for MPI launch specifics, cluster hardware setup and counts, and batch system types. Some aspects of a platform configuration must be given to vvtest so it can manage the resources.
2. Utilities pathing: Projects typically use a common set of scripting utilities in their tests. This reuse is critical in projects that have many hundreds or even thousands of tests. Configuration paths can be provided to vvtest that contain scripts and Python modules for use by the test scripts.
3. Operations plugin A project may want to adjust testing behavior on an individual test basis, such as setting timeouts or checking for test validity and style. A python plugin capability is available in vvtest to affect such behavior. The plugin can also be used to execute a prologue operation before the tests start, and an epilogue operation after the last last test finishes.

All three of these are managed using the concept of a "configuration directory", which vvtest uses to load plugin files and set pathing for the tests. A configdir can be given to vvtest in a few ways:

• The --config command line option
• The VVTEST_CONFIGDIR environment variable
• The existence of a directory called config in the same directory as vvtest itself (or a soft link to vvtest)
• The existence of a vvtest_bootstrap.py file in the same directory as vvtest (or soft link) that sets the VVTEST_CONFIGDIR environment variable

Platform configuration

A file in the configdir called idplatform.py is imported by vvtest and certain functions, if they exist, are called to get the platform name, the compiler name, and to set platform attributes (such as the number of cores per compute node). However, the environment variables VVTEST_PLATFORM and VVTEST_PLATFORM_SPECS can be used instead of an idplatform.py file.

Since vvtest was developed at Sandia National Laboratories, it has builtin knowledge of a number of platforms at SNL and also the established ASC ATS platforms. If neither the platform plugin nor the builtin recognizes the current platform, then the value of uname is used (such as Linux or Darwin).

While the platform name can be used for selecting tests to run (test filtering), the compiler name is only used for informational purposes, and has no default.

For basic workstations, no platform attributes need to be set (the number of cores is a simple probe). But, for example, a cluster platform consisting of compute nodes using a batch system for job allocation, requires a few attributes to be set. The most common are "batchsys", "cores_per_node", and "devices_per_node".

Utilities pathing

TODO

Operations plugin

TODO

Using multiple configdir paths

TODO