Skip to content

options

Richard R. Drake edited this page Jun 8, 2022 · 1 revision

Using arbitrary option strings

The -o command line option can be used to pass arbitrary option strings into each test. A test can use the options for any number of reasons, such as using -o debug to select and run with a debug executable.

Also, tests can be enabled or disabled based on option values, as described below.

The option strings are used to name the TestResults runtime directory, which allows a separation of multiple vvtest invocations with different options. For example, consider this test file named "opts.vvt":

import vvtest_util as vvt
print ( 'the options are ' + str(vvt.OPTIONS) )

Running this test without -o options results in

s974534$ vvtest -v
...
==================================================
pass        0:01 01/01 16:59:06 TestResults.iDarwin/opts
==================================================
Summary:
    completed: 1
          1 pass
    total: 1

Finish date: Fri Jan  1 16:59:07 2021 (elapsed time 1s)
Test directory: TestResults.iDarwin

then do

s974534$ tail -n 2 TestResults.iDarwin/opts/execute.log

the options are []

Note the empty option list in the test output. Now run again but with an option:

s974534$ vvtest -v -o debug
...
==================================================
pass        0:01 01/01 17:03:50 TestResults.iDarwin.ON=debug/opts
==================================================
Summary:
    completed: 1
          1 pass
    total: 1

Finish date: Fri Jan  1 17:03:51 2021 (elapsed time 1s)
Test directory: TestResults.iDarwin.ON=debug

Note that the execution directory contains the -o value. The test output is:

s974534$ tail -n 2 TestResults.iDarwin.ON\=debug/opts/execute.log

the options are ['debug']

Multiple -o options are allowed. For example,

s968057% vvtest -v -o gpu -o debug
...
Finish date: Fri Jan  1 17:08:27 2021 (elapsed time 1s)
Test directory: TestResults.iDarwin.ON=debug_gpu

For the test results name, the options are sorted then concatenated with underscores.

Using options in test directives

The options given on the command line are used to process "options=" attributes in test directives. This allows enabling/disabling a test, determining test parameterizations, and more.

For example, consider the test file "opts.vvt":

#VVT: enable (options="not debug")
pass

When this is run without a -o option, the test will be run:

s974534$ vvtest -v
...
==================================================
pass        0:01 01/01 19:41:55 TestResults.iDarwin/opts
==================================================
Summary:
    completed: 1
          1 pass
    total: 1

Finish date: Fri Jan  1 19:41:56 2021 (elapsed time 1s)
Test directory: TestResults.iDarwin

But when -o debug is given, the test is not run:

s974534$ vvtest -v -o debug
Test list:
    completed: 0
    skipped: 1
          1 due to "excluded by option expression"
    total: 1

Finish date: Fri Jan  1 19:44:46 2021 (elapsed time 1s)
Test directory: TestResults.iDarwin.ON=debug

Consider an example using option values to control test parameterizations:

#VVT: parameterize (options=debug)     : np = 1
#VVT: parameterize (options=not debug) : np = 4
import vvtest_util as vvt
print ( 'np = ' + vvt.np )

Run this without a -o option and look at the output:

s974534$ vvtest -v
...
Finish date: Fri Jan  1 17:20:09 2021 (elapsed time 1s)
Test directory: TestResults.iDarwin

s974534$ tail -n 2 TestResults.iDarwin/opts.np\=4/execute.log

np = 4

Then run it with -o debug

...
Finish date: Fri Jan  1 17:23:52 2021 (elapsed time 1s)
Test directory: TestResults.iDarwin.ON=debug

s974534$ tail -n 2 TestResults.iDarwin.ON\=debug/opts.np\=1/execute.log

np = 1

Note how the "np" parameter took on different values depending on the -o option given.

In general, option expressions in a test file are evaluated against the list of option values given on the command line. Most directives accept option expressions, including these:

directive example notes
#VVT: enable (options="debug") run the test if -o debug is given
#VVT: parameterize (options="debug") : np = 1 parameterize only if -o debug is given
#VVT: link (options="intel or debug") : file.txt link file.txt if intel or debug options are given
#VVT: copy (options="intel-*") : file.txt copy file.txt if any -o options start with "intel-"
#VVT: analyze (options="not intel") : --analyze don't analyze for intel option
#VVT: timeout (options="debug") : 3600 set timeout to 3600 seconds for debug
#VVT: baseline (options="not debug") : --baseline don't rebaseline for debug
#VVT: depends on (options="CUDA") : cuda-* add a dependency for CUDA option
#VVT: preload (options="debug") : alabel apply plugin preload label for debug

Note that the value can be any expression, including expressions with shell style wildcard pattern matching.

Clone this wiki locally