Validation and Verification Testsuite

The Public repository for the OpenACC Validation & Verification Testsuite.
Explore the docs »

View Results · Report Bug · Request Feature

Table of Contents

1. About The Project
2. Getting Started
   • Prerequisites
   • Installation
3. Roadmap
4. Contributing
5. License
6. Contact
7. Members
8. Acknowledgments
9. Publications

About The Project

This repository is updated with OpenACC test codes validating and verifying implementations of features and its conformance to the OpenACC specification. We are continuously adding and editing tests to conform to the latest version of the OpenACC Specification.

What we do:

• Revealing ambiguities in the OpenACC Specification
• Determining missing implementation of a feature
• Highlighting unmentioned restriction of a feature
• Evaluating implementations for multiple target platforms
• Identifying and reporting compiler bugs

Consult our website for more details on results and our project OpenACC V&V Website.

(back to top)

Getting Started

Prerequisites

Need to have the following installed on your system:

• Python

• OpenACC Compiler

Installation

In order to run the suite, we have provided a Python script, infrastructure.py that can be run. It is recommended to use Python 3.3 or later.

1. Clone the repo

   git clone https://github.com/OpenACCUserGroup/OpenACCV-V.git

2. Change the init_config.txt file to match your configuration

3. Run the script

   python3 infrastructure.py -c=<config_input_file> -o=<output_file>

   or

   python3 infrastructure.py -c=<configuration_file[,configuration_file2]> -o=<output_file> -in=<input_file[,input_file2]>  verbose -system=<system_name> -env=<environment_output>

(back to top)

Directive Based V&V Results Generator

Once you have processed your results and have your JSON file, parse through the results using the Directive Based V&V Results Generator: https://crpl.cis.udel.edu/oaccvv/generator/

1. Input the JSON file you received from running the testsuite into the generator inside of "Upload Results".

2. View a summary of your results, toggle between runtime/compiler to view each output.

3. Click on the "View Details" button inside the "Upload Results" section for downloadable Excel reports. Which will also include All Tests, Compiler Failures, and Runtime Failure drop downs that include all details about the individual tests.

4. Compare two JSON files by uploading them into the "Compare Two Results" section to generate a detailed graph. Toggle between runtime/compiler to view each output. You can hover over each bar on the graph to see more details.

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

• Please send results from a run or code for particular OpenACC features

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with tags. Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Branch (git checkout -b feature/missing)
3. Commit your Changes (git commit -m 'Add feature')
4. Push to the Branch (git push origin feature/missing)
5. Open a Pull Request

(back to top)

License

Distributed under the BSD 3-Clause License. See LICENSE.txt for more information.

(back to top)

Contact

Aaron Jarmusch - [email protected]

Sunita Chandrasekaran - [email protected]

CRPL - https://crpl.cis.udel.edu/

Website Link: https://crpl.cis.udel.edu/oaccvv/results/

Results Generator: https://crpl.cis.udel.edu/oaccvv/generator/

(back to top)

Members

Current Student Authors:

• Nathan Graddon
• Aaron Jarmusch

Contributors:

• Joel Denny
• Tobias Burnus
• Jeff Larkin
• Mathew Colgrove
• Christian Munley

Past Authors:

• Kyle Friedline
• Aaron Liu
• Vaidhyanathan Ravichandran
• Daniel Horta
• Will Gunter
• Olive Odida

(back to top)

Acknowledgments

Special thanks to the following organizations for their support

• OpenACC Organization

(back to top)

Publications

All publications relating to OpenACC V&V Testsuite

• Jarmusch, A. M., Liu, A., Munley, C., Horta, D., Ravichandran, V., Denny, J., & Chandrasekaran, S. (2022). Analysis of Validating and Verifying OpenACC Compilers 3.0 and Above. Presentation given at the SC22 Conference. SC22

• Jarmusch, A., Baker, N. & Chandrasekaran, Su. (2021). Using IU Jetstream for OpenMP offloading and OpenACC testsuites. Presentation given at the Jetstream Research Experience for Undergraduates Program Presentation Session. Also Presented at the Super Computer Conference in 2021. SC21

• "OpenACC 2.5 Validation Testsuite targeting multiple architectures" at the 2nd International Workshop on Performance Portable Programming Models for Accelerators (P^3MA) co-located with ISC, Germany, 2017. Paper: https://link.springer.com/chapter/10.1007/978-3-319-67630-2_39 Presentation Slides: http://www.csm.ornl.gov/workshops/p3ma2017/documents/Chandrasekaran_OpenACCV_V2.5.pdf

• "2014 IEEE International Parallel & Distributed Processing Symposium Workshops" Paper: https://ieeexplore.ieee.org/abstract/document/6969543?casa_token=dIj3u0IMYakAAAAA:iri-F4x3Hsbd0unFKtQlsMrb40KMHLX5-6IAD64-uzWZDjEux8_8ue1QgTMMZi7uVRurnWJkHw

(back to top)