- Overview
- Installation
- Command Line Features
- Dashboard HTML Features
- Version Details
- Usage
- Examples
- Custom Database Class
- Contributing
- License
The Robot Framework Dashboard is a tool for Robot Framework that provides insight of your test results across multiple runs. The tool makes use of the built in Robot Framework Result Visitor API to analyse output.xml files, stores these in a simple sqlite3 database and finally creates a HTML dashboard that makes use of Chart.js and Datatables for the graphs and tables and makes use of Bootstrap for styling.
If you need help, have suggestions or want to discuss anything, feel free to contact through the Slack Channel or through a GitHub issue
- Install Robot Framework 6.0 or higher (if not already installed):
- Install Robot Framework Dashboard
pip install robotframework-dashboard
When running the tool the following steps will be executed
- Database preparation creates robot_results.db or specified database if it does not yet exist and connects to it
- Optionally reads test execution result from 1 or more output XML file(s) and uploads to the database
- Optionally lists all runs currently available in the database, ordered by the time they were entered
- Optionally removes run(s) specified by run_date or index of being added
- Optionally generates a dashboard HTML file that can be used to look at the stored results
The Robot Framework Dashboard HTML will provide information across your runs across all levels. Currently available features:
- Global filters: run, run tag, run date
- Runs
- Statistics: pass, fail, skip
- Duration: total elapsed time
- Overview: table overview of runs
- Suites
-
- Filter: suite
- Statistics: pass, fail, skip
- Duration: total elapsed time
- Most failed: top 10 failing suites
- Overview: table overview of suites
- Tests
- Filter: suite, test
- Statistics: pass, fail, skip
- Duration: total elapsed time
- Most failed: top 10 failing tests
- Fail messages: top 10 fail messages
- Overview: table overview of tests
- Keywords
- Filter: keywords
- Keyword statistics: pass, fail, skip
- Keyword times run
- Keyword total duration
- Keyword average duration
- Keyword min duration
- Keyword max duration
- Overview: table overview of keywords
- Tested robot versions: 6.0, 6.0.1, 6.0.2, 6.1, 6.1.1, 7.0, 7.0.1, 7.1, 7.1.1
- When running robotdashboard with robot 6.x installed, output files that are generated by robot 7.x will have no duration fields when processed, this is caused by changes in the Result Visitor model
- When running robotdashboard with robot 7.x installed every output generated by any version above should be processed correctly
Run robotdashboard through the command line to initialize your database and generate the report:
robotdashboard -o output.xml
For detailed command line options take a look at:
robotdashboard -h
or
robotdashboard --help
Advanced examples:
- Generate the report without adding new outputs
robotdashboard
- Add tags to your output.xmls
robotdashboard -o output1.xml:tag1 -o reports/output2.xml:tag1:tag2
- Add all *output*.xml in an entire folder tree to the database and provide all these runs with tags
robotdashboard -f ./reports
robotdashboard -f ../../some_folder/sub_folder/logs
robotdashboard -f C:/nightly_runs:tag1:tag2:tag3
- Do not list runs and/or do not create the dashboard HTML
robotdashboard -l false -g false
- Custom path/file name for your dashboard HTML
robotdashboard -n results/result_robot_dashboard.html
- Custom database path/file location
robotdashboard -d result_data/robot_result_database.db
- Remove data from the database, can be based on run number (index) or run start time
robotdashboard -r 0
robotdashboard -r -1 -r "2024-10-17 15:05:04.563559"
- Add a custom dashboard html title
robotdashboard -t "My Cool Title"
- Show milliseconds in the graphs in the dashboard html (setting excludemilliseconds to False as it defaults to True)
robotdashboard -e False
- Make use of a custom DatabaseProcessor class see also Custom Database Class for examples and more details of the requirements.
robotdashboard -c ./path/to/custom_class.py
robotdashboard --databaseclass mysql.py
Here are some examples of generated files/output:
- Full Dashboard -> Download and open in a browser to use it!
- Database -> Download and use any tool to check the tables or use it as a base for using the "robotdashboard" command line interface
- Example Command Line Output (below)
This is an example after running robotdashboard for the first time. 3 outputs are added, stored and processed in the dashboard HTML.
Input:
robotdashboard -o output1.xml -o output2.xml -o output3.xml
Results:
======================================================================================
____ ___ ____ ___ _____ ____ _ ____ _ _ ____ ___ _ ____ ____
| _ \ / _ \| __ ) / _ |_ _| _ \ / \ / ___|| | | | __ ) / _ \ / \ | _ \| _ \
| |_) | | | | _ \| | | || | | | | |/ _ \ \___ \| |_| | _ \| | | |/ _ \ | |_) | | | |
| _ <| |_| | |_) | |_| || | | |_| / ___ \ ___) | _ | |_) | |_| / ___ \| _ <| |_| |
|_| \_\\___/|____/ \___/ |_| |____/_/ \_|____/|_| |_|____/ \___/_/ \_|_| \_|____/
======================================================================================
1. Database preparation
created database connection: 'robot_results.db'
======================================================================================
2. Processing output XML(s)
Processing output XML 'output1.xml'
Processed output XML 'output1.xml' in 0.04 seconds
Processing output XML 'output2.xml'
Processed output XML 'output2.xml' in 0.05 seconds
Processing output XML 'output3.xml'
Processed output XML 'output3.xml' in 0.05 seconds
======================================================================================
3. Listing all available runs in the database
Run 0 | 2024-10-17 15:05:04.563559 | RobotFramework-Project1
Run 1 | 2024-10-18 16:43:12.772757 | Robotframework-Project2
Run 2 | 2024-10-21 10:54:25.663094 | Robotframework-Project1
======================================================================================
4. Removing runs from the database
skipping step
======================================================================================
5. Creating dashboard HTML
created dashboard 'robot_dashboard_20241021-150726.html' in 0.01 seconds
See the Example Folder for some examples of custom database class implementations.
Currently available database type examples:
- template.py (only requirements are in the template file)
- sqlite3.py (for a sqlite3 database connection, this configuration is used by default in robotdashboard)
- mysql.py (for a mysql database connection)
- ...
If you have made an implementation that is not yet an example please feel free to add it through a pull request or submit it in an issue. This way you can help other people implement their own solutions!!
- File: The filename can be anything as long as it is a python file
- Class: The you should name the class DatabaseProcessor
- Methods:
- __init__(self, database_path: Path):, should handle the connection to the database and possible creation of the tables if necessary
- close_database(self):, should close the database connection
- insert_output_data(self, output_data: dict, tags: list):, should be able to handle the output_data dict and the run tags that are provided. Look at the example implementations for the content of output_data and tags.
- get_data(self):, should retrieve all data (runs/suites/tests/keywords) from all tables in the form of a dictionary containing the 4 data types. In which each data type is a list of dicts with entries. Example: {'runs': [{"run_start": "2024-10-13 22:33:19", "full_name": "Robotframework-Dashboard", "name": "Robotframework-Dashboard", "total": 6, "passed": 4, "failed": 1, "skipped": 1, "elapsed_s": "6.313", "start_time": "2024-10-13 22:33:19.673821", "tags": ""}, {"run_start"...}]}
- list_runs(self):, should print out the runs in the database with useful identiefiers. This function can be empty as long as it exists. It is purely for the command line overview.
- remove_runs(self, remove_runs):, should be able to handle either indexes to be removed, or run_starts that are provided which can then be used to delete the data in the database.
- Do not use relative imports! This will not work on runtime!
Contributions are welcome! If you encounter any issues, have suggestions for improvements, or would like to add new features, feel free to open an issue or submit a pull request.
This project is licensed under the MIT License.
Note: This project is not officially affiliated with or endorsed by robotframework.