Create GitHub Actions Workflows for Publishing Playwright Reports

Important

All of the files in this section should be created in the main branch.

Configure Playwright

Note

If you want to include traces in your Playwright report, you cannot have *.js in the exclude section of _config.yml in gh_pages.

Open your Playwright configurations. Set the report to use at least the html and json reporters. Specify the output files because we will need to access them in the GitHub Actions.

import { defineConfig, devices } from '@playwright/test';  
  
export default defineConfig({  
	// ... other configs

	/* Reporter to use. See https://playwright.dev/docs/test-reporters */
	/* This example uses different reports for CI and local runs */
  reporter: process.env.CI
    ? [
        [
          "html",
          { outputFolder: "playwright-reports/html-report", open: "never" },
        ], // required for this setup, must be nested within its own folder to avoid getting confused with the other reports
        ["json", { outputFile: "playwright-reports/playwright-summary.json" }], // required for this setup
        ["github"], // OPTIONAL, creates a nice summary that the GitHub Action can display
      ]
    : [
        ["list"], // use whatever reports you like locally
      ],
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://127.0.0.1:3000',

    /* Collect trace for test failures. See https://playwright.dev/docs/trace-viewer */
    trace: "retain-on-failure",
    /* Save the videos for test failures. See https://playwright.dev/docs/videos */
    video: "retain-on-failure",
  },
	// ... other configs
});

Create a GitHub Actions Workflow for Publishing Reports

Create the file .github/workflows/playwright-report.yml that will run the e2e tests, upload the artifacts for the html and json reports, then call the publish workflow.

name: Playwright Tests

on:
  push:
    branches: # specify branches you want to trigger on or ignore
      - "*"
      - "!gh-pages"

# ensures that currently running Playwright workflows are canceled if branch is deleted
concurrency:
  group: Playwright-${{ github.event.ref }}
  cancel-in-progress: true

# this action needs the same permissions as the Push to GitHub Pages action because it is calling it
permissions:
  contents: write
  actions: read

jobs:
  # add any set up jobs needed to get e2e tests running in GitHub Actions
  e2e:
    runs-on: ubuntu-latest
    continue-on-error: true
    env:
      JSON_ARTIFACT_ID: json-playwright-report
      HTML_ARTIFACT_ID: html-playwright-report
    outputs:
      # these outputs access outputs from the steps and will be used in the publish job
      json_artifact_id: ${{ env.JSON_ARTIFACT_ID }}
      json_deploy_to: _data/${{ steps.variables.outputs.report_path }}
      html_artifact_id: ${{ env.HTML_ARTIFACT_ID }}
      html_deploy_to: ${{ steps.variables.outputs.report_path }}${{ steps.variables.outputs.report_name }}/
      commit_msg: ${{ steps.variables.outputs.commit_msg }}
      output_url: ${{ steps.variables.outputs.report_path }}${{ steps.variables.outputs.report_name }}/
      actualResult: ${{ steps.e2e-test.conclusion }}
    steps:
      - name: Set Report Path and Name
        id: variables
        # create variables for renaming the reports and for where they need to be deployed to
        # if the branch is main, the reports should be named main and they should be stored under playwright-reports/
        # if the branch is a feature branch, the reports should be named after the run id and the should be stored under playwright-reports/branchs/[branch_name] 
        # the commit messages should also be specific to the type of branch
        run: |
          if [[ ${{ github.ref_name }} == 'main' ]]; then
            echo "report_path=playwright-reports/" >> "$GITHUB_OUTPUT"
            echo "report_name=main" >> "$GITHUB_OUTPUT"
            echo "commit_msg=update Playwright report for main" >> "$GITHUB_OUTPUT"
          else
            echo "report_path=playwright-reports/branches/${{ github.ref_name }}/" >> "$GITHUB_OUTPUT"
            echo "report_name=${{ github.run_id }}" >> "$GITHUB_OUTPUT"
            echo "commit_msg=add Playwright report for branch ${{ github.ref_name }}, run-id ${{ github.run_id }}" >> "$GITHUB_OUTPUT"
          fi
      - name: Checkout
        uses: actions/checkout@v4
#### this is our set up for installing and running the tests, update this with your set up and test run ####
      - uses: pnpm/action-setup@v4
      - name: Install dependencies
        run: pnpm install
      - name: Install Playwright Browsers
        run: pnpm exec playwright install --with-deps
      - name: Run e2e Tests
        id: e2e-test
        run: pnpm exec playwright test
#### end set up and test run ####
      - name: Rename Reports
        if: ${{ !cancelled() }}
        # working-directory should point to the location of your Playwright reports
        working-directory: playwright-reports/
        # rename the reports to the report_name from above
        # change the names and locations to match your Playwright configurations
        run: |
          mv playwright-summary.json ${{ steps.variables.outputs.report_name }}.json
          mv html-report ${{ steps.variables.outputs.report_name }}
      - name: Upload JSON Report
        uses: actions/upload-artifact@v4
        id: upload-json
        if: ${{ !cancelled() }}
        with:
          name: ${{ env.JSON_ARTIFACT_ID }}
          # path should point to the location of your json Playwright report
          path: playwright-reports/${{ steps.variables.outputs.report_name }}.json
          retention-days: 30
      - name: Upload HTML Report
        uses: actions/upload-artifact@v4
        id: upload-html
        if: ${{ !cancelled() }}
        with:
          name: ${{ env.HTML_ARTIFACT_ID }}
          # path should point to the location of your html Playwright report
          path: playwright-reports/${{ steps.variables.outputs.report_name }}
          retention-days: 30

  publish:
    needs: [e2e]
    # call the push-to-gh-pages workflow to add the artifacts to gh-pages
    uses: ./.github/workflows/push-to-gh-pages.yml
    with:
      FILE_1_ARTIFACT_ID: ${{ needs.e2e.outputs.json_artifact_id }}
      FILE_1_DEPLOY_TO: ${{ needs.e2e.outputs.json_deploy_to }}
      FILE_2_ARTIFACT_ID: ${{ needs.e2e.outputs.html_artifact_id }}
      FILE_2_DEPLOY_TO: ${{ needs.e2e.outputs.html_deploy_to }}
      COMMIT_MSG: ${{ needs.e2e.outputs.commit_msg }}
      URL_PATH: ${{ needs.e2e.outputs.output_url }}
    secrets: inherit # allow called workflow to use GitHub secrets

Create a GitHub Actions Workflow for Removing Reports

When a branch is deleted, we want to remove it’s reports from the site. Create the file .github/workflows/playwright-report-cleanup.yml, which triggers on a branch deletion and attempts to remove that branch’s folders from _data/playwright-reports/branches/ and /playwright-reports/branches.

# Adapted from https://dev.to/ysfaran/how-to-use-playwright-with-github-actions-and-github-pages-4gdl
name: Delete Playwright Report for Branch

on:
  delete:

# ensures that currently running Playwright workflows are canceled if branch is deleted
concurrency:
  group: Playwright-${{ github.event.ref }}

jobs:
  delete_reports:
    name: Delete Playwright Reports
    runs-on: ubuntu-latest
    if: ${{ github.event.ref_type == 'branch' }}
    env:
      # name of the branch that was just deleted
      BRANCH_NAME: ${{ github.event.ref }}
    steps:
      - uses: actions/create-github-app-token@v1
        id: app-token
        with:
          app-id: ${{ vars.APP_ID }}
          private-key: ${{ secrets.PRIVATE_KEY }}
      - name: Get GitHub App User ID
        id: get-user-id
        run: echo "user-id=$(gh api "/users/${{ steps.app-token.outputs.app-slug }}[bot]" --jq .id)" >> "$GITHUB_OUTPUT"
        env:
          GH_TOKEN: ${{ steps.app-token.outputs.token }}
      - run: |
          git config --global user.name '${{ steps.app-token.outputs.app-slug }}[bot]'
          git config --global user.email '${{ steps.get-user-id.outputs.user-id }}+${{ steps.app-token.outputs.app-slug }}[bot]@users.noreply.github.com'
      - name: Checkout GitHub Pages Branch
        uses: actions/checkout@v4
        with:
          ref: gh-pages
          token: ${{ steps.app-token.outputs.token }}
      - name: Delete Data for Branch
        timeout-minutes: 3
        working-directory: _data/playwright-reports/branches/
        run: |
          # delete the data branch directory if it exists
          if [[ -d $BRANCH_NAME ]]; then
            rm -rf $BRANCH_NAME
            echo "Deleted Playwright Data for $BRANCH_NAME from GitHub Pages" >> "$GITHUB_STEP_SUMMARY"
          else 
            echo "Could not find Playwright Data for $BRANCH_NAME to delete"
          fi
      - name: Delete Reports for Branch
        timeout-minutes: 3
        working-directory: playwright-reports/branches/
        run: |
          # delete the report branch directory it it exists
          if [[ -d $BRANCH_NAME ]]; then
            rm -rf $BRANCH_NAME
            echo "Deleted Playwright Reports for $BRANCH_NAME from GitHub Pages" >> "$GITHUB_STEP_SUMMARY"
          else 
            echo "Could not find Playwright Reports for $BRANCH_NAME to delete"
          fi
      - name: Push Changes
        timeout-minutes: 3
        run: |
          if [[ -z $(git status --porcelain) ]]; then
            # if there were no changes, do nothing
            echo "Did not have Playwright Data or Reports to delete" >> "$GITHUB_STEP_SUMMARY"
          else
            git add -A
            git commit -am "workflow: remove all reports and data for branch $BRANCH_NAME"
            git pull origin gh-pages --rebase -s recursive -X ours
            git push
            echo "Deleted Playwright Data or Reports for $BRANCH_NAME from GitHub Pages" >> "$GITHUB_STEP_SUMMARY"
          fi

---

Next: Create GitHub Actions Workflows for Publishing Code Coverage Reports