Skip to content

[CPP-808] decouple CI #788

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed

[CPP-808] decouple CI #788

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a719d4d
use private action
adrian-kong Oct 25, 2022
f284892
move backend benchmark out
adrian-kong Oct 25, 2022
9260084
typo
adrian-kong Oct 25, 2022
e206295
move dir
adrian-kong Oct 25, 2022
25013d1
specify path
adrian-kong Oct 25, 2022
bbf19cf
specify main branch
adrian-kong Oct 25, 2022
c7efead
trigger ci
adrian-kong Oct 25, 2022
1f90c3e
use gh token
adrian-kong Oct 25, 2022
b6a5aab
ignore token ?
adrian-kong Oct 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 16 additions & 139 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ jobs:
runs-on: ${{ matrix.os }}

steps:

- name: Checkout source
uses: actions/checkout@v2
with:
Expand All @@ -48,74 +47,15 @@ jobs:
with:
ssh-private-key: ${{ secrets.SSH_KEY }}

- name: Run ssh-keyscan
run: ssh-keyscan github.com >> ~/.ssh/known_hosts

- name: Setup SSH for Windows Git LFS
run: |
& "C:\\Program Files\\Git\\bin\\sh.exe" -c "ssh-keyscan github.com >> ~/.ssh/known_hosts"
& "C:\\Program Files\\Git\\bin\\sh.exe" -c "echo '${{ secrets.SSH_KEY }}' >> ~/.ssh/id_rsa"
if: matrix.os == 'windows-2019'

- name: Install ${{ runner.os }} Dependencies.
shell: bash
run: |
if [ "$RUNNER_OS" == "Linux" ]; then
sudo apt-get update && sudo apt-get install -y capnproto libudev-dev libssl-dev
elif [ "$RUNNER_OS" == "macOS" ]; then
brew install capnp llvm
elif [ "$RUNNER_OS" == "Windows" ]; then
choco install -y capnproto
fi

- name: Pull Git LFS objects
run: git lfs pull
env:
GIT_SSH_COMMAND: ssh -o StrictHostKeyChecking=no

- uses: actions-rs/toolchain@v1
- uses: actions/checkout@v3
with:
profile: minimal
toolchain: stable

- uses: Swatinem/rust-cache@v1
with:
key: ${{ secrets.CACHE_VERSION }}

- uses: davidB/rust-cargo-make@v1
with:
version: ${{ env.CARGO_MAKE_VERSION }}

- name: Cache pip
uses: actions/cache@v2
with:
path: ${{ env.PIP_CACHE_DIR }}
key: ${{ runner.os }}-pyproject-toml-${{ secrets.CACHE_VERSION }}-${{ hashFiles('pyproject.toml') }}
repository: swift-nav/toolbox-backend-ci
path: "${{ github.workspace }}/benchmark-backend-ci"
ref: "main"


- name: Install ImageMagick
shell: bash
run: |
cargo make install-imagemagick

- name: Set up python builder
shell: bash
run: |
cargo make setup-builder
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Run Backend Benchmarks (Windows)
env:
LIBCLANG_PATH: ${{ env.LIBCLANG_PATH_WIN }}
run: |
cargo make backend-benches
if: matrix.os == 'windows-2019'

- name: Run Backend Benchmarks
run: |
cargo make backend-benches
if: matrix.os != 'windows-2019'
- run: |
./benchmark-backend-ci
rm -rf ./benchmark-backend-ci

checks:

Expand Down Expand Up @@ -615,83 +555,20 @@ jobs:
- macOS
- Windows
- Linux
runs-on: [self-hosted, '${{ matrix.os }}', bench]
runs-on: [ self-hosted, '${{ matrix.os }}', bench ]
env:
WINDOWPOS_NAME: windowpos
steps:
- name: Remove previous build.
shell: bash
run: |
rm -rf ${{ env.APP_NAME }}
rm -rf bench

- uses: actions/download-artifact@v2
- uses: actions/checkout@v3
with:
name: ${{ matrix.os }}-artifacts
path: |
${{ env.APP_NAME }}
repository: swift-nav/toolbox-frontend-ci
token: ${{ secrets.GITHUB_TOKEN }}
path: "${{ github.workspace }}/benchmark-frontend-ci"
ref: "main"

- uses: actions/download-artifact@v2
with:
name: ${{ matrix.os }}-artifacts-bench
path: |
bench

- name: Extract binary and data.
shell: bash
run: |
cd ${{ env.APP_NAME }}
archive=$(cat release-archive.filename)
7z x $archive -aoa
7z x ${archive%.xz} -aoa
echo "extracted release-archive"
mv ../bench/${{ matrix.os }}.zip .
7z x ${{ matrix.os }}.zip -aoa
echo "extracted bench data"

- name: Run CPU Frontend Benchmark.
uses: nick-invision/retry@v2
with:
shell: bash
timeout_minutes: 3
max_attempts: 3
retry_on: error
command: |
cd ${{ env.APP_NAME }}
if [ "$RUNNER_OS" == "Windows" ]; then
python ./bench_runner.py --frontend_cpu --executable="${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.APP_NAME }}.exe"
elif [ "$RUNNER_OS" == "macOS" ] || [ "$RUNNER_OS" == "Linux" ]; then
chmod +x "${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.APP_NAME }}"
if [ "$RUNNER_OS" == "Linux" ]; then
chmod +x "${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.WINDOWPOS_NAME }}"
fi
python ./bench_runner.py --frontend_cpu --executable="${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.APP_NAME }}"
else
echo "Invalid platform"
exit 1
fi

- name: Run MEM Frontend Benchmark.
uses: nick-invision/retry@v2
with:
shell: bash
timeout_minutes: 3
max_attempts: 3
retry_on: error
command: |
cd ${{ env.APP_NAME }}
if [ "$RUNNER_OS" == "Windows" ]; then
python ./bench_runner.py --frontend_mem --executable="${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.APP_NAME }}.exe"
elif [ "$RUNNER_OS" == "macOS" ] || [ "$RUNNER_OS" == "Linux" ]; then
chmod +x "${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.APP_NAME }}"
if [ "$RUNNER_OS" == "Linux" ]; then
chmod +x "${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.WINDOWPOS_NAME }}"
fi
python ./bench_runner.py --frontend_mem --executable="${GITHUB_WORKSPACE}/${{ env.APP_NAME }}/${{ env.APP_NAME }}"
else
echo "Invalid platform"
exit 1
fi
- run: |
./benchmark-frontend-ci
rm -rf ./benchmark-frontend-ci

sign_binaries:
name: Sign Binaries
Expand Down