Commit 9620cce
committed
[CI] Reworked the CI to Make it Faster
The CI for VTR was doing far more work than it needed to, which was
leading to long CI run times of around 1.5 hours on average. Overall,
the CI used 12.5 hours of compute and used more than 20 GitHub runners
at a time which hurt concurrency of runs.
The PR reduces the compute by only building VTR once for general builds
and using that build throughout when needed. This reduces the CI compute
down to 7.5 hours. In this process, I also added dependency chains to
try and schedule the runs efficiently to keep the number of active
GitHub runners below 10 (which is our current maximum).
I also fixed a bug with the way we have been using CCache. We were using
the same cache for all builds, which works fine for some projects; but
for ours that causes tons of cache misses when a gcc-11 build cache is
used for a Clang-18 cache for example. This PR makes each build's cache
unique, which enables better cache hit rates. I have found that when the
cache hit rate is perfect (i.e. the build is unchanged from the last
run), the CI uses less than 3 hours of compute and the test portion only
takes 20 minutes. When this happens, building the container actually
becomes the tall-pole since it often takes a little over 30 minutes.1 parent 4bb340a commit 9620cce
File tree
13 files changed
+271
-161
lines changed- .github
- scripts
- workflows
- dev/vtr_test_suite_verifier
- vtr_flow/tasks/regression_tests
- vtr_reg_multiclock_odin/func_multiclock
- blanket/config
- iterative/config
- once/config
- vanilla/config
- vtr_reg_valgrind_small_odin
- valgrind_small_odin/config
- vtr_reg_valgrind_small
13 files changed
+271
-161
lines changedThis file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
135 | 135 | | |
136 | 136 | | |
137 | 137 | | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
0 commit comments