Switch checked coreclr + libraries tests configs around in PRs #69180

jakobbotsch · 2022-05-11T11:36:36Z

Previously we did not run any checked coreclr + release libraries tests jobs in PR validation -- instead, we were running all platforms with checked coreclr + debug libraries tests.

We now run checked coreclr + libraries tests, where the libraries tests are compiled in the following configurations in PRs/rolling:

 # ________________________________________
 # | Platform         | PR      | Rolling |
 # | ---------------- | ------- | ------- |
 # | windows-x86      | Debug   | Release |
 # | linux-musl-x64   | Debug   | Release |
 # | OSX-x64          | Debug   | Release |
-# | linux-arm        | Debug   | Release |
+# | linux-arm64      | Debug   | Release |
-# | linux-musl-arm   | Debug   | Release |
-# | linux-musl-arm64 | Debug   | Release |
-# | linux-x64        | Debug   | Release |
-# | windows-x64      | Debug   | Release |
+# | linux-musl-arm   | Release | Release |
+# | linux-musl-arm64 | Release | Release |
+# | linux-x64        | Release | Release |
+# | windows-x64      | Release | Release |

ghost · 2022-05-11T11:36:43Z

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

Previously we did not run any checked coreclr + release libraries tests jobs in PR validation -- instead, we were running all platforms with checked coreclr + debug libraries tests.

The change in matrix is the following:

 # ________________________________________
 # | Platform         | PR      | Rolling |
 # | ---------------- | ---     | ------- |
 # | linux-musl-arm64 | Debug   | Release |
 # | linux-arm        | Debug   | Release |
 # | windows-x86      | Debug   | Release |
 # | Linux-musl-x64   | Debug   | Release |
-# | Linux-musl-arm   | Debug   | Release |
-# | Linux-x64        | Debug   | Release |
-# | Windows-x64      | Debug   | Release |
+# | Linux-musl-arm   | Release | Release |
+# | Linux-x64        | Release | Release |
+# | Windows-x64      | Release | Release |
 # | OSX-x64          | Debug   | Release |

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	`area-Infrastructure-libraries`
Milestone:	-

jakobbotsch · 2022-05-11T11:42:05Z

Note that the only arm64 job we run with checked coreclr + libraries tests is linux-musl-arm64. Yet we run both linux-arm and linux-musl-arm in the configuration. I think it would make sense to switch linux-arm or linux-musl-arm to linux-arm64 due to the importance of these platforms. Any objections or thoughts on this?

@dotnet/runtime-infrastructure @dotnet/area-infrastructure-libraries @jkotas @danmoseley

jakobbotsch · 2022-05-11T11:43:57Z

We may also consider switching OSX-x64 to OSX-arm64. I believe OSX-x64 follows the normal SysV ABI while macOS-arm64 is special, so at least for the JIT macOS-arm64 is more interesting.

danmoseley · 2022-05-11T13:54:26Z

Seems reasonable to me

jakobbotsch · 2022-05-11T14:54:48Z

The new change looks like:

 # ________________________________________
 # | Platform         | PR      | Rolling |
 # | ---------------- | ------- | ------- |
 # | windows-x86      | Debug   | Release |
 # | linux-musl-x64   | Debug   | Release |
 # | OSX-x64          | Debug   | Release |
-# | linux-arm        | Debug   | Release |
+# | linux-arm64      | Debug   | Release |
-# | linux-musl-arm   | Debug   | Release |
-# | linux-musl-arm64 | Debug   | Release |
-# | linux-x64        | Debug   | Release |
-# | windows-x64      | Debug   | Release |
+# | linux-musl-arm   | Release | Release |
+# | linux-musl-arm64 | Release | Release |
+# | linux-x64        | Release | Release |
+# | windows-x64      | Release | Release |

This requires us to add extra linux-x64 and windows-x64 release libraries builds for coreclr changes, but we can also get rid of multiple other builds:

win-x86 libraries does not need to be built in debug always, only for coreclr changes (preexisting)
linux-arm, linux-musl-arm and linux-musl-arm64 no longer need to be built in debug for coreclr changes

Changing OSX-x64 to OSX-arm64 is still left, but I think I will leave it to a follow up change as it takes some non-trivial evaluation of how the build dependencies change.

ghost · 2022-05-12T08:49:01Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Previously we did not run any checked coreclr + release libraries tests jobs in PR validation -- instead, we were running all platforms with checked coreclr + debug libraries tests.

We now run checked coreclr + libraries tests, where the libraries tests are compiled in the following configurations in PRs/rolling:

 # ________________________________________
 # | Platform         | PR      | Rolling |
 # | ---------------- | ------- | ------- |
 # | windows-x86      | Debug   | Release |
 # | linux-musl-x64   | Debug   | Release |
 # | OSX-x64          | Debug   | Release |
-# | linux-arm        | Debug   | Release |
+# | linux-arm64      | Debug   | Release |
-# | linux-musl-arm   | Debug   | Release |
-# | linux-musl-arm64 | Debug   | Release |
-# | linux-x64        | Debug   | Release |
-# | windows-x64      | Debug   | Release |
+# | linux-musl-arm   | Release | Release |
+# | linux-musl-arm64 | Release | Release |
+# | linux-x64        | Release | Release |
+# | windows-x64      | Release | Release |

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

jakobbotsch · 2022-05-16T10:16:30Z

eng/pipelines/runtime.yml

-    - ${{ if eq(variables['isRollingBuild'], false) }}:
-      # we need to build windows_x86 for debug on PRs in order to test
-      # against a checked runtime when the PR contains coreclr changes
-      - windows_x86


This build was only necessary on coreclr changes.

jakobbotsch · 2022-05-16T10:17:43Z

eng/pipelines/runtime.yml

-        - Linux_arm
-        - Linux_musl_arm
-        - Linux_musl_arm64


These three debug builds are no longer needed: we no longer run Linux_arm in PRs (replaced by Linux_arm64) and Linux_musl_arm/Linux_musl_arm64 now run in release for coreclr changes. However, windows-x86 is moved here (see comment above).

jakobbotsch · 2022-05-16T10:19:00Z

eng/pipelines/runtime.yml

      - windows_x86
      - windows_arm
      - windows_arm64
+      - Linux_arm


Unintentional change, but don't want to run CI again.

jakobbotsch · 2022-05-19T10:21:57Z

ping @dotnet/runtime-infrastructure

agocke

LGTM. There’s also a markdown doc in /docs/infra that I’ve been trying to add some of this info to. I think your table would be helpful there.

jakobbotsch · 2022-05-24T08:43:54Z

There’s also a markdown doc in /docs/infra that I’ve been trying to add some of this info to. I think your table would be helpful there.

I'll open a PR to add the table to that document.

Switch checked coreclr + libraries tests configs around in PRs

d86b184

ghost added the area-Infrastructure-libraries label May 11, 2022

ghost assigned jakobbotsch May 11, 2022

Random JIT change

bb13a7e

jakobbotsch added 6 commits May 11, 2022 16:24

Do win-x86 debug builds for coreclr changes only

a39ff64

Delete Linux_musl_arm debug libraries build on coreclr changes

b922360

Build linux-x64 windows-x64 release libraries on coreclr changes

f993aea

Move Linux_musl_arm64 to release on PRs

212d972

Switch linux-arm to linux-arm64

9ce6df9

Reorganize table

b6e7de8

jakobbotsch mentioned this pull request May 11, 2022

TaskCanceledException in msbuild while building on OSX-x64 #69190

Closed

jakobbotsch added 4 commits May 11, 2022 19:51

Libraries change instead of jit change

b27e1d7

Exclude runtime.yml from libraries/coreclr change evaluation temporarily

6bfd21c

Should have seen this one coming

5a7591a

Revert libraries change, make random coreclr change

01a0a3e

jakobbotsch added area-Infrastructure-coreclr and removed area-Infrastructure-libraries labels May 12, 2022

jakobbotsch added 6 commits May 12, 2022 10:54

Revert coreclr change, make random mono change

1720e1d

Some more temporary exclude paths

6dcc4d3

Back to coreclr change

bd305a8

Final libraries change try

73c91a5

Fix

eeef30f

Revert changes for testing

065416b

jakobbotsch mentioned this pull request May 16, 2022

Failure in SocketsHttpHandlerTest_Http3_MsQuic.SendMoreThanStreamLimitRequestsConcurrently_LastWaits: The server returned an invalid or unrecognized response #69387

Closed

jakobbotsch marked this pull request as ready for review May 16, 2022 10:15

jakobbotsch requested a review from a team May 16, 2022 10:15

jakobbotsch commented May 16, 2022

View reviewed changes

agocke approved these changes May 23, 2022

View reviewed changes

jakobbotsch merged commit 2a99e18 into dotnet:main May 24, 2022

jakobbotsch deleted the checked-coreclr-runtime-libraries-tests-on-pr-validation branch May 24, 2022 08:43

jakobbotsch mentioned this pull request May 24, 2022

Add matrix for libraries tests on coreclr change #69706

Closed

ghost locked as resolved and limited conversation to collaborators Jun 23, 2022

Switch checked coreclr + libraries tests configs around in PRs #69180

Switch checked coreclr + libraries tests configs around in PRs #69180

Uh oh!

Conversation

jakobbotsch commented May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented May 11, 2022

Uh oh!

jakobbotsch commented May 11, 2022

Uh oh!

jakobbotsch commented May 11, 2022

Uh oh!

danmoseley commented May 11, 2022

Uh oh!

jakobbotsch commented May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented May 12, 2022

Uh oh!

jakobbotsch May 16, 2022

Choose a reason for hiding this comment

Uh oh!

jakobbotsch May 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakobbotsch May 16, 2022

Choose a reason for hiding this comment

Uh oh!

jakobbotsch commented May 19, 2022

Uh oh!

agocke left a comment

Choose a reason for hiding this comment

Uh oh!

jakobbotsch commented May 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jakobbotsch commented May 11, 2022 •

edited

Loading

jakobbotsch commented May 11, 2022 •

edited

Loading

jakobbotsch May 16, 2022 •

edited

Loading