-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Remove the "run extended tests" github PR commend action #17119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
run extended tests |
I don't expect this to do anything, but. want to verify that is the case |
timsaucer
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, Andrew. This seems reasonable to disable based on the discussion and the limited benefit it has shown.
comphead
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @alamb
|
Here is another example of extended tests failing on main without us catching it 😢 |
WDYT if we enforce |
Sounds good to me |
|
Thank you @timsaucer and @comphead |
|
😢 |
Which issue does this PR close?
extendedtest suite from a PR #14319Run extended tests#15101Run extended testscommand #15463Rationale for this change
@danila-b made a very cool set of github runner workflows to run the extended tests on demand on CI. The idea was that
it could be used to run the extended tests on each PR that might cause breakages before merging to main.
However, in practice it seems it is not used much and hasn't stopped regressions
as we had hoped. For example here are some recent issues that were closed that
mentioned extended tests where extended tests broke after merging to main even
after the
run extended testscommand was added (on Mar 27, 2025):I think the challenge is twofold:
I am sure we could the second point, but the first remains very hard.
Also, the complexity of the extended workflow makes it harder to understanding
and maintain and verify the security of the CI workflows in general.
Thus, I think it is best to remove the extended github comment workflow for now.
What changes are included in this PR?
Are these changes tested?
I tested it manually
Are there any user-facing changes?
Less PR automation