Skip to content

Conversation

@jkotas
Copy link
Member

@jkotas jkotas commented Feb 3, 2020

Reverts #1195

@stephentoub stephentoub merged commit 729276d into master Feb 3, 2020
@stephentoub stephentoub deleted the revert-1195-type-isassignedfrom branch February 3, 2020 06:12
@trylek
Copy link
Member

trylek commented Feb 3, 2020

@jkotas, @AndyAyersMS - in the latest runtime-coreclr outerloop runs I see a systematic failure in two JIT tests (_il_reli_conv, _il_relu_conv) crashing on a JIT assert:

Assertion failed '(tree->gtFlags & (GTF_SIDE_EFFECT & ~GTF_CALL)) == 0' in 'Test:Main():int' (IL size 561)

At today infra standup @safern speculated that this rollback commit should fix the issue but I'm not sure and I see the problem reproing in the latest outerloop runs - can you please clarify so that I know whether I should file a new issue tracking this?

An example of such failed run is here:

https://dev.azure.com/dnceng/public/_build/results?buildId=505212&view=results

Thanks

Tomas

@safern
Copy link
Member

safern commented Feb 3, 2020

I see that for that assertion in particular hitting the outerloop job is this issue: #31665

@AndyAyersMS
Copy link
Member

That assert is from a different change, #1751, that wasn't rolled back. There is already an issue, #31665.

@trylek
Copy link
Member

trylek commented Feb 3, 2020

OK, great, thanks for clarification! Is there an ETA for a fix - in essence, do we need to temporarily disable the tests to avoid showing Steve and Chris a zero per cent outerloop pass rate again?

@jkotas
Copy link
Member Author

jkotas commented Feb 3, 2020

the tests to avoid showing Steve and Chris a zero per cent outerloop pass rate again?

For dotnet/runtime, outerloop is designed to catch what falls through the cracks and there is always going to be something falling through the cracks given the scope of the repo.

We should fix the metrics that we are tracking where we are trying to hit 100%. I think we should be tracking pass rates of the inner loop tests.

@trylek
Copy link
Member

trylek commented Feb 3, 2020

We should fix the metrics that we are tracking where we are trying to hit 100%. I think we should be tracking pass rates of the inner loop tests.

Hmm, the problem is that today innerloop tests = PR tests that are quite noisy as people are (naturally) sometimes testing buggy changes. We would need to create a new flavor of "innerloop rolling build" that doesn't exist today AFAIK.

@jkotas
Copy link
Member Author

jkotas commented Feb 3, 2020

We would need to create a new flavor of "innerloop rolling build" that doesn't exist today AFAIK.

That; or we can be tracing number of failing tests on merged PRs, or number of retries people end up doing to get a green CI before merging the PR.

@safern
Copy link
Member

safern commented Feb 3, 2020

the tests to avoid showing Steve and Chris a zero per cent outerloop pass rate again?

I'm not sure if the coreclr-outerloop job is tracked by CI council, I think for dotnet/runtime we only track runtime CI build definition. @ViktorHofer @Chrisboh ?

@ViktorHofer
Copy link
Member

You can find the build definitions that we track here in the dashboard: https://dnceng.visualstudio.com/public/_dashboards/dashboard/40ac4990-3498-4b3a-85dd-2ffde961d672

@trylek
Copy link
Member

trylek commented Feb 3, 2020

Hmm, I just tried that in three different browsers and it claims that "I'm not a member of this team project". When I logged into Azure DevOps it shows me a bunch of "organizations" but I don't see "dnceng.visualstudio.com" among them. Have you got any idea how the access rights to the dashboard are managed? Even weirder, I was looking at the dashboards during our CI council meetings and I'm sure that it worked then.

@ViktorHofer
Copy link
Member

cc @chcosta

@chcosta
Copy link
Member

chcosta commented Feb 3, 2020

When I click on the link that Viktor provided, for some reason it says I'm not signed in. So, I then have to sign in and then navigate to the dashboard. Perhaps that's what you're seeing?

@trylek
Copy link
Member

trylek commented Feb 3, 2020

Thanks @chcosta for your quick response; the initial part works the same for me - the infra prompts me to log in - but then it just shows me a list of "organizations" - presumably the ones I'm authorized to access - and dnceng.visualstudio.com is not among them, nor can I circumvent this by pasting the link to the dashboard URL in the login web page, at least I haven't found a way yet.

@chcosta
Copy link
Member

chcosta commented Feb 3, 2020

that doesn't make sense, I see you as a "User" on dnceng. In the list of organizations, I see dnceng as dev.azure.com/dnceng rather than dnceng.visualstudio.com.

@trylek
Copy link
Member

trylek commented Feb 3, 2020

Ahh, great, thanks @chcosta, after massaging Viktor's link as follows it works for me just great:

https://dev.azure.com/dnceng/public/_dashboards/dashboard/40ac4990-3498-4b3a-85dd-2ffde961d672

jkotas added a commit that referenced this pull request Feb 4, 2020
jkotas added a commit that referenced this pull request Feb 4, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants