-
Notifications
You must be signed in to change notification settings - Fork 285
Re-enable a benchmark was disabled earlier due to an issue #2590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
CI is not very usable right now, so I can't build this on |
|
@adamsitnik I'm curious about this from the original issue #2575: The run starts, and it throws: This shows | Method | numberString | arguments | Mean | Error | Median | Min | Max | Allocated |
|------------ |------------- |------------------ |---------:|------:|---------:|---------:|---------:|----------:|
| ToByteArray | -2147483648 | ? | 406.0 us | NA | 406.0 us | 406.0 us | 406.0 us | 104 B |
| ModPow | ? | 1024,1024,64 bits | NA | NA | NA | NA | NA | - |The reason I'm looking at this is because of dotnet/perf-autofiling-issues#7979 which says the value improved to |
@radical I've re-run the benchmark locally and the exported JSON file contains no statistics, but a single measurement (the jitting time). From BDN perspective I think it's a valid output. @DrewScoggins how the Reporting System handles such cases? {
"DisplayInfo":"Perf_BigInteger.ModPow: Job-QRRJIW(PowerPlanMode=00000000-0000-0000-0000-000000000000, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [arguments=1024,1024,64 bits]",
"Namespace":"System.Numerics.Tests",
"Type":"Perf_BigInteger",
"Method":"ModPow",
"MethodTitle":"ModPow",
"Parameters":"arguments=1024,1024,64 bits",
"FullName":"System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 1024,1024,64 bits)",
"HardwareIntrinsics":"AVX2,AES,BMI1,BMI2,FMA,LZCNT,PCLMUL,POPCNT VectorSize=256",
"Statistics":null,
"Memory":{
"Gen0Collections":0,
"Gen1Collections":0,
"Gen2Collections":0,
"TotalOperations":0,
"BytesAllocatedPerOperation":0
},
"Measurements":[
{
"IterationMode":"Overhead",
"IterationStage":"Jitting",
"LaunchIndex":1,
"IterationIndex":1,
"Operations":1,
"Nanoseconds":364600
}
]
} |
|
The issue doesn't seem to have been fixed by dotnet/runtime#74585 . Still getting the same failure in |
|
I took a look at the code and the way that we handle this is here, performance/src/harness/BenchmarkDotNet.Extensions/PerfLabExporter.cs Lines 33 to 122 in 5f63e8d
The gist is that we inspect the BDN object model and pull out results from the |
|
@DrewScoggins thanks! I've created new issue #2600 |
That is because we have not had an update of the runtime into the installer branch in 16 days, and the fix for this went in 12 days ago. I assume it is so broken because most of the focus is on the release branches these days, but I am not fully sure. |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
LoopedBard3
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM assuming generally clear run
adamsitnik
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thank you @radical !
Issue: #2575