Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Windows/x64: 35 Regressions on 5/2/2023 10:35:24 AM #17605

Open
performanceautofiler bot opened this issue May 9, 2023 · 2 comments
Open

[Perf] Windows/x64: 35 Regressions on 5/2/2023 10:35:24 AM #17605

performanceautofiler bot opened this issue May 9, 2023 · 2 comments

Comments

@performanceautofiler
Copy link

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MemoryFromArray - Duration of single invocation 5.31 ns 8.56 ns 1.61 0.05 False
MemoryFromArrayStartLength - Duration of single invocation 5.51 ns 9.40 ns 1.70 0.09 False
SpanFromArrayStartLength - Duration of single invocation 4.36 ns 7.17 ns 1.64 0.03 False
ArrayAsSpan - Duration of single invocation 5.00 ns 7.17 ns 1.43 0.04 False
ArrayAsMemory - Duration of single invocation 6.14 ns 9.12 ns 1.48 0.06 False
ArrayAsSpanStartLength - Duration of single invocation 6.70 ns 8.84 ns 1.32 0.02 False
SpanFromArray - Duration of single invocation 3.68 ns 6.61 ns 1.80 0.04 False
ArrayAsMemoryStartLength - Duration of single invocation 7.58 ns 11.07 ns 1.46 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Constructors<String>.MemoryFromArray


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.5628071127807 > 5.576861737856464.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -114.29696864484852 (T) = (0 -8.56335967797707) / Math.Sqrt((0.012549942720872486 / (16)) + (6.253382033457137E-06 / (30))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (30) - 2, .025) and -0.5970871430828291 = (5.361861257894399 - 8.56335967797707) / 5.361861257894399 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Constructors<String>.MemoryFromArrayStartLength


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.398950750317228 > 5.791729372601059.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -56.14036431163663 (T) = (0 -9.45874529620186) / Math.Sqrt((0.01982487842745171 / (16)) + (0.10873033855118588 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.6917898214345194 = (5.590969502453624 - 9.45874529620186) / 5.590969502453624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Constructors<String>.SpanFromArrayStartLength


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.169314647696989 > 4.580104133495271.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -450.55367231294326 (T) = (0 -7.1721130414871235) / Math.Sqrt((0.0006130631234442999 / (16)) + (7.0028805852530025E-06 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.6393535919007765 = (4.374964057126501 - 7.1721130414871235) / 4.374964057126501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Constructors<String>.ArrayAsSpan


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.169400897681058 > 5.25197093830835.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -14.58038539395956 (T) = (0 -7.178042033747532) / Math.Sqrt((0.3138218647088309 / (16)) + (0.0011417692380134845 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.3980975048374429 = (5.134149806369996 - 7.178042033747532) / 5.134149806369996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Constructors<String>.ArrayAsMemory


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.119449323928043 > 6.45028325642643.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -186.3893540830592 (T) = (0 -9.121704730905492) / Math.Sqrt((0.0041578465481122324 / (16)) + (1.4438612304434867E-05 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.4918511186238148 = (6.1143532468038595 - 9.121704730905492) / 6.1143532468038595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Constructors<String>.ArrayAsSpanStartLength


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.839990538722486 > 7.038701374398039.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1400.091479845232 (T) = (0 -8.842444950641594) / Math.Sqrt((3.2044950324718434E-05 / (15)) + (5.839239863127528E-06 / (31))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (31) - 2, .025) and -0.3182463118426248 = (6.707733502612086 - 8.842444950641594) / 6.707733502612086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Constructors<String>.SpanFromArray


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.611695106307649 > 3.854121377477046.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -3345.7347868060615 (T) = (0 -6.614649111595471) / Math.Sqrt((8.693099725988118E-06 / (16)) + (7.143163256485408E-06 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.801546236905135 = (3.671651038476112 - 6.614649111595471) / 3.671651038476112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Constructors<String>.ArrayAsMemoryStartLength


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.07024710086991 > 7.964445498962209.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -47.237088328240226 (T) = (0 -11.14503850073057) / Math.Sqrt((0.0015381990195552661 / (16)) + (0.1746076273820724 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.4723091869938087 = (7.569767681397641 - 11.14503850073057) / 7.569767681397641 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Add - Duration of single invocation 12.39 ns 15.58 ns 1.26 0.08 True
Remainder - Duration of single invocation 6.93 ns 10.23 ns 1.48 0.07 True
Ctor_ByteArray - Duration of single invocation 10.81 ns 12.80 ns 1.18 0.05 False
Divide - Duration of single invocation 7.49 ns 11.91 ns 1.59 0.05 True
Subtract - Duration of single invocation 54.41 ns 60.57 ns 1.11 0.02 False
Ctor_ByteArray - Duration of single invocation 7.92 ns 9.83 ns 1.24 0.03 True
Subtract - Duration of single invocation 13.37 ns 16.34 ns 1.22 0.09 False
Multiply - Duration of single invocation 10.07 ns 13.86 ns 1.38 0.01 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Add(arguments: 16,16 bits)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.581370779445141 > 12.968371075357716.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -20.906313446665564 (T) = (0 -15.470450822042853) / Math.Sqrt((0.0019033542101310893 / (16)) + (0.6887269187340401 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.2530783519578782 = (12.34595649814792 - 15.470450822042853) / 12.34595649814792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 16,8 bits)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.23349919324876 > 7.278155594520967.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -23.40402924517638 (T) = (0 -11.065734584093102) / Math.Sqrt((8.108752297878292E-05 / (16)) + (0.9360561467251515 / (30))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (30) - 2, .025) and -0.5964878245485067 = (6.931299076598055 - 11.065734584093102) / 6.931299076598055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: -2147483648)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.79576397472724 > 11.369937956395981.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 3/5/2023 11:54:18 PM, 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -17.511841009317905 (T) = (0 -11.762767580954609) / Math.Sqrt((0.0010535368992931466 / (16)) + (0.08816747842965245 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.0873220373615365 = (10.818108321889431 - 11.762767580954609) / 10.818108321889431 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.907938729981035 > 7.861394233866898.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -4866.475256836527 (T) = (0 -11.90852583148143) / Math.Sqrt((6.372669969972835E-06 / (16)) + (1.3245857942141041E-05 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.5906065021392861 = (7.486783070146551 - 11.90852583148143) / 7.486783070146551 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.56822193041907 > 57.079355939834144.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/7/2023 3:57:10 PM, 4/9/2023 11:04:55 PM, 4/25/2023 3:46:39 AM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -90.45165337700212 (T) = (0 -60.968442012715535) / Math.Sqrt((0.03994901728245559 / (16)) + (0.08816064862423996 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.1216047585788907 = (54.35822338161663 - 60.968442012715535) / 54.35822338161663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 123)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.833492807799011 > 8.372652657595594.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -78.1752056227852 (T) = (0 -9.923811339662212) / Math.Sqrt((0.00987403611657527 / (16)) + (0.0012024263169792366 / (30))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (30) - 2, .025) and -0.2530529622204308 = (7.919706220619001 - 9.923811339662212) / 7.919706220619001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.339767242826124 > 14.040911395721242.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -9.864867333782232 (T) = (0 -14.98089511806153) / Math.Sqrt((0.27767234083977627 / (16)) + (0.5850881451991788 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.14329685789724605 = (13.103241747392207 - 14.98089511806153) / 13.103241747392207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 16,16 bits)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.863552389845244 > 10.58804450760119.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -651.6642675225054 (T) = (0 -13.857738455304965) / Math.Sqrt((0.0005191068133591713 / (16)) + (2.658043329776799E-05 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.37244142024577526 = (10.09714385683823 - 13.857738455304965) / 10.09714385683823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanArray - Duration of single invocation 6.57 ns 11.47 ns 1.75 0.01 True
FirstSpanTenSegments - Duration of single invocation 7.22 ns 10.07 ns 1.40 0.08 False
FirstSpanMemory - Duration of single invocation 7.28 ns 12.21 ns 1.68 0.02 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanArray


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.47276283046604 > 6.927530188944186.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -127.05603998230478 (T) = (0 -11.35228783246569) / Math.Sqrt((0.0058877183726034465 / (16)) + (0.032136680094136456 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.7225495331956076 = (6.59039848416165 - 11.35228783246569) / 6.59039848416165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanTenSegments


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.073611038211213 > 7.683240410027857.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -25.44183933876455 (T) = (0 -9.815846024266609) / Math.Sqrt((0.10434611339523595 / (16)) + (0.05441361379801162 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.3085682732211152 = (7.501210464246046 - 9.815846024266609) / 7.501210464246046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanMemory


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.209979172708131 > 7.640756237301494.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -155.26474471812668 (T) = (0 -12.341566913945524) / Math.Sqrt((0.003177384365732622 / (16)) + (0.027036648437336514 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.6997116384048677 = (7.260976882836281 - 12.341566913945524) / 7.260976882836281 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 8.31 ns 15.67 ns 1.89 0.02 True
TryParse - Duration of single invocation 9.98 ns 24.83 ns 2.49 0.03 True
TryParse - Duration of single invocation 2.45 ns 10.74 ns 4.38 0.08 True
Parse - Duration of single invocation 10.53 ns 22.56 ns 2.14 0.02 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Boolean.TryParse(value: "Bogus")


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.669525574502511 > 8.71929353058046.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -248.5314573566136 (T) = (0 -15.730624207104661) / Math.Sqrt((0.014032100358382444 / (16)) + (0.0009087461660479719 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.9070786922821502 = (8.248544892649523 - 15.730624207104661) / 8.248544892649523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.TryParse(value: " True ")


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.82722600439258 > 10.475032178008643.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -175.3090670126055 (T) = (0 -25.11984107718479) / Math.Sqrt((6.0728805274680866E-05 / (16)) + (0.23134257830076416 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -1.5191342606331608 = (9.971616626289364 - 25.11984107718479) / 9.971616626289364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.TryParse(value: "0")


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.739960949854812 > 2.5762244072223277.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -106.701793528524 (T) = (0 -10.51461447405426) / Math.Sqrt((0.01847957294244507 / (15)) + (0.13736244637440556 / (31))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (31) - 2, .025) and -3.231264784754782 = (2.4849814438317224 - 10.51461447405426) / 2.4849814438317224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.Parse(value: " True ")


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.558552721057534 > 11.056567713801948.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -116.57583910448999 (T) = (0 -23.493254163881762) / Math.Sqrt((0.010919178652641477 / (16)) + (0.35930568893834797 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -1.2208303888434087 = (10.578590009350902 - 23.493254163881762) / 10.578590009350902 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 126.97 μs 142.66 μs 1.12 0.30 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.66362012987014 > 133.5468004416823.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -8.580090999982545 (T) = (0 -140300.62568017803) / Math.Sqrt((21055529.5147007 / (16)) + (2030181.3722807285 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.07744561851113702 = (130215.96939068884 - 140300.62568017803) / 130215.96939068884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 9, 2023

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CopyTo<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 364.91 ns 397.10 ns 1.09 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<String>.Span(Size: 2048)


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 397.0958221784412 > 386.2307287241636.
IsChangePoint: Marked as a change because one of 3/25/2023 2:21:39 PM, 4/25/2023 1:57:48 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -18.267800411048366 (T) = (0 -386.30958511394783) / Math.Sqrt((7.389789693468262 / (16)) + (24.28120164841517 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.05570579878839437 = (365.9254174385564 - 386.30958511394783) / 365.9254174385564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanTenSegments - Duration of single invocation 6.00 ns 9.97 ns 1.66 0.07 True
FirstSpanArray - Duration of single invocation 6.28 ns 10.80 ns 1.72 0.02 True
FirstSpanSingleSegment - Duration of single invocation 7.84 ns 10.07 ns 1.28 0.07 True
FirstSpanMemory - Duration of single invocation 7.00 ns 11.64 ns 1.66 0.02 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanTenSegments


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.968378104182433 > 6.323055925891868.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -80.36048389961907 (T) = (0 -10.18151806344495) / Math.Sqrt((0.016252674323012552 / (16)) + (0.05444660643662212 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.7110491576128177 = (5.950453274907523 - 10.18151806344495) / 5.950453274907523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanArray


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.804839619001282 > 6.59259593161083.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -100.51585444416533 (T) = (0 -11.004309869600434) / Math.Sqrt((0.00306777605393438 / (16)) + (0.06332641774549227 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.7598870186981552 = (6.252850184519614 - 11.004309869600434) / 6.252850184519614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanSingleSegment


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.074278959924868 > 8.215930033506588.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -45.681833179286265 (T) = (0 -10.267218849275306) / Math.Sqrt((0.016876064565112082 / (16)) + (0.05172083570062803 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.30239141186491364 = (7.883358839546955 - 10.267218849275306) / 7.883358839546955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanMemory


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.63580740019065 > 7.378867764591119.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -120.48538711120126 (T) = (0 -11.822974470166313) / Math.Sqrt((0.0007981146526995756 / (16)) + (0.047717042395719486 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.6841987177697249 = (7.019940310738813 - 11.822974470166313) / 7.019940310738813 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsyncThenReadAsync - Duration of single invocation 41.61 ns 47.30 ns 1.14 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests*'

Payloads

Baseline
Compare

Histogram

System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.WriteAsyncThenReadAsync


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.300902706575584 > 43.74131464124931.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 3/5/2023 11:54:18 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -41.34098256813756 (T) = (0 -47.21752181182109) / Math.Sqrt((0.0852072411897934 / (16)) + (0.3884207407582591 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.1324936896683594 = (41.693408309982125 - 47.21752181182109) / 41.693408309982125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 123.45 μs 135.83 μs 1.10 0.25 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.8252119252874 > 129.5917304379921.
IsChangePoint: Marked as a change because one of 4/5/2023 12:23:38 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -17.058606208551424 (T) = (0 -136066.6298595319) / Math.Sqrt((6267147.612328222 / (15)) + (1169027.6783045053 / (31))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (31) - 2, .025) and -0.0924361618221872 = (124553.39233056174 - 136066.6298595319) / 124553.39233056174 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsyncThenReadAsync - Duration of single invocation 53.24 ns 59.91 ns 1.13 0.20 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'

Payloads

Baseline
Compare

Histogram

System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.90771725596099 > 55.97854237134709.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -21.690492887761653 (T) = (0 -60.96472984460045) / Math.Sqrt((1.612637358386664 / (16)) + (0.2398686782718312 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.13277087221006692 = (53.819118535116104 - 60.96472984460045) / 53.819118535116104 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.Ackermann

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 3.11 μs 3.35 μs 1.08 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Ackermann*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Ackermann.Test


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3487483329913834 > 3.267992898109253.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/25/2023 3:46:39 AM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -13.019588287863106 (T) = (0 -3308.4054969544986) / Math.Sqrt((1438.3670356320806 / (16)) + (3090.725680222969 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.05729161444898007 = (3129.132447228112 - 3308.4054969544986) / 3129.132447228112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Channels.Tests.UnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsyncThenReadAsync - Duration of single invocation 46.11 ns 52.38 ns 1.14 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'

Payloads

Baseline
Compare

Histogram

System.Threading.Channels.Tests.UnboundedChannelPerfTests.WriteAsyncThenReadAsync


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.375716235662104 > 48.512105873171826.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -54.97311750480259 (T) = (0 -53.12581211101907) / Math.Sqrt((0.12384993090204167 / (16)) + (0.2714799708865788 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.15328415897797013 = (46.06480692329866 - 53.12581211101907) / 46.06480692329866 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f
Compare c62f69be1405a8e41b56ffc05f22d791bf4c7d2d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor_String - Duration of single invocation 1.69 ns 6.06 ns 3.59 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Ctor_String


Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.055686655652301 > 1.7724912574149165.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -2397.1249688587736 (T) = (0 -6.056773409625581) / Math.Sqrt((4.879803516538901E-05 / (16)) + (8.246656850795078E-06 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -2.5802823850528425 = (1.691702708957184 - 6.056773409625581) / 1.691702708957184 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@cincuranet
Copy link

The ranges are dotnet/runtime@3e8f17a...4772b5d. Could it be dotnet/runtime#85620 @jakobbotsch? Worth transferring?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants