[Perf] Linux/x64: 3 Regressions on 4/9/2023 1:58:52 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	1411364699b5784040e86f76cb3db8200f6a2c8c
Compare	082c5b7aeb401cce9e5a4dafbc01f1b20a8e624e
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	135.94 ns	176.88 ns	1.30	0.30	False
ToString - Duration of single invocation	359.46 ns	590.32 ns	1.64	0.17	True
ToString - Duration of single invocation	384.79 ns	588.27 ns	1.53	0.13	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.ToString(format: "o")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 176.880452137717 > 139.48936323267776.
IsChangePoint: Marked as a change because one of 3/11/2023 8:53:40 AM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM.
IsRegressionStdDev: Marked as regression because -21.913889597876825 (T) = (0 -187.78447385689694) / Math.Sqrt((27.762036945366173 / (43)) + (37.106050471920845 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.39776223914291636 = (134.34650657900383 - 187.78447385689694) / 134.34650657900383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_DateTimeOffset.ToString(format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 590.3247581516064 > 379.55497874593624.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM.
IsRegressionStdDev: Marked as regression because -22.26156683030572 (T) = (0 -596.9021124846057) / Math.Sqrt((227.8616747856383 / (42)) + (725.409350736849 / (7))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (7) - 2, .025) and -0.6379253879813885 = (364.4257039206405 - 596.9021124846057) / 364.4257039206405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

System.Tests.Perf_DateTimeOffset.ToString(format: "s")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 588.2702821895048 > 398.38011618075404.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM.
IsRegressionStdDev: Marked as regression because -27.56908889360104 (T) = (0 -557.596635371516) / Math.Sqrt((139.53165116867052 / (42)) + (303.11652365372305 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.4649960108905367 = (380.61307418343483 - 557.596635371516) / 380.61307418343483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

caused by dotnet/runtime#84469