[Perf] Windows/arm64: 21 Improvements on 3/15/2023 2:24:48 AM #14116
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Collections.ContainsFalse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 1.08 ms | 992.99 μs | 0.92 | 0.14 | False | |||||
| ICollection - Duration of single invocation | 1.08 ms | 994.07 μs | 0.92 | 0.14 | False |
_1.png)
_2.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'Details
Payloads
Histogram
System.Collections.ContainsFalse<String>.Queue(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 992.9930588942308 < 1.029389413175366.
IsChangePoint: Marked as a change because one of 11/29/2022 7:47:47 PM, 11/30/2022 3:45:04 PM, 12/13/2022 5:38:37 AM, 12/14/2022 12:26:48 AM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 31.733176559758405 (T) = (0 -995904.244767819) / Math.Sqrt((1945729485.7873638 / (299)) + (7569605.251814509 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08738617153695434 = (1091265.7837379526 - 995904.244767819) / 1091265.7837379526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<String>.ICollection(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 994.0663461538461 < 1.0236871153846152.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 12/19/2022 8:13:55 PM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 32.37670473350883 (T) = (0 -993948.2066544566) / Math.Sqrt((2509011497.5058722 / (299)) + (34278.101813836285 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08627693095875733 = (1087800.27596041 - 993948.2066544566) / 1087800.27596041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Memory.Constructors<String>
_1.png)
_2.png)
_3.png)
_4.png)
_5.png)
_6.png)
_7.png)
_8.png)
_9.png)
_10.png)
_11.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors<String>*'Details
Payloads
Histogram
System.Memory.Constructors<String>.SpanImplicitCastFromArraySegment
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.5347189635242025 < 11.347313371478482.
IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 45.05496757852895 (T) = (0 -5.663666517569443) / Math.Sqrt((0.36175659795554677 / (299)) + (0.05231536303347596 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.5206913680878782 = (11.816324890655924 - 5.663666517569443) / 11.816324890655924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.051031872489473 < 11.287755257337714.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 15.429099283869716 (T) = (0 -8.504271223518307) / Math.Sqrt((0.5417648369803071 / (299)) + (0.15674968163753944 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.29668647899912565 = (12.091721500556414 - 8.504271223518307) / 12.091721500556414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.598071685584395 < 11.015371517331804.
IsChangePoint: Marked as a change because one of 2/19/2023 7:20:18 PM, 2/21/2023 12:38:54 AM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 52.590274331379945 (T) = (0 -7.55175868131814) / Math.Sqrt((1.8578911217607588 / (299)) + (0.006375725637887203 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.388728565869446 = (12.354182217036584 - 7.55175868131814) / 12.354182217036584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.599284924581779 < 9.474570674694316.
IsChangePoint: Marked as a change because one of 1/30/2023 1:22:46 AM, 2/20/2023 2:00:07 PM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 67.95368333740129 (T) = (0 -5.598593474446074) / Math.Sqrt((1.74851601062982 / (299)) + (8.422971628192883E-06 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4814367549394247 = (10.7963561393405 - 5.598593474446074) / 10.7963561393405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.649583238970737 < 9.305046134301909.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 13.244229470384061 (T) = (0 -5.344741348934512) / Math.Sqrt((0.16298372810835055 / (299)) + (0.3624464094547738 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.46330463491840973 = (9.958612830803908 - 5.344741348934512) / 9.958612830803908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.5228091828893926 < 5.258281845587495.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 55.98755683050754 (T) = (0 -3.5831637583490914) / Math.Sqrt((0.07456588395639553 / (299)) + (0.0028361114237913324 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3506862999555411 = (5.5183861946910255 - 3.5831637583490914) / 5.5183861946910255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.108135533481588 < 8.852464583811523.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 108.06717567223276 (T) = (0 -5.094369386384681) / Math.Sqrt((0.12269833787698735 / (299)) + (0.0035862931888502435 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.45947620225403685 = (9.424875292500898 - 5.094369386384681) / 9.424875292500898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.05926451559199 < 9.90931218999372.
IsChangePoint: Marked as a change because one of 2/19/2023 10:06:54 AM, 2/20/2023 3:29:46 AM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 13.798003690783595 (T) = (0 -5.648154896965255) / Math.Sqrt((1.467311786236657 / (299)) + (0.4745252608595965 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4966114050627315 = (11.220267907875662 - 5.648154896965255) / 11.220267907875662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9071236617158456 < 8.981547771166918.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 16.02185591567877 (T) = (0 -3.639936641471296) / Math.Sqrt((0.24172459237998703 / (299)) + (0.40404355400258984 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.6183515892202396 = (9.537408092527892 - 3.639936641471296) / 9.537408092527892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.980822733648756 < 10.131659217122143.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 213.81359406522932 (T) = (0 -6.979603464193512) / Math.Sqrt((0.08864767162478909 / (299)) + (5.101850508976593E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3516568250676445 = (10.765291799241544 - 6.979603464193512) / 10.765291799241544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.24233288293553 < 3.455665991795308.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 4.522174909097116 (T) = (0 -2.653065778018641) / Math.Sqrt((0.043419901736388364 / (299)) + (0.12652717196590801 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2596171361445903 = (3.583370047495807 - 2.653065778018641) / 3.583370047495807 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Memory.Slice<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MemorySpanStart - Duration of single invocation | 8.62 ns | 7.60 ns | 0.88 | 0.03 | False | |||||
| ReadOnlyMemorySpanStart - Duration of single invocation | 8.34 ns | 7.20 ns | 0.86 | 0.02 | False |
_1.png)
_2.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Slice<String>*'Details
Payloads
Histogram
System.Memory.Slice<String>.MemorySpanStart
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.597736719825684 < 8.185339785109502.
IsChangePoint: Marked as a change because one of 1/5/2023 10:03:03 PM, 1/12/2023 4:43:01 PM, 2/9/2023 1:41:04 PM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 48.44643669793812 (T) = (0 -7.577516388513821) / Math.Sqrt((0.10561751476980524 / (299)) + (0.0005306399110369164 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12831563774788937 = (8.692958961585953 - 7.577516388513821) / 8.692958961585953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.ReadOnlyMemorySpanStart
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.202741706323987 < 7.875955201348438.
IsChangePoint: Marked as a change because one of 1/12/2023 11:13:59 PM, 2/9/2023 1:41:04 PM, 2/24/2023 9:01:57 AM, 3/9/2023 9:18:02 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 12.73952161736953 (T) = (0 -7.6542965043366875) / Math.Sqrt((0.07127495951025059 / (299)) + (0.05087583397662499 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.08070398980828639 = (8.326258810522228 - 7.6542965043366875) / 8.326258810522228 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Collections.IterateForEach<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 3.63 μs | 3.34 μs | 0.92 | 0.07 | False | |||||
| ImmutableQueue - Duration of single invocation | 4.43 μs | 4.10 μs | 0.93 | 0.10 | False | |||||
| Stack - Duration of single invocation | 3.50 μs | 3.14 μs | 0.90 | 0.15 | False |
_1.png)
_2.png)
_3.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<String>*'Details
Payloads
Histogram
System.Collections.IterateForEach<String>.Queue(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.338490137647226 < 3.47970275266977.
IsChangePoint: Marked as a change because one of 11/24/2022 1:04:57 PM, 12/15/2022 5:36:17 AM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 66.3683138532551 (T) = (0 -3330.985163766353) / Math.Sqrt((4242.7892645397205 / (299)) + (44.190726003147205 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09677928633329984 = (3687.897225301598 - 3330.985163766353) / 3687.897225301598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<String>.ImmutableQueue(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.102874345788704 < 4.210363929796282.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 37.653922926099355 (T) = (0 -4102.08634007) / Math.Sqrt((14895.95417610393 / (299)) + (90.09863655807871 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07580732780169194 = (4438.5618534636005 - 4102.08634007) / 4438.5618534636005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<String>.Stack(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1360354747221835 < 3.323344384272037.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 6.0786746887708345 (T) = (0 -3180.5997264686084) / Math.Sqrt((9566.526865754395 / (299)) + (9349.339554743996 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09685160448877736 = (3521.6800940760636 - 3180.5997264686084) / 3521.6800940760636 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentStack - Duration of single invocation | 1.96 secs | 739.30 ms | 0.38 | 0.56 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>*'Details
Payloads
Histogram
System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>.ConcurrentStack(Size: 2000000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 739.2980263157895 < 2.0998615462499997.
IsChangePoint: Marked as a change because one of 1/26/2023 7:41:12 PM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 47.58380996714285 (T) = (0 -770036715.7017542) / Math.Sqrt((1.1130590021759826E+17 / (299)) + (2567669440994931 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.6841017828977016 = (2437610198.5165386 - 770036715.7017542) / 2437610198.5165386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Collections.ContainsTrue<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableArray - Duration of single invocation | 550.68 μs | 507.59 μs | 0.92 | 0.14 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'Details
Payloads
Histogram
System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 507.5941397849462 < 523.134487759726.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 12/19/2022 8:13:55 PM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 35.827245032933554 (T) = (0 -507278.8778604907) / Math.Sqrt((602412006.5233744 / (299)) + (78765.14175003834 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09165192653128432 = (558463.0965565227 - 507278.8778604907) / 558463.0965565227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MultipleSerial - Duration of single invocation | 22.61 μs | 21.06 μs | 0.93 | 0.00 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Object>*'Details
Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.058344066666663 < 21.462826819166665.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 28.57349122482898 (T) = (0 -20956.572695726492) / Math.Sqrt((11378.721844171187 / (299)) + (9374.69433528647 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07122005716709985 = (22563.55001789359 - 20956.572695726492) / 22563.55001789359 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository