[Perf] Windows/arm64: 11 Improvements on 3/15/2023 2:24:48 AM #14128
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Memory.Constructors<String>
_1.png)
_2.png)
_3.png)
_4.png)
_5.png)
_6.png)
_7.png)
_8.png)
_9.png)
_10.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors<String>*'Details
Payloads
Histogram
System.Memory.Constructors<String>.SpanFromArrayStartLength
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.908332865211708 < 14.501331728623706.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 6.971215030111913 (T) = (0 -8.477902619979325) / Math.Sqrt((1.7039113562353805 / (299)) + (2.5242355124527713 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.43078830902778975 = (14.894111899738245 - 8.477902619979325) / 14.894111899738245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.445591200573007 < 14.80093111598009.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 16.366696731658788 (T) = (0 -10.452476159360115) / Math.Sqrt((2.435171429543888 / (299)) + (0.3195354547373216 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.34649112734481724 = (15.994390584005517 - 10.452476159360115) / 15.994390584005517 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.888832865822897 < 14.214940784079905.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 12.15715568373995 (T) = (0 -5.627049983139867) / Math.Sqrt((2.6328459416311802 / (299)) + (1.5587793559903138 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.610967401161557 = (14.464211996477601 - 5.627049983139867) / 14.464211996477601 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.64646519713435 < 13.535836937598654.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 15.732810888727254 (T) = (0 -7.90226245066767) / Math.Sqrt((2.6728953387180536 / (299)) + (0.5085647494553355 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.456834418047253 = (14.548533105242171 - 7.90226245066767) / 14.548533105242171 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.37879598344653 < 18.86174156777134.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 7.6669696671847465 (T) = (0 -12.72344286437626) / Math.Sqrt((3.009004008215086 / (299)) + (1.356060991676269 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.290588175213111 = (17.935199865322865 - 12.72344286437626) / 17.935199865322865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.867410184822242 < 9.04485899579586.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 39.14323535849796 (T) = (0 -5.929555413132614) / Math.Sqrt((1.3760561734350085 / (299)) + (0.006684573177851319 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3529930445902121 = (9.164592997886823 - 5.929555413132614) / 9.164592997886823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.444688014990003 < 15.261574491519166.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 20.374557903971386 (T) = (0 -9.479675939633525) / Math.Sqrt((4.937351078172965 / (299)) + (0.26539902316282393 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4105094854703314 = (16.081133972438874 - 9.479675939633525) / 16.081133972438874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.189295735835113 < 13.822639247831287.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 17.584428739155122 (T) = (0 -8.85332349715419) / Math.Sqrt((2.52024739546612 / (299)) + (0.338312554572469 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.40879798065901807 = (14.975123912842967 - 8.85332349715419) / 14.975123912842967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.149022523424634 < 16.621043605533703.
IsChangePoint: Marked as a change because one of 1/4/2023 10:05:35 PM, 1/7/2023 8:48:01 PM, 3/14/2023 3:18:57 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 8.048436105774492 (T) = (0 -10.628325061714314) / Math.Sqrt((11.257158878572671 / (299)) + (3.6616106606633045 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.4250472057465499 = (18.4855611937928 - 10.628325061714314) / 18.4855611937928 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.431179022041104 < 19.09134087052532.
IsChangePoint: Marked as a change because one of 1/12/2023 11:13:59 PM, 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 10.583280679389796 (T) = (0 -8.392287341766805) / Math.Sqrt((3.9735668877480013 / (299)) + (2.9021241560095956 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.5553225135306837 = (18.872750694892424 - 8.392287341766805) / 18.872750694892424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
Improvements in System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentStack - Duration of single invocation | 493.84 ms | 277.10 ms | 0.56 | 0.76 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>*'Details
Payloads
Histogram
System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>.ConcurrentStack(Size: 2000000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 277.098728 < 462.94990835.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 29.866615767433366 (T) = (0 -285418462.1713106) / Math.Sqrt((3039327736084074.5 / (299)) + (67465647831749.17 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.37420041193543135 = (456086049.9349861 - 285418462.1713106) / 456086049.9349861 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository