Abstract image of a woman sitting on books and studying on a laptop

BKK16-317 How to generate power models for EAS and IPA

Linaro, profile picture
Linaro

The document discusses methods for generating power models for energy-aware scheduling (EAS) and idle power management (IPA) without relying on hardware engineer assistance. It emphasizes the importance of platform-specific parameters, outlines the power measurement methodology on a Hikey board, and presents findings on CPU and cluster power states. The conclusions highlight the effectiveness of IPA in performance improvement while suggesting further considerations for recording temperature data alongside power measurements.

Technology
1 of 44
Downloaded 30 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
Presented by Date Event How to generate power models for EAS and IPA … without talking to a hardware engineerLeo Yan Daniel Thompson BKK16-317 March 9, 2016 Linaro Connect BKK16
Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
Background ● Generating platform-specific parameters for power models are a prerequisite for deploying EAS and IPA ○ Software engineers need these parameters to work on these features ○ Gaining assistance from hardware engineers is difficult and, for engineers who don’t work for SoC vendors, impossible ○ It is possible to generate an acceptable (though perhaps not optimal) model using simple tools ● Today we will focus on CPU and cluster level power modeling ○ Excludes SoC level power modeling (GPU, DDR, ...)
The role of power modeling Power Modeling EAS/IPA Profiling Generate power modeling 1 Enable EAS/IPA 2 Enable profiling environment 3 Check if use bad power modeling 4 Refine algorithm 5 Refine power modeling 6
Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
CPU power states CPU State PD_CPUx CPU P-State On WFI On, internal clock gating CPU Off Off
CPU P-State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU0 Powered Off Powered On Clock Gating
CPU WFI State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0: WFI SCU Cluster 0 CLKIN PDCPU0 Powered Off Powered On Clock Gating
CPU Off State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU0 Powered Off Powered On Clock Gating X X
Cluster power states Cluster State PD_CPUx PDCORTEXA53 PD_L2 Cluster P-State On or Off On On Cluster L2 Retention Off Off Retention Cluster Off Off Off Off Cannot keep cache coherent between two clusters
Cluster P-State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU<n> Powered Off Powered On Clock Gating PDCORTEXTA53 PDL2 X X
Cluster L2 Retention State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU<n> Powered Off Powered On Clock Gating PDCORTEXTA53 PDL2 X X X
Cluster Off State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU<n> Powered Off Powered On Clock Gating PDCORTEXTA53 PDL2 X X X X
Link power states in Linux Kernel Power State Linux Kernel CPU P-state P-state WFI C-state CPU Off C-state Cluster P-state P-state Cluster L2 Retention C-state Cluster Off C-state CPUFreq does not distinguish “CPU P-state” and “Cluster P- state” status as usually binding CPU and Cluster frequency from hardware design. But EAS needs to model these two states separately.
Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
Hikey PMIC Hierarchy SoC: Hi6220 DC IN TO 4.2VDC, 4A PMIC: Hi6553 12V VDD_4V2 LX0 LX1 BUCK1_1V05 Ideal measure point, but hardware design does not encourage measurement here Select CPU’s nearest measurement point BUCKs & LDOs ACPU R247R7
Connect with ARM energy probe DC IN TO 4.2 VDC, 4A PMIC: Hi6553 VDD_4V2 0.033Ω ARM energy probe Hikey Board PC: Linux V+ V-Ground USB
Methodology For Power Measurement ● Use single CPU to measure power ○ All other CPUs will be hot unplugged (CPU off state) ○ CPU0 is left connected ● Use ARM Trusted Firmware to perform system suspend ○ Force cluster and CPU0 to enter specific C-states and P-states ● Measure power data based on different operating points ○ Each OPP implies specific pair of voltage and frequency ○ Measure all C-states and P-states at given OPP ○ Use “dhrystone” to generate 100% CPU utilization for CPU P-state
hikey_afflvl_off Linux kernel ARM CPUARM CPU0ARM CPU0ARM CPU0 Setup power state ARM CPUARM CPU0ARM CPU1 CPU1..7 echo 0 > /sys/devices/system/cpu/cpu[1..7]/online psci_cpu_off ARM TF ARM CPUARM CPU0 ARM CPU0 ARM CPU4 echo mem > /sys/power/state ARM CPU0 psci_system_suspend_enter Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC ARM CPUARM CPU0 ARM CPU0 ARM CPU4 Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC ARM CPUARM CPU0 ARM CPU0 ARM CPU4 Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC Hack low level code to only power off CPU but leave cluster on 1 2 ARM CPUARM CPU0 ARM CPU0 ARM CPU4 Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC hisi_ipc_cpu_suspend hisi_ipc_psci_system_off hisi_ipc_cluster_suspendhikey_affinst_suspend
Cluster and CPU’s Power Data OPP (MHz) Voltage (v) Cluster Power Off State (mW) Cluster P-State (mW) CPU WFI State (mW) CPU P-State (mW) 208 1.04 344 360 379 429 432 1.04 345 374 387 498 729 1.09 346 393 408 617 960 1.18 352 427 443 794 1200 1.33 367 479 508 1149 Difference is caused by other components in SoC that share the same regulator “Cluster P-state” means all CPUs in cluster have been powered off but cluster is powered on
Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
EAS energy model CPU0 CPU1 Cluster Time C-state P-state ● Formula: energy = energy_cluster + ∑ energy_CPU in cluster ● C-states: Idle energy, include cpu level and cluster level ● P-states: Busy energy, include cpu level and cluster level
Normalized CPU capacity ● Normalized CPU capacity is a ratio value ○ Ratio between actual performance of CPU and the maximum performance of CPU in the system ○ Normalized into range [0..1024] ● Normally use “dhrystone” to measure CPU capacity ○ Safe to derive from CPU frequency if all CPUs are homogeneous static struct capacity_state cap_states_core_a53[] = { /* Power per cpu */ { .cap = 178, .power = 69, }, /* 208MHz */ { .cap = 369, .power = 124, }, /* 432MHz */ { .cap = 622, .power = 224, }, /* 729MHz */ { .cap = 819, .power = 367, }, /* 960MHz */ { .cap = 1024, .power = 670, }, /* 1.2GHz */ };
Basic method to calculate for power data ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU0 OPP: 729 MHz SCU Cluster 0 ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU0 SCU Cluster 0 ARM CPU0 OPP: 729 MHz Remove cluster level power consumption CPU level power consumption
Power of CPU C-State OPP (MHz) Cluster Power On State (mW) CPU WFI (mW) Power of CPU WFI (mW) 208 360 379 19 432 374 387 13 729 393 408 15 960 427 443 16 1200 479 508 31 static struct idle_state idle_states_core_a53[] = { { .power = 15 }, /* active idle state */ { .power = 15 }, /* WFI state */ { .power = 0 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; Select 15mW as middle value for WFI state’s power data
Power of CPU P-State OPP (MHz) Cluster P-State (mW) CPU P-State (mW) Power of CPU P-state (mW) 208 360 429 69 432 374 498 124 729 393 617 224 960 427 794 367 1200 479 1149 670 static struct capacity_state cap_states_core_a53[] = { /* Power per cpu */ { .cap = 178, .power = 69, }, /* 208MHz */ { .cap = 369, .power = 124, }, /* 432MHz */ { .cap = 622, .power = 224, }, /* 729MHz */ { .cap = 819, .power = 367, }, /* 960MHz */ { .cap = 1024, .power = 670, }, /* 1.2GHz */ }; CPU P-state is calculated with static leakage and dynamic power, so should use Power (CPU P-state) - Power (Cluster P-state)
Cluster’s active idle state ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU: Off SCU Cluster 0 Powered Off Powered On Clock Gating ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU: WFI SCU Cluster 0 ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU: WFI or Off SCU Cluster 0 At CPU level, all CPUs in the cluster are in one of the idle states (WFI or Off state) and the cluster is not processing any software. At cluster level, the cluster remains active. Is Active idle state a C-state or a P-state? The main difference is if we need consider it with different OPP...
Power of Cluster C-State OPP (MHz) Cluster Power Off State (mW) Cluster Power On State (mW) Power of cluster level active idle state (mW) 208 344 360 16 432 345 374 29 729 346 393 47 960 352 427 75 1200 367 479 112 static struct idle_state idle_states_cluster_a53[] = { { .power = 47 }, /* active idle state */ { .power = 47 }, /* WFI state */ { .power = 47 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; Only in cluster power off state will we totally power off whole cluster, in other idle states the cluster actually is powered on even when all CPUs are powered off.
Power of Cluster P-State OPP (MHz) Cluster Power Off State (mW) Cluster Power On State (mW) Power of cluster level active idle state (mW) 208 344 360 16 432 345 374 29 729 346 393 47 960 352 427 75 1200 367 479 112 static struct capacity_state cap_states_cluster_a53[] = { /* Power per cluster */ { .cap = 178, .power = 16, }, { .cap = 369, .power = 29, }, { .cap = 622, .power = 47, }, { .cap = 819, .power = 75, }, { .cap = 1024, .power = 112, }, };
static struct idle_state idle_states_cluster_a53[] = { { .power = 47 }, /* active idle state */ { .power = 47 }, /* WFI state */ { .power = 47 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; static struct capacity_state cap_states_cluster_a53[] = { /* Power per cluster */ { .cap = 178, .power = 16, }, { .cap = 369, .power = 29, }, { .cap = 622, .power = 47, }, { .cap = 819, .power = 75, }, { .cap = 1024, .power = 112, }, }; static struct idle_state idle_states_core_a53[] = { { .power = 15 }, /* active idle state */ { .power = 15 }, /* WFI state */ { .power = 0 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; static struct capacity_state cap_states_core_a53[] = { /* Power per cpu */ { .cap = 178, .power = 69, }, /* 208MHz */ { .cap = 369, .power = 124, }, /* 432MHz */ { .cap = 622, .power = 224, }, /* 729MHz */ { .cap = 819, .power = 367, }, /* 960MHz */ { .cap = 1024, .power = 670, }, /* 1.2GHz */ }; Cluster level CPU level P-State C-State Active Idle State EAS power modeling parameters
Overview for IPA power model IPA Thermal sensor driver CPU Cooling Device CPUFreq Framework Get dynamic power 3 Get static power 4 Updated allocated budget 5 Update CPUFreq policy 6 Report temperature, reach trip points 1 Gather actor’s request budget 2 Power Model for CPU actor
Power model based on dynamic power and static leakage Dynamic power CPU static leakage Cluster static leakage SoC temperature impacts static leakage
Power model as a linear equation +power static_power= "capacitance" * (freq * volt^2) Dynamic power Y = m * X + b Can easily calculate best-fit values for m and b using the LINEST function found in most spreadsheet programs.
OPP (MHz) Voltage (v) Cluster Power Off State (mW) Cluster P-state (mW) Cluster Power (mW) F * V^2 208 1.04 344 360 16 225 432 1.04 345 374 29 467 729 1.09 346 393 47 866 960 1.18 352 427 75 1337 1200 1.33 367 479 112 2123 Review cluster power XY
Linear regression on the cluster power Intercept gives static power (~5mW) Gradient gives “capacitance” = 0.051
OPP (MHz) Voltage (v) Cluster P-state (mW) CPU P-state (mW) CPU Dynamic Power (mW) F * V^2 208 1.04 360 429 69 225 432 1.04 374 498 124 467 729 1.09 393 617 224 866 960 1.18 427 794 367 1337 1200 1.33 479 1149 670 2123 Review CPU power XY
Linear regression on the CPU power A negative intercept (static power) makes no sense physically and strongly suggests the line is nonlinear. In the graph we see the highest two OPPs’ power increase sharply Intercept (static power) is negative value -27 Gradient (capacitance) = 0.317
Potential solutions to generate a model ● Three possible options ○ Can't have negative static power because get_static_power() cannot return negative number, but we could force linear regression through zero if we accept that there will be some flaws in the model. ○ Could correct for temperature. It is likely the highest two OPPs start to curve upwards as die temperature rises and static leakage increases. ○ Could modify kernel to remove IPA model and use EAS figures directly. These physical measurements already include a contribution from die temperature... ● We didn’t collect temperature data for Hikey and DT bindings for EAS are not yet agreed so, for now, linear regression through zero is the simplest choice.
Linear regression through zero on the CPU power Using linear regression through zero gives worst case error of ~10% Intercept (static power) is 0 Gradient (capacitance) = 0.298
IPA power coefficients struct cluster_power_coefficients cluster_data = { .dyn_coeff = 311, }; (0.298 + (0.054 / 4 CPUs)) * 1000 = 311 A simplistic power modeling without taking account of static leakage: 1. Given the non-linearity for CPU power data then modeling static power is pointless 2. Updated value for cluster “capacitance” which is now forced through zero as well 3. Error in model can be tolerated in some use cases; dynamic power remains main contributor to power consumption 4. PID controller’s integral term will partly compensate for temperature drift
Decide sustainable power based on OPP Temperature increases very quickly when 8CPUs@1.2GHz, meaning the power consumption is above max sustainable level. Temperature increases slowly when 8CPUs@960MHz, system is only very slightly above sustainable power. When CPU runs at 729MHz the temperature doesn’t increase. Its power consumption is less than the sustainable power. OPP (MHz) Sustainable power 729 2155 960 3326 1200 5285
Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
Conclusions ● IPA was very effective on HiKey even with a poor model ○ Increase of ~40% performance compared to step-wise governor ● Allow EAS model to be used for IPA ○ Allows direct link between observed values to power mode ○ Using EAS values directly is convenient for simple systems although it makes it impossible to explicitly model die temperature effects ● Recommend to record temperature alongside power data ○ If possible enable temperature sensor for SoC before gathering power info ● Other suggestions ○ Use CFS’s CPU utilization for accurate duty cycle and calculate dynamic power
Thanks! Linaro Premium Services includes the delivery of tailored training courses for Linaro Club and Core members. This presentation is an abridgement of material prepared as part our training programme.

More Related Content

PDF
BKK16-208 EAS
Linaro
 
PDF
LCU14-410: How to build an Energy Model for your SoC
Linaro
 
PDF
Process Scheduler and Balancer in Linux Kernel
Haifeng Li
 
PDF
WALT vs PELT : Redux - SFO17-307
Linaro
 
PDF
semaphore & mutex.pdf
Adrian Huang
 
PDF
LAS16-105: Walkthrough of the EAS kernel adaptation to the Android Common Kernel
Linaro
 
PDF
The Linux Kernel Scheduler (For Beginners) - SFO17-421
Linaro
 
PDF
Kernel Recipes 2017 - Understanding the Linux kernel via ftrace - Steven Rostedt
Anne Nicolas
 
BKK16-208 EAS
Linaro
 
LCU14-410: How to build an Energy Model for your SoC
Linaro
 
Process Scheduler and Balancer in Linux Kernel
Haifeng Li
 
WALT vs PELT : Redux - SFO17-307
Linaro
 
semaphore & mutex.pdf
Adrian Huang
 
LAS16-105: Walkthrough of the EAS kernel adaptation to the Android Common Kernel
Linaro
 
The Linux Kernel Scheduler (For Beginners) - SFO17-421
Linaro
 
Kernel Recipes 2017 - Understanding the Linux kernel via ftrace - Steven Rostedt
Anne Nicolas
 

What's hot (20)

PDF
SFO15-302: Energy Aware Scheduling: Progress Update
Linaro
 
PPTX
Linux Initialization Process (2)
shimosawa
 
PDF
Vmlinux: anatomy of bzimage and how x86 64 processor is booted
Adrian Huang
 
PDF
Making Linux do Hard Real-time
National Cheng Kung University
 
PDF
Arm device tree and linux device drivers
Houcheng Lin
 
ODP
The Theory and Implementation of DVFS on Linux
Picker Weng
 
PDF
LCA14: LCA14-306: CPUidle & CPUfreq integration with scheduler
Linaro
 
PDF
LAS16-307: Benchmarking Schedutil in Android
Linaro
 
PPTX
Linux Memory Management with CMA (Contiguous Memory Allocator)
Pankaj Suryawanshi
 
PDF
Browsing Linux Kernel Source
Motaz Saad
 
PDF
malloc & vmalloc in Linux
Adrian Huang
 
PDF
Exploring Your Apple M1 devices with Open Source Tools
Koan-Sin Tan
 
PDF
Linux kernel debugging
libfetion
 
PDF
Scheduling in Android
Opersys inc.
 
PPTX
Slab Allocator in Linux Kernel
Adrian Huang
 
PDF
Boosting I/O Performance with KVM io_uring
ShapeBlue
 
PDF
Physical Memory Models.pdf
Adrian Huang
 
PPTX
Linux Kernel Module - For NLKB
shimosawa
 
PDF
Linux Systems Performance 2016
Brendan Gregg
 
PPTX
Ceph Tech Talk -- Ceph Benchmarking Tool
Ceph Community
 
SFO15-302: Energy Aware Scheduling: Progress Update
Linaro
 
Linux Initialization Process (2)
shimosawa
 
Vmlinux: anatomy of bzimage and how x86 64 processor is booted
Adrian Huang
 
Making Linux do Hard Real-time
National Cheng Kung University
 
Arm device tree and linux device drivers
Houcheng Lin
 
The Theory and Implementation of DVFS on Linux
Picker Weng
 
LCA14: LCA14-306: CPUidle & CPUfreq integration with scheduler
Linaro
 
LAS16-307: Benchmarking Schedutil in Android
Linaro
 
Linux Memory Management with CMA (Contiguous Memory Allocator)
Pankaj Suryawanshi
 
Browsing Linux Kernel Source
Motaz Saad
 
malloc & vmalloc in Linux
Adrian Huang
 
Exploring Your Apple M1 devices with Open Source Tools
Koan-Sin Tan
 
Linux kernel debugging
libfetion
 
Scheduling in Android
Opersys inc.
 
Slab Allocator in Linux Kernel
Adrian Huang
 
Boosting I/O Performance with KVM io_uring
ShapeBlue
 
Physical Memory Models.pdf
Adrian Huang
 
Linux Kernel Module - For NLKB
shimosawa
 
Linux Systems Performance 2016
Brendan Gregg
 
Ceph Tech Talk -- Ceph Benchmarking Tool
Ceph Community
 
Ad

Similar to BKK16-317 How to generate power models for EAS and IPA (20)

PPTX
Computer Architecture and Organization
ssuserdfc773
 
PPTX
How to achieve 95%+ Accurate power measurement during architecture exploration?
Deepak Shankar
 
PDF
Q4.11: Sched_mc on dual / quad cores
Linaro
 
PDF
Linux_swspnd_v0.3_pub1
padmanabha srinivasaiah
 
PDF
Where Did My Cpu Go?
Enkitec
 
PPTX
The Hidden Face of Cost-Based Optimizer: PL/SQL Specific Statistics
Michael Rosenblum
 
PDF
在Aix6.1上安装11g r2 rac grid infrastructure集群
maclean liu
 
PDF
LCA13: CPUIDLE: One driver to rule them all?
Linaro
 
PDF
ISCA2021 Tutorial-Methods for Characterization and Analysis of Voltage Margin...
Behzad Salami
 
PDF
PowerManagement
Ayan Halder
 
PDF
Balancing Power & Performance Webinar
Qualcomm Developer Network
 
PPT
Vpu technology &gpgpu computing
Arka Ghosh
 
PPT
Vpu technology &gpgpu computing
Arka Ghosh
 
PPT
Vpu technology &gpgpu computing
Arka Ghosh
 
PDF
BKK16-311 EAS Upstream Stategy
Linaro
 
PDF
IT Essentials (Version 7.0) - ITE Practice Final Exam Answers
ITExamAnswers.net
 
PPT
Altera trcak g
Alona Gradman
 
PPTX
Six-Core AMD Opteron EE Processor
AMD
 
PDF
Project ACRN schedule framework introduction
Project ACRN
 
PDF
20171206 PGconf.ASIA LT gstore_fdw
Kohei KaiGai
 
Computer Architecture and Organization
ssuserdfc773
 
How to achieve 95%+ Accurate power measurement during architecture exploration?
Deepak Shankar
 
Q4.11: Sched_mc on dual / quad cores
Linaro
 
Linux_swspnd_v0.3_pub1
padmanabha srinivasaiah
 
Where Did My Cpu Go?
Enkitec
 
The Hidden Face of Cost-Based Optimizer: PL/SQL Specific Statistics
Michael Rosenblum
 
在Aix6.1上安装11g r2 rac grid infrastructure集群
maclean liu
 
LCA13: CPUIDLE: One driver to rule them all?
Linaro
 
ISCA2021 Tutorial-Methods for Characterization and Analysis of Voltage Margin...
Behzad Salami
 
PowerManagement
Ayan Halder
 
Balancing Power & Performance Webinar
Qualcomm Developer Network
 
Vpu technology &gpgpu computing
Arka Ghosh
 
Vpu technology &gpgpu computing
Arka Ghosh
 
Vpu technology &gpgpu computing
Arka Ghosh
 
BKK16-311 EAS Upstream Stategy
Linaro
 
IT Essentials (Version 7.0) - ITE Practice Final Exam Answers
ITExamAnswers.net
 
Altera trcak g
Alona Gradman
 
Six-Core AMD Opteron EE Processor
AMD
 
Project ACRN schedule framework introduction
Project ACRN
 
20171206 PGconf.ASIA LT gstore_fdw
Kohei KaiGai
 
Ad

More from Linaro (20)

PDF
Deep Learning Neural Network Acceleration at the Edge - Andrea Gallo
Linaro
 
PDF
Arm Architecture HPC Workshop Santa Clara 2018 - Kanta Vekaria
Linaro
 
PDF
Huawei’s requirements for the ARM based HPC solution readiness - Joshua Mora
Linaro
 
PDF
Bud17 113: distribution ci using qemu and open qa
Linaro
 
PDF
OpenHPC Automation with Ansible - Renato Golin - Linaro Arm HPC Workshop 2018
Linaro
 
PDF
HPC network stack on ARM - Linaro HPC Workshop 2018
Linaro
 
PDF
It just keeps getting better - SUSE enablement for Arm - Linaro HPC Workshop ...
Linaro
 
PDF
Intelligent Interconnect Architecture to Enable Next Generation HPC - Linaro ...
Linaro
 
PDF
Yutaka Ishikawa - Post-K and Arm HPC Ecosystem - Linaro Arm HPC Workshop Sant...
Linaro
 
PDF
Andrew J Younge - Vanguard Astra - Petascale Arm Platform for U.S. DOE/ASC Su...
Linaro
 
PDF
HKG18-501 - EAS on Common Kernel 4.14 and getting (much) closer to mainline
Linaro
 
PDF
HKG18-100K1 - George Grey: Opening Keynote
Linaro
 
PDF
HKG18-318 - OpenAMP Workshop
Linaro
 
PDF
HKG18-501 - EAS on Common Kernel 4.14 and getting (much) closer to mainline
Linaro
 
PDF
HKG18-315 - Why the ecosystem is a wonderful thing, warts and all
Linaro
 
PDF
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
Linaro
 
PDF
HKG18-TR08 - Upstreaming SVE in QEMU
Linaro
 
PDF
HKG18-113- Secure Data Path work with i.MX8M
Linaro
 
PPTX
HKG18-120 - Devicetree Schema Documentation and Validation
Linaro
 
PPTX
HKG18-223 - Trusted FirmwareM: Trusted boot
Linaro
 
Deep Learning Neural Network Acceleration at the Edge - Andrea Gallo
Linaro
 
Arm Architecture HPC Workshop Santa Clara 2018 - Kanta Vekaria
Linaro
 
Huawei’s requirements for the ARM based HPC solution readiness - Joshua Mora
Linaro
 
Bud17 113: distribution ci using qemu and open qa
Linaro
 
OpenHPC Automation with Ansible - Renato Golin - Linaro Arm HPC Workshop 2018
Linaro
 
HPC network stack on ARM - Linaro HPC Workshop 2018
Linaro
 
It just keeps getting better - SUSE enablement for Arm - Linaro HPC Workshop ...
Linaro
 
Intelligent Interconnect Architecture to Enable Next Generation HPC - Linaro ...
Linaro
 
Yutaka Ishikawa - Post-K and Arm HPC Ecosystem - Linaro Arm HPC Workshop Sant...
Linaro
 
Andrew J Younge - Vanguard Astra - Petascale Arm Platform for U.S. DOE/ASC Su...
Linaro
 
HKG18-501 - EAS on Common Kernel 4.14 and getting (much) closer to mainline
Linaro
 
HKG18-100K1 - George Grey: Opening Keynote
Linaro
 
HKG18-318 - OpenAMP Workshop
Linaro
 
HKG18-501 - EAS on Common Kernel 4.14 and getting (much) closer to mainline
Linaro
 
HKG18-315 - Why the ecosystem is a wonderful thing, warts and all
Linaro
 
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
Linaro
 
HKG18-TR08 - Upstreaming SVE in QEMU
Linaro
 
HKG18-113- Secure Data Path work with i.MX8M
Linaro
 
HKG18-120 - Devicetree Schema Documentation and Validation
Linaro
 
HKG18-223 - Trusted FirmwareM: Trusted boot
Linaro
 

Recently uploaded (20)

PDF
1 - Historical Antecedents, Social Consideration.pdf
tuazon2030627
 
PPT
Module 1.ppt Iot fundamentals and Architecture
nanditha7766
 
PDF
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
TrustArc
 
PDF
CloudStack 4.21: First Look Webinar slides
ShapeBlue
 
PPT
What is a Computer? Input Devices /output devices
GulJan8
 
PDF
August Patch Tuesday
Ivanti
 
PDF
A contest of sentiment analysis: k-nearest neighbor versus neural network
IAESIJAI
 
PDF
Getting started with AI Agents and Multi-Agent Systems
Maxim Salnikov
 
PPTX
Modernising the Digital Integration Hub
Daniel Toomey
 
PPTX
Final SEM Unit 1 for mit wpu at pune .pptx
anishtilekar08
 
PDF
A review of recent deep learning applications in wood surface defect identifi...
IAESIJAI
 
PPTX
The various Industrial Revolutions .pptx
bandileshozi3
 
PDF
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDCA Consulting
 
PPTX
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
SilvioHayashi
 
PPTX
Web Crawler for Trend Tracking Gen Z Insights.pptx
Actowiz Solustions
 
PDF
Zenith AI: Advanced Artificial Intelligence
Biz Preneurs
 
PDF
DP Operators-handbook-extract for the Mautical Institute
LeonelMejiaLarios
 
PDF
DASA ADMISSION 2024_FirstRound_FirstRank_LastRank.pdf
rameswartudu05
 
PDF
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
Mindy Bohannon
 
PDF
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
IAESIJAI
 
1 - Historical Antecedents, Social Consideration.pdf
tuazon2030627
 
Module 1.ppt Iot fundamentals and Architecture
nanditha7766
 
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
TrustArc
 
CloudStack 4.21: First Look Webinar slides
ShapeBlue
 
What is a Computer? Input Devices /output devices
GulJan8
 
August Patch Tuesday
Ivanti
 
A contest of sentiment analysis: k-nearest neighbor versus neural network
IAESIJAI
 
Getting started with AI Agents and Multi-Agent Systems
Maxim Salnikov
 
Modernising the Digital Integration Hub
Daniel Toomey
 
Final SEM Unit 1 for mit wpu at pune .pptx
anishtilekar08
 
A review of recent deep learning applications in wood surface defect identifi...
IAESIJAI
 
The various Industrial Revolutions .pptx
bandileshozi3
 
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDCA Consulting
 
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
SilvioHayashi
 
Web Crawler for Trend Tracking Gen Z Insights.pptx
Actowiz Solustions
 
Zenith AI: Advanced Artificial Intelligence
Biz Preneurs
 
DP Operators-handbook-extract for the Mautical Institute
LeonelMejiaLarios
 
DASA ADMISSION 2024_FirstRound_FirstRank_LastRank.pdf
rameswartudu05
 
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
Mindy Bohannon
 
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
IAESIJAI
 

BKK16-317 How to generate power models for EAS and IPA

  • 1. Presented by Date Event How to generate power models for EAS and IPA … without talking to a hardware engineerLeo Yan Daniel Thompson BKK16-317 March 9, 2016 Linaro Connect BKK16
  • 2. Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
  • 3. Background ● Generating platform-specific parameters for power models are a prerequisite for deploying EAS and IPA ○ Software engineers need these parameters to work on these features ○ Gaining assistance from hardware engineers is difficult and, for engineers who don’t work for SoC vendors, impossible ○ It is possible to generate an acceptable (though perhaps not optimal) model using simple tools ● Today we will focus on CPU and cluster level power modeling ○ Excludes SoC level power modeling (GPU, DDR, ...)
  • 4. The role of power modeling Power Modeling EAS/IPA Profiling Generate power modeling 1 Enable EAS/IPA 2 Enable profiling environment 3 Check if use bad power modeling 4 Refine algorithm 5 Refine power modeling 6
  • 5. Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
  • 6. CPU power states CPU State PD_CPUx CPU P-State On WFI On, internal clock gating CPU Off Off
  • 7. CPU P-State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU0 Powered Off Powered On Clock Gating
  • 8. CPU WFI State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0: WFI SCU Cluster 0 CLKIN PDCPU0 Powered Off Powered On Clock Gating
  • 9. CPU Off State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU0 Powered Off Powered On Clock Gating X X
  • 10. Cluster power states Cluster State PD_CPUx PDCORTEXA53 PD_L2 Cluster P-State On or Off On On Cluster L2 Retention Off Off Retention Cluster Off Off Off Off Cannot keep cache coherent between two clusters
  • 11. Cluster P-State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU<n> Powered Off Powered On Clock Gating PDCORTEXTA53 PDL2 X X
  • 12. Cluster L2 Retention State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU<n> Powered Off Powered On Clock Gating PDCORTEXTA53 PDL2 X X X
  • 13. Cluster Off State ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0 ARM CPU0 SCU Cluster 0 CLKIN PDCPU<n> Powered Off Powered On Clock Gating PDCORTEXTA53 PDL2 X X X X
  • 14. Link power states in Linux Kernel Power State Linux Kernel CPU P-state P-state WFI C-state CPU Off C-state Cluster P-state P-state Cluster L2 Retention C-state Cluster Off C-state CPUFreq does not distinguish “CPU P-state” and “Cluster P- state” status as usually binding CPU and Cluster frequency from hardware design. But EAS needs to model these two states separately.
  • 15. Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
  • 16. Hikey PMIC Hierarchy SoC: Hi6220 DC IN TO 4.2VDC, 4A PMIC: Hi6553 12V VDD_4V2 LX0 LX1 BUCK1_1V05 Ideal measure point, but hardware design does not encourage measurement here Select CPU’s nearest measurement point BUCKs & LDOs ACPU R247R7
  • 17. Connect with ARM energy probe DC IN TO 4.2 VDC, 4A PMIC: Hi6553 VDD_4V2 0.033Ω ARM energy probe Hikey Board PC: Linux V+ V-Ground USB
  • 18. Methodology For Power Measurement ● Use single CPU to measure power ○ All other CPUs will be hot unplugged (CPU off state) ○ CPU0 is left connected ● Use ARM Trusted Firmware to perform system suspend ○ Force cluster and CPU0 to enter specific C-states and P-states ● Measure power data based on different operating points ○ Each OPP implies specific pair of voltage and frequency ○ Measure all C-states and P-states at given OPP ○ Use “dhrystone” to generate 100% CPU utilization for CPU P-state
  • 19. hikey_afflvl_off Linux kernel ARM CPUARM CPU0ARM CPU0ARM CPU0 Setup power state ARM CPUARM CPU0ARM CPU1 CPU1..7 echo 0 > /sys/devices/system/cpu/cpu[1..7]/online psci_cpu_off ARM TF ARM CPUARM CPU0 ARM CPU0 ARM CPU4 echo mem > /sys/power/state ARM CPU0 psci_system_suspend_enter Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC ARM CPUARM CPU0 ARM CPU0 ARM CPU4 Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC ARM CPUARM CPU0 ARM CPU0 ARM CPU4 Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC Hack low level code to only power off CPU but leave cluster on 1 2 ARM CPUARM CPU0 ARM CPU0 ARM CPU4 Cluster 1 ARM CPUARM CPU0 ARM CPU0 Cluster 0 ARM CPU0 SoC hisi_ipc_cpu_suspend hisi_ipc_psci_system_off hisi_ipc_cluster_suspendhikey_affinst_suspend
  • 20. Cluster and CPU’s Power Data OPP (MHz) Voltage (v) Cluster Power Off State (mW) Cluster P-State (mW) CPU WFI State (mW) CPU P-State (mW) 208 1.04 344 360 379 429 432 1.04 345 374 387 498 729 1.09 346 393 408 617 960 1.18 352 427 443 794 1200 1.33 367 479 508 1149 Difference is caused by other components in SoC that share the same regulator “Cluster P-state” means all CPUs in cluster have been powered off but cluster is powered on
  • 21. Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
  • 22. EAS energy model CPU0 CPU1 Cluster Time C-state P-state ● Formula: energy = energy_cluster + ∑ energy_CPU in cluster ● C-states: Idle energy, include cpu level and cluster level ● P-states: Busy energy, include cpu level and cluster level
  • 23. Normalized CPU capacity ● Normalized CPU capacity is a ratio value ○ Ratio between actual performance of CPU and the maximum performance of CPU in the system ○ Normalized into range [0..1024] ● Normally use “dhrystone” to measure CPU capacity ○ Safe to derive from CPU frequency if all CPUs are homogeneous static struct capacity_state cap_states_core_a53[] = { /* Power per cpu */ { .cap = 178, .power = 69, }, /* 208MHz */ { .cap = 369, .power = 124, }, /* 432MHz */ { .cap = 622, .power = 224, }, /* 729MHz */ { .cap = 819, .power = 367, }, /* 960MHz */ { .cap = 1024, .power = 670, }, /* 1.2GHz */ };
  • 24. Basic method to calculate for power data ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU0 OPP: 729 MHz SCU Cluster 0 ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU0 SCU Cluster 0 ARM CPU0 OPP: 729 MHz Remove cluster level power consumption CPU level power consumption
  • 25. Power of CPU C-State OPP (MHz) Cluster Power On State (mW) CPU WFI (mW) Power of CPU WFI (mW) 208 360 379 19 432 374 387 13 729 393 408 15 960 427 443 16 1200 479 508 31 static struct idle_state idle_states_core_a53[] = { { .power = 15 }, /* active idle state */ { .power = 15 }, /* WFI state */ { .power = 0 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; Select 15mW as middle value for WFI state’s power data
  • 26. Power of CPU P-State OPP (MHz) Cluster P-State (mW) CPU P-State (mW) Power of CPU P-state (mW) 208 360 429 69 432 374 498 124 729 393 617 224 960 427 794 367 1200 479 1149 670 static struct capacity_state cap_states_core_a53[] = { /* Power per cpu */ { .cap = 178, .power = 69, }, /* 208MHz */ { .cap = 369, .power = 124, }, /* 432MHz */ { .cap = 622, .power = 224, }, /* 729MHz */ { .cap = 819, .power = 367, }, /* 960MHz */ { .cap = 1024, .power = 670, }, /* 1.2GHz */ }; CPU P-state is calculated with static leakage and dynamic power, so should use Power (CPU P-state) - Power (Cluster P-state)
  • 27. Cluster’s active idle state ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU: Off SCU Cluster 0 Powered Off Powered On Clock Gating ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU: WFI SCU Cluster 0 ARM CPUARM CPU0 ACP ACE L2 RAM ARM CPU0ARM CPU: WFI or Off SCU Cluster 0 At CPU level, all CPUs in the cluster are in one of the idle states (WFI or Off state) and the cluster is not processing any software. At cluster level, the cluster remains active. Is Active idle state a C-state or a P-state? The main difference is if we need consider it with different OPP...
  • 28. Power of Cluster C-State OPP (MHz) Cluster Power Off State (mW) Cluster Power On State (mW) Power of cluster level active idle state (mW) 208 344 360 16 432 345 374 29 729 346 393 47 960 352 427 75 1200 367 479 112 static struct idle_state idle_states_cluster_a53[] = { { .power = 47 }, /* active idle state */ { .power = 47 }, /* WFI state */ { .power = 47 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; Only in cluster power off state will we totally power off whole cluster, in other idle states the cluster actually is powered on even when all CPUs are powered off.
  • 29. Power of Cluster P-State OPP (MHz) Cluster Power Off State (mW) Cluster Power On State (mW) Power of cluster level active idle state (mW) 208 344 360 16 432 345 374 29 729 346 393 47 960 352 427 75 1200 367 479 112 static struct capacity_state cap_states_cluster_a53[] = { /* Power per cluster */ { .cap = 178, .power = 16, }, { .cap = 369, .power = 29, }, { .cap = 622, .power = 47, }, { .cap = 819, .power = 75, }, { .cap = 1024, .power = 112, }, };
  • 30. static struct idle_state idle_states_cluster_a53[] = { { .power = 47 }, /* active idle state */ { .power = 47 }, /* WFI state */ { .power = 47 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; static struct capacity_state cap_states_cluster_a53[] = { /* Power per cluster */ { .cap = 178, .power = 16, }, { .cap = 369, .power = 29, }, { .cap = 622, .power = 47, }, { .cap = 819, .power = 75, }, { .cap = 1024, .power = 112, }, }; static struct idle_state idle_states_core_a53[] = { { .power = 15 }, /* active idle state */ { .power = 15 }, /* WFI state */ { .power = 0 }, /* CPU Off state */ { .power = 0 }, /* Cluster Off state */ }; static struct capacity_state cap_states_core_a53[] = { /* Power per cpu */ { .cap = 178, .power = 69, }, /* 208MHz */ { .cap = 369, .power = 124, }, /* 432MHz */ { .cap = 622, .power = 224, }, /* 729MHz */ { .cap = 819, .power = 367, }, /* 960MHz */ { .cap = 1024, .power = 670, }, /* 1.2GHz */ }; Cluster level CPU level P-State C-State Active Idle State EAS power modeling parameters
  • 31. Overview for IPA power model IPA Thermal sensor driver CPU Cooling Device CPUFreq Framework Get dynamic power 3 Get static power 4 Updated allocated budget 5 Update CPUFreq policy 6 Report temperature, reach trip points 1 Gather actor’s request budget 2 Power Model for CPU actor
  • 32. Power model based on dynamic power and static leakage Dynamic power CPU static leakage Cluster static leakage SoC temperature impacts static leakage
  • 33. Power model as a linear equation +power static_power= "capacitance" * (freq * volt^2) Dynamic power Y = m * X + b Can easily calculate best-fit values for m and b using the LINEST function found in most spreadsheet programs.
  • 34. OPP (MHz) Voltage (v) Cluster Power Off State (mW) Cluster P-state (mW) Cluster Power (mW) F * V^2 208 1.04 344 360 16 225 432 1.04 345 374 29 467 729 1.09 346 393 47 866 960 1.18 352 427 75 1337 1200 1.33 367 479 112 2123 Review cluster power XY
  • 35. Linear regression on the cluster power Intercept gives static power (~5mW) Gradient gives “capacitance” = 0.051
  • 36. OPP (MHz) Voltage (v) Cluster P-state (mW) CPU P-state (mW) CPU Dynamic Power (mW) F * V^2 208 1.04 360 429 69 225 432 1.04 374 498 124 467 729 1.09 393 617 224 866 960 1.18 427 794 367 1337 1200 1.33 479 1149 670 2123 Review CPU power XY
  • 37. Linear regression on the CPU power A negative intercept (static power) makes no sense physically and strongly suggests the line is nonlinear. In the graph we see the highest two OPPs’ power increase sharply Intercept (static power) is negative value -27 Gradient (capacitance) = 0.317
  • 38. Potential solutions to generate a model ● Three possible options ○ Can't have negative static power because get_static_power() cannot return negative number, but we could force linear regression through zero if we accept that there will be some flaws in the model. ○ Could correct for temperature. It is likely the highest two OPPs start to curve upwards as die temperature rises and static leakage increases. ○ Could modify kernel to remove IPA model and use EAS figures directly. These physical measurements already include a contribution from die temperature... ● We didn’t collect temperature data for Hikey and DT bindings for EAS are not yet agreed so, for now, linear regression through zero is the simplest choice.
  • 39. Linear regression through zero on the CPU power Using linear regression through zero gives worst case error of ~10% Intercept (static power) is 0 Gradient (capacitance) = 0.298
  • 40. IPA power coefficients struct cluster_power_coefficients cluster_data = { .dyn_coeff = 311, }; (0.298 + (0.054 / 4 CPUs)) * 1000 = 311 A simplistic power modeling without taking account of static leakage: 1. Given the non-linearity for CPU power data then modeling static power is pointless 2. Updated value for cluster “capacitance” which is now forced through zero as well 3. Error in model can be tolerated in some use cases; dynamic power remains main contributor to power consumption 4. PID controller’s integral term will partly compensate for temperature drift
  • 41. Decide sustainable power based on OPP Temperature increases very quickly when [email protected], meaning the power consumption is above max sustainable level. Temperature increases slowly when 8CPUs@960MHz, system is only very slightly above sustainable power. When CPU runs at 729MHz the temperature doesn’t increase. Its power consumption is less than the sustainable power. OPP (MHz) Sustainable power 729 2155 960 3326 1200 5285
  • 42. Agenda ● Background ● Low power states ● Measurement on Hikey ● Generate power model ● Conclusions
  • 43. Conclusions ● IPA was very effective on HiKey even with a poor model ○ Increase of ~40% performance compared to step-wise governor ● Allow EAS model to be used for IPA ○ Allows direct link between observed values to power mode ○ Using EAS values directly is convenient for simple systems although it makes it impossible to explicitly model die temperature effects ● Recommend to record temperature alongside power data ○ If possible enable temperature sensor for SoC before gathering power info ● Other suggestions ○ Use CFS’s CPU utilization for accurate duty cycle and calculate dynamic power
  • 44. Thanks! Linaro Premium Services includes the delivery of tailored training courses for Linaro Club and Core members. This presentation is an abridgement of material prepared as part our training programme.