Uploaded byEndgameInc
PDF, PPTX1,236 views

Hardware-Assisted Rootkits & Instrumentation

The document presents insights from Matt Spisak's talk at Recon 2016 concerning ARM architecture, focusing on hardware-assisted rootkits and tools for debugging embedded systems. It discusses the importance of performance monitoring units (PMUs) in diagnosing errors and captures various features of ARM's debug architecture, including invasive and non-invasive debugging techniques. The talk emphasizes software-based approaches over hardware for efficient security practices and malware research in the context of mobile and embedded systems.

Embed presentation

Download as PDF, PPTX
ARM EDITION HARDWARE-ASSISTED ROOTKITS & INSTRUMENTATION: Matt Spisak REcon 2016, Montreal
RECON 2016 ▸ Offense-based approach to security and hunting adversaries ▸ Research thrusts in malware, threat intel, data science, and exploit prevention ▸ Matt Spisak (@matspisak) ▸ Vulnerability and exploit mitigation research at Endgame ▸ Mobile security since Nokia N series (before iPhone) ABOUT
RECON 2016 OUTLINE ▸ Motivation ▸ ARM Debug Architecture ▸ Tracing and Instrumentation ▸ Rootkits ▸ TrustZone ▸ Exploit Mitigations
RECON 2016MOTIVATION DEBUGGING EMBEDDED SYSTEMS IS COMPLICATED Hardware ‣ JTAG is a gold standard ‣ Custom dev boards + Virtualization extensions ‣ JTAG access can be hit/miss ‣ Destructive ‣ Expensive Software ‣ Portable, scalable ‣ existing tools for HLOS like iOS, Android ‣ Can be tightly coupled to OS ‣ Often limited to PL0/EL0 ‣ Lots of reinventing wheel Emulation ‣ Scalable and powerful ‣ Cost-effective ‣ Sometimes a good option (e.g. CTF) ‣ Lack support for HW interfaces ‣ Requires big time investment
RECON 2016MOTIVATION SEARCHING FOR ALTERNATIVES ▸ Whats a good general approach? ▸ Personal philosophy: ▸ Always make use of real hardware ▸ Lean towards software-based tools ▸ GOAL: find common ARM architectural debug features accessible from software (on COTS devices)
ARM DEBUG ARCHITECTURE
RECON 2016ARM DEBUG ARCHITECTURE INVASIVE DEBUG ▸ Debug-modes: Monitor, Halting, or None ▸ Software debug events: BKPT, breakpoint, watchpoint, vector trap ▸ Halting debug events result in processor entering debug state ▸ Support driven by DBGEN and SPIDEN authentication signals ▸ if DBGEN is low —> BKPT instruction only event supported ▸ Authentication signals typically controlled externally ▸ Without DBGEN, options are limited
RECON 2016ARM DEBUG ARCHITECTURE NON-INVASIVE DEBUG ▸ Trace: Embedded Trace Buffer (ETB) / CoreSight Program Flow Trace (PFT) ▸ PFT/PTM generates traces for waypoints: branch & exception instructions ▸ Accessible from external and software (coprocessor or memory-mapped) ▸ PFT/PTM can be locked (ETMLAR) - only writeable in memory-mapped ▸ memory-mapped access is IMPLEMENTATION DEFINED ▸ Trace drivers in Android kernel check CoreSight fuse status ▸ A potential software-based debug feature for COTS devices
RECON 2016ARM DEBUG ARCHITECTURE NON-INVASIVE DEBUG ▸ Sample-based Profiling ▸ Registers for sampling Program Counter and Context ID ▸ No CP14 visibility, optional memory-mapped and external interfaces ▸ PMU ▸ Focus of remainder of talk
NOT THIS PMU.
21 3THIS PMU. performance counters
RECON 2016ARM DEBUG ARCHITECTURE PERFORMANCE MONITORING UNIT (PMU) ▸ Optional extension, but recommended ▸ Interfaces: CP15 (mandatory), memory-mapped (optional), external (optional) ▸ Dates back to ARMv6, common in ARM11, Cortex-R, Cortex-A ▸ 1 cycle counter, up to 31 general counters ▸ Set of event filters for counting ▸ Support for interrupts on counter overflow 
 sampling period
RECON 2016ARM DEBUG ARCHITECTURE PERFORMANCE MONITORING UNIT (PMU) ▸ Provides real-time feedback on system ▸ Useful for software/hardware engineers ▸ Diagnose bugs ▸ Tools: ▸ ARM DS-5 Streamline ▸ Linux perf / oprofile ARM DS-5 Streamline
RECON 2016ARM DEBUG ARCHITECTURE TERMINOLOGY & ABBREVIATIONS ▸ PMU - Performance Monitoring Unit ▸ PMI - Performance Monitoring Interrupt ▸ PMC - Performance Monitoring Counter Least Privileged Most Privileged USER MODE KERNEL MODE HYPERVISOR SECURE MONITORPL3/EL3 PL2/EL2 PL1/EL1 PL0/EL0 ARM x86 Ring 3 Ring 0 Ring -1 Ring -2 EXCEPTION Reset Undefined Instruction SVC Supervisor Call (e.g. SYSCALL) Prefetch Abort BKPT, or code Page Fault Data Abort Data Page Fault IRQ Interrupts (Normal World) FIQ Fast Interrupts (Secure World) ARM Exception Vector Table (EVT)
RECON 2016ARM DEBUG ARCHITECTURE PMU RELATED WORK ▸ “Using Hardware Performance Events for Instruction-Level Monitoring on the x86 Architecture”, [Vogl, Eckert] ▸ ROP detection with PMU using mispredicted RET [Wicherski], [Li, Crouse] ▸ Rootkit detection with performance counters [Wang, Karri] ▸ Control-flow integrity using BTS [Xia et al] ▸ Control-flow integrity using PMU [Endgame] - BlackHat USA 2016 ▸ All prior art is focused on Intel / x86 architecture
RECON 2016ARM DEBUG ARCHITECTURE SAMPLE ARM PMU EVENTS EVENT TYPE EVENT CODE LD_RETIRED: Load instruction executed 0x06 ST_RETIRED: Store instruction executed 0x07 INST_RETIRED: Instruction executed 0x08 PC_WRITE_RETIRED: Software change of PC 0x0C BR_RETURN_RETIRED: Branch Return retired 0x0E BR_MISP_PRED: Branch mispredicted 0x10 L1I_CACHE: Level 1 instruction cache access 0x14
RECON 2016ARM DEBUG ARCHITECTURE PMU REGISTERS ▸ PMCR - Control Register ▸ N: Number of counters ▸ E: Enable / Disable all counters ▸ ARMv6: MRC/MCR p15, 0, <Rd>, c15, c12, 0 ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 0
RECON 2016ARM DEBUG ARCHITECTURE PMU REGISTERS - CONFIGURE COUNTERS ▸ PMCNTENSET - Enable Counter ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 1 ▸ PMCNTENCLR - Disable Counter ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 2 ▸ PMSELR - Counter Selection Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 5 Use this register prior to read/write
 of event type or counter registers
RECON 2016ARM DEBUG ARCHITECTURE PMU REGISTERS - CONFIGURE COUNTERS ▸ PMXEVTYPER - Counter Event Filter Register ▸ Selects event and modes to count ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c13, 1 ▸ PMXEVTCNTR - Event Counter Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c13, 2 EVENT CODE MODES INCLUDED 0x6800000C Branches in Secure PL1 and HYP 0x6000000C Branches in Secure PL1 0x9800000C Branches in Secure PL0 and HYP 0x9000000C Branches in Secure PL0 0x3800000C Branches in Secure PL0,PL1,HYP 0x4000000C Branches in non-secure PL1 0x8000000C Branches in non-secure PL0
RECON 2016ARM DEBUG ARCHITECTURE PMU REGISTERS - CONFIGURE COUNTERS //Enable armv7 PMU Counters
 MRC p15, 0, R1, c9, c12, 0
 ORR R1, R1, #1
 MCR p15, 0, R1, c9, c12, 0 //Set PMC1 to count Instructions Executed
 MOV R1, #1
 MCR p15, 0, R1, c9, c12, 5 //PMSELR
 MOV R1, #0x8
 MCR p15, 0, R1, c9, c13, 1 //PMXEVTYPER //Initialize PMC1 to -3
 MOV R1, #0xFFFFFFFD
 MCR p15, 0, R1, c9, c13, 2 //PMXEVTCNTR //Enable PMC1
 MOV R1, #1
 MCR p15, 0, R1, c9, c12, 1 //PMCNTENSET
RECON 2016ARM DEBUG ARCHITECTURE PMU REGISTERS - CONFIGURE INTERRUPTS ▸ PMINTENSET - Interrupt Enable Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c14, 1 ▸ PMINTENCLR - Interrupt Disable Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c14, 2 ▸ PMOVSR - Overflow Status Register
 PMOVSET - Overflow Status Set Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 3 ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c14, 3
RECON 2016ARM DEBUG ARCHITECTURE PMU REGISTERS - CONFIGURE INTERRUPTS //Enable Interrupts for PMC1 and PMC2
 MOV R1, #3
 MCR p15, 0, R1, c9, c14, 1 //PMINTENSET //Read and Clear Overflow on Interrupt
 MRC p15, 0, R0, c9, c12, 3 //PMOVSR
 MCR p15, 0, R0, c9, c12, 3 //PMOVSR
RECON 2016ARM DEBUG ARCHITECTURE DO YOU EVEN COUNT? ▸ DBGAUTHSTATUS ▸ Lists whether invasive/non-invasive debug are supported in secure and non-secure worlds ▸ ARMv7: MRC/MCR p14, 0, <Rd>, c7, c14, 6 ▸ ID_DFR0 ▸ Lists PMU version supported (if any) ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c0, c1, 2
RECON 2016ARM DEBUG ARCHITECTURE THE CENTER FOR CHIPS WHO CAN COUNT GOOD
CASE STUDY: PMU TRACING
RECON 2016CASE STUDY: PMU TRACING APPROACH ▸ Make the PMU more invasive with frequent PMC-based traps ▸ CoreSight Program Flow Trace (PFT) captures waypoints (i.e. branches) ▸ We can come pretty close to PFT Trace using the PMU: ▸ Count all branches: predicted and mispredicted ▸ Interrupt all the things: set our counter(s) to -1 ▸ Use our ISR as the instrumentation logic BX BL B
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING func: error: -1 BL func
 
 STMFD SP!, {R0-R2,R4-R9,LR}
 MOV R8, R1
 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches)
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 MOV R8, R1
 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING func: error: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 -1 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT
 - RESET COUNTER func: error: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 -1 BEQ error
 0 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC overflow
RECON 2016CASE STUDY: PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT
 - RESET COUNTER func: error: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 -1 BEQ error
 0 MOV R4, #0xFFFFFFF7
 -1 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
RECON 2016CASE STUDY: PMU TRACING BUT WHAT ABOUT LINUX PERF? ▸ We want a custom ISR for instrumentation ▸ Too tightly coupled to Linux ▸ Invoking API’s != learning ▸ But perf source can be useful for understanding PMU interfaces
WHERE’S THE PMU
 INTERRUPT?
RECON 2016CASE STUDY: PMU TRACING ARM GENERIC INTERRUPT CONTROLLER (GIC) SPECIFICATION ▸ SGI: Software Generated Interrupts
 PPI: Private Peripheral Interrupts
 SPI: Shared Peripheral Interrupts ▸ ARM GIC spec recommends PMU Overflows to use INTID 23 ARM GIC Architecture Specification
RECON 2016 ▸ Device Tree Source ▸ Brute Force ▸ Register all unused PPI’s & SPI’s, trigger PMIs, diff /proc/interrupts
 cpu-pmu { compatible = "qcom,krait-pmu"; qcom,irq-is-percpu; interrupts = <1 7 0xf00>; }; CASE STUDY: PMU TRACING CHALLENGE: FINDING PMU INTERRUPTS INT# = 16 + 7 = 23 ▸ Implementation: ▸ Android: request_percpu_irq(), request_threaded_irq() ▸ Embedded firmware: patch IRQ vector handler PPI
RECON 2016CASE STUDY: PMU TRACING CHALLENGE: INTERRUPT SHADOW func: error: -1 BL func
 
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches)
RECON 2016CASE STUDY: PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW -1 BL func
 
 0 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC error:
RECON 2016CASE STUDY: PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC error:
RECON 2016CASE STUDY: PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC error:
RECON 2016CASE STUDY: PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW error: -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 1 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC
RECON 2016CASE STUDY: PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW error: PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 1 MOV R4, #0xFFFFFFF7
 1 ADD SP, SP, #0xC
RECON 2016CASE STUDY: PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW error: PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 1 MOV R4, #0xFFFFFFF7
 1 ADD SP, SP, #0xC } Interrupt Shadow
 Skid = 4 Instructions Causes miss of up to 15% covered basic blocks
RECON 2016CASE STUDY: PMU TRACING OTHER CHALLENGES ▸ CPU Hot-Plugging — easy solution for Android: register_hotcpu_notifier() ▸ Lack of Last Branch Recording feature on ARM ▸ Complicated kernel mode instrumentation: use sampling period of -2
 ▸ Requires small patch to entry-armv.S (or hot patch) PMU ISR 2) RET from ISR causes overflow 3) PMI Infinite Interrupt Loop __IRQ_SVC Set PMC1 = -2Set PMC1 = -1 Sampling Period: 0xFFFFFFFE (-2) 2) RET from IRQ vector increments PMC1 Next branch triggers overflow Sampling Period: 0xFFFFFFFF (-1) 1) PMI 1) PMI
RECON 2016CASE STUDY: PMU TRACING ANDROID PROTOTYPE PERFMON
 ISR PERFMON
 ISR PERFMON
 ISR … RELAY THREAD RELAY THREAD RELAY THREAD … KERNELUSER SPACE IDA Plugin CORE 1 CORE 2 CORE N … pmutrace.kopmu_server Captures PC at
 time of interrupt,
 buffered per core PC PC PC PC PC PC PC PC PC PC Visualize coverage and control pmu_server to select threads, mode, and start/stop
RECON 2016CASE STUDY: PMU TRACING CONNECTING THE DOTS ▸ Use IDA to our advantage ▸ For each PMU waypoint: ▸ Color/count all instructions in Basic Block ▸ If only 1 xref from basic block: count/color it ▸ If only 1 xref to basic block: count/color it Example of a perfect PMU branch tracing run
RECON 2016CASE STUDY: PMU TRACING CONNECTING THE DOTS ▸ Interrupt shadow ▸ Basic block xref algorithm helps fill in missed blocks ▸ Fuzzing / code coverage will eventually be interrupted in this block ▸ Could improve by adding 2nd counter to count instructions between interrupts Interrupt 
 Shadow Example of PMU trace missing basic block
DEMO: PMU TRACING DEVICE REQUIREMENTS: ‣ ROOTED ‣ CONFIG_MODULES OPTION (NOT AS COMMON) ‣ CONFIG_PREEMPT OPTION (COMMON) ‣ IRQ HANDLER PATCH (PL1/EL1)
RECON 2016CASE STUDY: PMU TRACING ANDROID INSTRUMENTATION. SO WHAT? ▸ Recall approach is hardware-assisted - not tied to a specific OS ▸ Less invasive than BKPT tracing ▸ Supports both user mode and kernel mode instrumentation ▸ Not limited to branch tracing, other potential instrumentation use-cases ▸ And these chips can count too: ▸ Broadcom WiFi; Intel/Infineon, MediaTek + other ARM Cellular Basebands ▸ Apple ARM SoCs ▸ PowerPC, MIPS
CASE STUDY: PMU ROOTKITS
RECON 2016CASE STUDY: PMU ROOTKITS PRIOR ART IN ARM ROOTKITS ▸ Traditional rootkits: modify syscall table or EVT [Phrack Issue 68] ▸ Suterusu performs hot patching of kernel functions [Coppola] ▸ Cloaker toggles SCTLR to move EVT [David et al] ▸ Clock Locking Beats explores using CPU governor for hiding cycles [Thomas] ▸ TrustZone based rootkit [Roth]
RECON 2016CASE STUDY: PMU ROOTKITS INSPIRATION ARM Architecture Manual ARMv7-A&R - Appendix C AH AH AH
 very interesting….

RECON 2016TEXT TRACING ROOTKITS TRUSTZONE HYPERVISOR EXPLOIT
 PREVENTION DEFENSENSTRUMENTATION PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED
RECON 2016CASE STUDY: PMU ROOTKITS QUICK NOTE ON ARM LICENSES ▸ ARM Core License ▸ Use core ARM designs ▸ ARM Architectural license ▸ Enables custom cores provided it implements an ARM instruction set ▸ Examples: Qualcomm Scorpion/Krait/Kryo, Apple A6/A7/etc.
RECON 2016CASE STUDY: PMU ROOTKITS COUNTING THE EXCEPTION VECTOR TABLE EVENT Cortex-A7 Cortex-A53 Cortex-A57 Cortex-A72 Scorpion Krait Kryo Undefined Instruction √ √ √ √ ? SVC √ √ √ √ ? Prefetch Abort √ √ √ √ ? Data Abort √ √ √ √ ? IRQ √ √ √ √ √ √ ? FIQ √ √ √ √ √ √ ? SMC * * √ √ √ √ ? HVC √ √ ? ? ? ARM Design Custom ARM-based Design
RECON 2016CASE STUDY: PMU ROOTKITS DOWN THE RABBIT HOLE ARM Architecture Manual ARMv7-A&R ▸ Chipset vendors with proprietary PMU implementations: ▸ Qualcomm ▸ Apple ▸ Likely others Covered in earlier slides
RECON 2016CASE STUDY: PMU ROOTKITS SCORPION KRAIT KRYO 2008 2012 2015 ARMv7 ARMv7 ARMv8 1-2 Cores 2 or 4 cores 4 cores Snapdragon S1/S2/S3 Snapdragon S4/400/600/800/805 Snapdragon 818/820/823 BlackBerry Bold 9900
 Samsung Galaxy S2 (LTE)
 Nokia Lumia 900
 HTC Droid Incredible Nexus 4/5/6/7
 Samsung Galaxy S4/S5
 HTC One M8
 LG G3 LG G5
 Samsung Galaxy S7
 HTC 10
 Xiomi Mi 5
RECON 2016CASE STUDY: PMU ROOTKITS QUALCOMM KRAIT PMU ▸ Adds 4 event select registers: 1 for Venum VFP, 3 for other components of CPU ▸ Krait event encoded using code + group + region => (code << 8 * group) ▸ ARM event select register (PMXEVTYPER) set to link to Krait region and group Krait Region 0 Krait Region 1 Krait Region 2 MRC/MCR p15, 1, <Rd>, c9, c15, 0 MRC/MCR p15, 1, <Rd>, c9, c15, 1 MRC/MCR p15, 1, <Rd>, c9, c15, 2 Interrupts/Exceptions + other ? ? ~100 event codes ~128 event codes ~156 event codes PMXEVTYPER = 0xCC | group PMXEVTYPER = 0xD0 | group PMXEVTYPER = 0xD4 | group Only a few documented in old Scorpion src. Black-box analysis used to determine # of events
RECON 2016CASE STUDY: PMU ROOTKITS QUALCOMM KRAIT PMU ▸ Configure Krait + ARM PMU to count Prefetch Aborts: ▸ Krait Event Code: 0x0B group: 3 Region: 0 /*Set Krait Region 0 event selection register
 To count Prefetch Aborts*/
 MRC p15, 0, R1, c9, c15, 0
 ORR R1, R1, #0x8b000000
 MCR p15, 0, R1, c9, c15, 0 //Set PMXEVTYPER to point to krait region 0
 MOV R1, #0xCF
 MCR p15, 0, R1, c9, c13, 1

RECON 2016CASE STUDY: PMU ROOTKITS PMU-ASSISTED ROOTKITS ▸ Trap SVC instructions via PMU ▸ Use ISR to filter system calls, and redirect code execution after servicing PMI ▸ Avoids patch protection* ▸ Installation: a few instructions to initialize PMU registers, and then register ISR for PMU interrupts 1 2 3 4
RECON 2016CASE STUDY: PMU ROOTKITS CHALLENGE: DELAYED INSTRUCTION SKID ▸ PMI serviced at some point after IRQs enabled in vector_swi ▸ 3 cases we must deal with: 1. PMI before branch to syscall routine within vector_swi 2. PMI at entry point of syscall routine 3. PMI in middle of syscall routine IRQs enabled
RECON 2016CASE STUDY: PMU ROOTKITS CASE 1: INTERRUPT BEFORE BRANCH TO SYSCALL ROUTINE #define CPSIE_ADDR 0xC01064D0
 …
 irq_regs = get_irq_regs(); //get SVC mode regs
 pregs = task_pt_regs(current); //get user mode regs
 …
 if (pregs->ARM_r7 == 0x3) //sys_read
 { 
 switch (irq_regs->ARM_pc - CPSIE_ADDR) //offset after CPSIE
 {
 //emulate remaining instructions up to LDRCC
 //can skip those involved in resolving syscall routine
 case 0x0:
 case 0x4:
 irq_regs->ARM_r9 = irq_regs->ARM_sp & 0xFFFFE000;
 …
 case 0x14:
 case 0x18:
 case 0x1C:
 case 0x20:
 irq_regs->ARM_lr = ret_fast_syscall;
 case 0x24:
 irq_regs->ARM_pc = (uint32_t)hook_sysread;

RECON 2016CASE STUDY: PMU ROOTKITS CASE 2: SYSCALL ROUTINE ENTRY POINT ▸ Replace saved PC with address of hook 
 irq_regs = get_irq_regs();
 pregs = task_pt_regs(current);
 …
 if (pregs->ARM_r7 == 0x3) //sys_read
 { 
 //Check if PMU interrupted at entry point addr of sys_read 
 if (pregs->ARM_pc == orig_sys_read)
 {
 pregs->ARM_pc = (uint32_t)hook_sys_read; 
 

RECON 2016CASE STUDY: PMU ROOTKITS CASE 3: MIDDLE OF SYSCALL ROUTINE ▸ We will let syscall routine complete ▸ Find address of ret_fast_syscall on the stack and replace with address of trampoline ▸ Trampoline loads LR with ret_fast_syscall, and branches to appropriate post_hook function ▸ post_hook can retrieve original params from saved user mode registers, and modify as necessary Case 3: Beyond entry point Find and replace on stack
DEMO: PMU ROOTKITPROCESS AND FILE HIDING WITH SYS_GETDENTS64 PMU SVC TRAPS MOTOROLA NEXUS 6
 QCOM APQ8084 (KRAIT) CPU
RECON 2016CASE STUDY: PMU ROOTKITS FUN WITH QMI ▸ Linux rootkits are boring. This is a phone… ▸ Hook sys_read in context of qmuxd in order to intercept all QMI comms from modem to Android (using only the PMU) QMUXD
 QMI PROXY KERNELUSER SPACE sys_write MODEM sys_read PMU Traps SMS APP PHONE APP …
DEMO: PMU ROOTKITINTERCEPTING QMI WITH SYS_READ PMU SVC TRAPS MOTOROLA NEXUS 6
 QCOM APQ8084 (KRAIT) CPU
RECON 2016CASE STUDY: PMU ROOTKITS ANALYSIS AND LIMITATIONS ▸ PMU trap on SVC instructions adds less than 5% overhead (2-3%) ▸ Should evade current kernel integrity monitor algorithms ▸ PMU registers do not persist a core reset ▸ Any other code at PL1/EL1 or higher can read/write the registers
RECON 2016CASE STUDY: PMU ROOTKITS DETECTION STRATEGIES ▸ /proc/interrupts —> easy to modify and cloak ▸ Reading PMU registers looking for someone counting SVCs ▸ Access to PMU registers can be trapped to HYP mode ▸ Not all usage of PMU in this way is malicious… ▸ irq_handler_entry/irq_handler_exit tracepoints ▸ Validate IRQ handler addresses by iterating radix tree structure ▸ PMU Traps on Data & Prefetch Aborts for ShadowWalker?
CASE STUDY: PMU DEFENSE
RECON 2016CASE STUDY: PMU DEFENSE EXPLOIT DETECTION FROM THE KERNEL ▸ Trap SVC instructions to perform syscall monitoring ▸ Detect ROP behavior (e.g. EMET / ROPGuard checks) ▸ Doesn’t increase attack surface to protected user space binaries ▸ Much easier to implement than Rootkit since no re-direction required ▸ Protect COTS binaries (i.e no source/compiler required) ▸ No modifications to kernel image - just need ISR registered
RECON 2016CASE STUDY: PMU DEFENSE ANDROID CVE’S IN MEDIA 0 6 12 18 24 30 INFORMATION 
 DISCLOSURE REMOTE CODE
 EXECUTION ELEVATION OF
 PRIVILEGE DENIAL OF
 SERVICE 4 25 18 10 13 23 4 libstagefright mediaserver Aug 2015 - Jun 2016
DEMO: PMU DEFENSEBLOCKING STAGEFRIGHT ROP CHAIN FROM THE KERNEL LG NEXUS 5
 QCOM MSM8974 (KRAIT) CPU CVE-2015-3864
 POC’s courtesy Mark Brand, Google &
 NorthBit’s Metaphor
RECON 2016 FUTURE WORK ▸ Port instrumentation approach to basebands ▸ Analyze Apple hardware for PMU features and explore iOS kernel tracing
RECON 2016 ACKNOWLEDGEMENTS ▸ Cody Pierce, Endgame ▸ Eric Miller, Endgame ▸ Jamie Butler, Endgame ▸ Several others at Endgame ▸ Researchers that paved the way for PMU assisted security research
QUESTIONS?OR FEEDBACK mspisak at endgame.com
 @matspisak
RECON 2016 REFERENCES 
 S. Vogl and C. Eckert, “Using Hardware Performance Events for Instruction-Level Monitoring on the x86 Architecture,” in Proceedings of EuroSec’12, 5th European Workshop on System Security, ACM Press, Apr. 2012. 
 
 G. Wicherski, “Taming ROP on Sandy Bridge: Using Performance Counters to Detect Kernel Return-Oriented Programming.” SyScan 2013.
 X. Li and M. Crouse, “Transparent ROP Detection using CPU Performance Counters.”https://www.trailofbits.com/threads/2014/ transparent_rop_detection_using_cpu_perfcounters.pdf . Threads 2014. 
 X. Wang and R. Karri, “NumChecker: detecting kernel control-flow modifying rootkits by using hardware performance counters,” in DAC, ACM, 2013. 
 Y. Xia, Y. Liu, H. Chen, and B. Zang, “CFIMon: Detecting violation of control flow integrity using performance counters,” in Proceedings of the 2012 42nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 1–12, IEEE Computer Society, 2012. 
 
 M. Coppola, “Suterusu Rootkit:Inline Kernel Function Hooking on x86 and ARM.” https://github.com/mncoppola/suterusu
 
 
 J. M. Thomas, “Clock Locking Beats: Exploring the Android Kernel and Processor Interactions.” https://github.com/monk-dot/ClockLockingBeats 
 
 
 T. Roth, “Next Generation Mobile Rootkits.” Hack In Paris 2013. https://hackinparis.com/data/slides/2013/Slidesthomasroth.pdf 
 F. M. David, E. M. Chan, J. C. Carlyle, and R. H. Campbell, “Cloaker: Hardware supported rootkit concealment,” in Proceedings - IEEE Symposium on Security and Privacy, pp. 296–310, 2008. 


More Related Content

PDF
Dp.dynamic.positioning.system.converteam.training
byZaw Aung
 
PPTX
EMC2 Xilinx SDSoC presentation
bySundance Multiprocessor Technology Ltd.
 
PPT
Synopsys User Group Presentation
byemlawgr
 
PPTX
Pc 104 series 1 application showcase
bySundance Multiprocessor Technology Ltd.
 
PDF
BUD17-209: Reliability, Availability, and Serviceability (RAS) on ARM64
byLinaro
 
PDF
Debugging 2013- Jesper Brouer
byMediehuset Ingeniøren Live
 
PDF
C041221821
byIOSR-JEN
 
PDF
Model-driven Extraction and Analysis of Network Security Policies (at MoDELS'13)
byJordi Cabot
 
Dp.dynamic.positioning.system.converteam.training
byZaw Aung
 
EMC2 Xilinx SDSoC presentation
bySundance Multiprocessor Technology Ltd.
 
Synopsys User Group Presentation
byemlawgr
 
Pc 104 series 1 application showcase
bySundance Multiprocessor Technology Ltd.
 
BUD17-209: Reliability, Availability, and Serviceability (RAS) on ARM64
byLinaro
 
Debugging 2013- Jesper Brouer
byMediehuset Ingeniøren Live
 
C041221821
byIOSR-JEN
 
Model-driven Extraction and Analysis of Network Security Policies (at MoDELS'13)
byJordi Cabot
 

What's hot

PPT
Arm
bySuresh Kaliyaperumal
 
PPT
Microcontroller 8051
byDenish Vaniyawala
 
PPT
Introduction to Vortex86DX2 Motion-Control Evaluation Board
byroboard
 
PPTX
ARM Architecture in Details
byGlobalLogic Ukraine
 
PPTX
Tools for Practical Attacks on Analog-to-Digital Conversion
byAlexander Bolshev
 
PDF
[DCG 25] Александр Большев - Never Trust Your Inputs or How To Fool an ADC
byDefconRussia
 
PDF
한컴MDS_TRACE32 Debug Environment
byHANCOM MDS
 
PDF
4 io port-programming
byChannabasappa Kudarihal
 
PDF
ARM
byPune University
 
TXT
Ericsson commond list, bss+nss=oss
byHossein Abbasi
 
PDF
arm-cortex-a8
byAndrew Daws
 
PDF
Q4.11: ARM Architecture
byLinaro
 
TXT
Comande oss
byTOUATIPHON
 
PDF
AAME ARM Techcon2013 002v02 Advanced Features
byAnh Dung NGUYEN
 
PDF
Prezentacja_Profil_Portfolio - EN
byTomasz Janicki
 
PPT
W10: Interrupts
byDaniel Roggen
 
PPT
Interrupts
byMonica Sabharwal
 
PDF
Arm architecture overview
bySunil Thorat
 
PDF
C:\Documents And Settings\Fredlin\Desktop\Ic Design\Synthesis200301
byguest77988fe
 
PPT
Plc projectoverview
byJames Chang
 
Arm
bySuresh Kaliyaperumal
 
Microcontroller 8051
byDenish Vaniyawala
 
Introduction to Vortex86DX2 Motion-Control Evaluation Board
byroboard
 
ARM Architecture in Details
byGlobalLogic Ukraine
 
Tools for Practical Attacks on Analog-to-Digital Conversion
byAlexander Bolshev
 
[DCG 25] Александр Большев - Never Trust Your Inputs or How To Fool an ADC
byDefconRussia
 
한컴MDS_TRACE32 Debug Environment
byHANCOM MDS
 
4 io port-programming
byChannabasappa Kudarihal
 
ARM
byPune University
 
Ericsson commond list, bss+nss=oss
byHossein Abbasi
 
arm-cortex-a8
byAndrew Daws
 
Q4.11: ARM Architecture
byLinaro
 
Comande oss
byTOUATIPHON
 
AAME ARM Techcon2013 002v02 Advanced Features
byAnh Dung NGUYEN
 
Prezentacja_Profil_Portfolio - EN
byTomasz Janicki
 
W10: Interrupts
byDaniel Roggen
 
Interrupts
byMonica Sabharwal
 
Arm architecture overview
bySunil Thorat
 
C:\Documents And Settings\Fredlin\Desktop\Ic Design\Synthesis200301
byguest77988fe
 
Plc projectoverview
byJames Chang
 

Similar to Hardware-Assisted Rootkits & Instrumentation

PPTX
ARM-7 ADDRESSING MODES INSTRUCTION SET
bySasiBhushan22
 
PPT
AdvancedRiscMachineryss-INTRODUCTION.ppt
bymailshivaiah
 
PPT
Arm architecture
byPantech ProLabs India Pvt Ltd
 
PPT
LPC 2148 Instructions Set.ppt
byProfBadariNathK
 
PPT
ARM7_Architecture.ppt, RISC-processor core
byRadhakrishna Singh
 
PDF
Arm architecture overview
bySathish Arumugasamy
 
PPTX
ARM Cortex-M3 Training
byRaghav Nayak
 
PDF
HKG18-TR14 - Postmortem Debugging with Coresight
byLinaro
 
PPT
Cache profiling on ARM Linux
byPrabindh Sundareson
 
PPTX
Arm architecture
byAAQIB PARREY
 
PDF
ARM Architecture and Meltdown/Spectre
byGlobalLogic Ukraine
 
PPTX
ARM Architecture for Kernel Development
byGlobalLogic Ukraine
 
PDF
Advanced debugging on ARM Cortex devices such as STM32, Kinetis, LPC, etc.
byAtollic
 
PDF
ARM 64bit has come!
byTetsuyuki Kobayashi
 
PPTX
ARM_Advanced_Exceptions_Interrupts_Extensions.pptx
byhemalathacse1
 
PPT
ARM7TDM
byRamasubbu .P
 
PDF
ARM uVisor Debug Refinement Project(debugging facility improvements)
by家榮 張
 
PDF
Source Boston 2009 - Anti-Debugging A Developers Viewpoint
byTyler Shields
 
PDF
POWER processor and features presentation
byGanesan Narayanasamy
 
PDF
Dmitriy D1g1 Evdokimov - DBI Intro
byDefconRussia
 
ARM-7 ADDRESSING MODES INSTRUCTION SET
bySasiBhushan22
 
AdvancedRiscMachineryss-INTRODUCTION.ppt
bymailshivaiah
 
Arm architecture
byPantech ProLabs India Pvt Ltd
 
LPC 2148 Instructions Set.ppt
byProfBadariNathK
 
ARM7_Architecture.ppt, RISC-processor core
byRadhakrishna Singh
 
Arm architecture overview
bySathish Arumugasamy
 
ARM Cortex-M3 Training
byRaghav Nayak
 
HKG18-TR14 - Postmortem Debugging with Coresight
byLinaro
 
Cache profiling on ARM Linux
byPrabindh Sundareson
 
Arm architecture
byAAQIB PARREY
 
ARM Architecture and Meltdown/Spectre
byGlobalLogic Ukraine
 
ARM Architecture for Kernel Development
byGlobalLogic Ukraine
 
Advanced debugging on ARM Cortex devices such as STM32, Kinetis, LPC, etc.
byAtollic
 
ARM 64bit has come!
byTetsuyuki Kobayashi
 
ARM_Advanced_Exceptions_Interrupts_Extensions.pptx
byhemalathacse1
 
ARM7TDM
byRamasubbu .P
 
ARM uVisor Debug Refinement Project(debugging facility improvements)
by家榮 張
 
Source Boston 2009 - Anti-Debugging A Developers Viewpoint
byTyler Shields
 
POWER processor and features presentation
byGanesan Narayanasamy
 
Dmitriy D1g1 Evdokimov - DBI Intro
byDefconRussia
 

More from EndgameInc

PPTX
Machine Learning for Malware Classification and Clustering
byEndgameInc
 
PPTX
Hunting on the Cheap
byEndgameInc
 
PPTX
Filar seymour oreilly_bot_story_
byEndgameInc
 
PPTX
Worst-Case Scenario: Being Detected without Knowing You are Detected
byEndgameInc
 
PPTX
Extracting the Malware Signal from Internet Noise
byEndgameInc
 
PPTX
​Dynamic Detection of Malicious Behavior
byEndgameInc
 
PPTX
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
byEndgameInc
 
PPTX
Hunting before a Known Incident
byEndgameInc
 
Machine Learning for Malware Classification and Clustering
byEndgameInc
 
Hunting on the Cheap
byEndgameInc
 
Filar seymour oreilly_bot_story_
byEndgameInc
 
Worst-Case Scenario: Being Detected without Knowing You are Detected
byEndgameInc
 
Extracting the Malware Signal from Internet Noise
byEndgameInc
 
​Dynamic Detection of Malicious Behavior
byEndgameInc
 
Vulnerability and Exploit Trends: Combining behavioral analysis and OS defens...
byEndgameInc
 
Hunting before a Known Incident
byEndgameInc
 

Recently uploaded

PDF
Revolutionizing SQL Database Design for the Modern Era.pdf
bySQL DBM
 
PDF
Automating ArcGIS Enterprise Compliance for Operational Excellence
bySafe Software
 
PDF
Do more with the HP Z2 Mini G1a
byPrincipled Technologies
 
PPTX
Odoo_by_Infinys_All_in_One_Business_Solution_for_Small_Companies.pptx
byAgusto Sipahutar
 
PPTX
Enhancing Web Security: Key Concepts & Strategies.pptx
byParham Abolghasemi
 
PDF
Transparency Exchange API: How Do You Find the SBOM for a Smart Light Bulb?
byPavel Shukhman
 
PDF
Configure and Secure SSH - RHCSA (RH124).pdf
byLinuxCert Guru
 
PDF
The Accel 2025 Globalscape: Race for compute
byPhilippe Botteri
 
PDF
Install and Update Software - RHCSA (RH124).pdf
byLinuxCert Guru
 
PPTX
Getting Started with MSP360 Backup for M365/Google
byMSP360
 
PDF
How to not make bugs (in JSC), and the future of 32-bits
byIgalia
 
PDF
ENTSO-E's Response to the European Commission Call for Evidence on the Strate...
byReynir Orn Bachmann Gudmundsson
 
PDF
Upskill to Agentic Automation - Accelerating Your Job Search using AI
byDianaGray10
 
PDF
Create, View and Edit Text Files - RHCSA (RH124).pdf
byLinuxCert Guru
 
PDF
Inclusive India_Samir_Dash_10 NOV_FINAL 2025.pdf
bySamir Dash
 
PDF
Dr. Robert Krug - Specializes In Predictive Modeling
byDr. Robert Krug
 
PDF
Control Access to Files - RHCSA (RH124).pdf
byLinuxCert Guru
 
PDF
Getting started with Agent Framework.pdf
byNilesh Gule
 
PPTX
A GREEN BUSINESS TOOLKIT FOR EARLY-STAGE ENTREPRENEURS (1).pptx.pptx
bylearnskillsofficial1
 
PDF
Private Governance: How Decentralized Mechanisms Can Regulate the Crypto Economy
byFederico Ast
 
Revolutionizing SQL Database Design for the Modern Era.pdf
bySQL DBM
 
Automating ArcGIS Enterprise Compliance for Operational Excellence
bySafe Software
 
Do more with the HP Z2 Mini G1a
byPrincipled Technologies
 
Odoo_by_Infinys_All_in_One_Business_Solution_for_Small_Companies.pptx
byAgusto Sipahutar
 
Enhancing Web Security: Key Concepts & Strategies.pptx
byParham Abolghasemi
 
Transparency Exchange API: How Do You Find the SBOM for a Smart Light Bulb?
byPavel Shukhman
 
Configure and Secure SSH - RHCSA (RH124).pdf
byLinuxCert Guru
 
The Accel 2025 Globalscape: Race for compute
byPhilippe Botteri
 
Install and Update Software - RHCSA (RH124).pdf
byLinuxCert Guru
 
Getting Started with MSP360 Backup for M365/Google
byMSP360
 
How to not make bugs (in JSC), and the future of 32-bits
byIgalia
 
ENTSO-E's Response to the European Commission Call for Evidence on the Strate...
byReynir Orn Bachmann Gudmundsson
 
Upskill to Agentic Automation - Accelerating Your Job Search using AI
byDianaGray10
 
Create, View and Edit Text Files - RHCSA (RH124).pdf
byLinuxCert Guru
 
Inclusive India_Samir_Dash_10 NOV_FINAL 2025.pdf
bySamir Dash
 
Dr. Robert Krug - Specializes In Predictive Modeling
byDr. Robert Krug
 
Control Access to Files - RHCSA (RH124).pdf
byLinuxCert Guru
 
Getting started with Agent Framework.pdf
byNilesh Gule
 
A GREEN BUSINESS TOOLKIT FOR EARLY-STAGE ENTREPRENEURS (1).pptx.pptx
bylearnskillsofficial1
 
Private Governance: How Decentralized Mechanisms Can Regulate the Crypto Economy
byFederico Ast
 

Hardware-Assisted Rootkits & Instrumentation

  • 1.
    ARM EDITION HARDWARE-ASSISTED ROOTKITS& INSTRUMENTATION: Matt Spisak REcon 2016, Montreal
  • 2.
    RECON 2016 ▸ Offense-basedapproach to security and hunting adversaries ▸ Research thrusts in malware, threat intel, data science, and exploit prevention ▸ Matt Spisak (@matspisak) ▸ Vulnerability and exploit mitigation research at Endgame ▸ Mobile security since Nokia N series (before iPhone) ABOUT
  • 3.
    RECON 2016 OUTLINE ▸ Motivation ▸ARM Debug Architecture ▸ Tracing and Instrumentation ▸ Rootkits ▸ TrustZone ▸ Exploit Mitigations
  • 4.
    RECON 2016MOTIVATION DEBUGGING EMBEDDEDSYSTEMS IS COMPLICATED Hardware ‣ JTAG is a gold standard ‣ Custom dev boards + Virtualization extensions ‣ JTAG access can be hit/miss ‣ Destructive ‣ Expensive Software ‣ Portable, scalable ‣ existing tools for HLOS like iOS, Android ‣ Can be tightly coupled to OS ‣ Often limited to PL0/EL0 ‣ Lots of reinventing wheel Emulation ‣ Scalable and powerful ‣ Cost-effective ‣ Sometimes a good option (e.g. CTF) ‣ Lack support for HW interfaces ‣ Requires big time investment
  • 5.
    RECON 2016MOTIVATION SEARCHING FORALTERNATIVES ▸ Whats a good general approach? ▸ Personal philosophy: ▸ Always make use of real hardware ▸ Lean towards software-based tools ▸ GOAL: find common ARM architectural debug features accessible from software (on COTS devices)
  • 6.
  • 7.
    RECON 2016ARM DEBUGARCHITECTURE INVASIVE DEBUG ▸ Debug-modes: Monitor, Halting, or None ▸ Software debug events: BKPT, breakpoint, watchpoint, vector trap ▸ Halting debug events result in processor entering debug state ▸ Support driven by DBGEN and SPIDEN authentication signals ▸ if DBGEN is low —> BKPT instruction only event supported ▸ Authentication signals typically controlled externally ▸ Without DBGEN, options are limited
  • 8.
    RECON 2016ARM DEBUGARCHITECTURE NON-INVASIVE DEBUG ▸ Trace: Embedded Trace Buffer (ETB) / CoreSight Program Flow Trace (PFT) ▸ PFT/PTM generates traces for waypoints: branch & exception instructions ▸ Accessible from external and software (coprocessor or memory-mapped) ▸ PFT/PTM can be locked (ETMLAR) - only writeable in memory-mapped ▸ memory-mapped access is IMPLEMENTATION DEFINED ▸ Trace drivers in Android kernel check CoreSight fuse status ▸ A potential software-based debug feature for COTS devices
  • 9.
    RECON 2016ARM DEBUGARCHITECTURE NON-INVASIVE DEBUG ▸ Sample-based Profiling ▸ Registers for sampling Program Counter and Context ID ▸ No CP14 visibility, optional memory-mapped and external interfaces ▸ PMU ▸ Focus of remainder of talk
  • 10.
  • 11.
  • 12.
    RECON 2016ARM DEBUGARCHITECTURE PERFORMANCE MONITORING UNIT (PMU) ▸ Optional extension, but recommended ▸ Interfaces: CP15 (mandatory), memory-mapped (optional), external (optional) ▸ Dates back to ARMv6, common in ARM11, Cortex-R, Cortex-A ▸ 1 cycle counter, up to 31 general counters ▸ Set of event filters for counting ▸ Support for interrupts on counter overflow 
 sampling period
  • 13.
    RECON 2016ARM DEBUGARCHITECTURE PERFORMANCE MONITORING UNIT (PMU) ▸ Provides real-time feedback on system ▸ Useful for software/hardware engineers ▸ Diagnose bugs ▸ Tools: ▸ ARM DS-5 Streamline ▸ Linux perf / oprofile ARM DS-5 Streamline
  • 14.
    RECON 2016ARM DEBUGARCHITECTURE TERMINOLOGY & ABBREVIATIONS ▸ PMU - Performance Monitoring Unit ▸ PMI - Performance Monitoring Interrupt ▸ PMC - Performance Monitoring Counter Least Privileged Most Privileged USER MODE KERNEL MODE HYPERVISOR SECURE MONITORPL3/EL3 PL2/EL2 PL1/EL1 PL0/EL0 ARM x86 Ring 3 Ring 0 Ring -1 Ring -2 EXCEPTION Reset Undefined Instruction SVC Supervisor Call (e.g. SYSCALL) Prefetch Abort BKPT, or code Page Fault Data Abort Data Page Fault IRQ Interrupts (Normal World) FIQ Fast Interrupts (Secure World) ARM Exception Vector Table (EVT)
  • 15.
    RECON 2016ARM DEBUGARCHITECTURE PMU RELATED WORK ▸ “Using Hardware Performance Events for Instruction-Level Monitoring on the x86 Architecture”, [Vogl, Eckert] ▸ ROP detection with PMU using mispredicted RET [Wicherski], [Li, Crouse] ▸ Rootkit detection with performance counters [Wang, Karri] ▸ Control-flow integrity using BTS [Xia et al] ▸ Control-flow integrity using PMU [Endgame] - BlackHat USA 2016 ▸ All prior art is focused on Intel / x86 architecture
  • 16.
    RECON 2016ARM DEBUGARCHITECTURE SAMPLE ARM PMU EVENTS EVENT TYPE EVENT CODE LD_RETIRED: Load instruction executed 0x06 ST_RETIRED: Store instruction executed 0x07 INST_RETIRED: Instruction executed 0x08 PC_WRITE_RETIRED: Software change of PC 0x0C BR_RETURN_RETIRED: Branch Return retired 0x0E BR_MISP_PRED: Branch mispredicted 0x10 L1I_CACHE: Level 1 instruction cache access 0x14
  • 17.
    RECON 2016ARM DEBUGARCHITECTURE PMU REGISTERS ▸ PMCR - Control Register ▸ N: Number of counters ▸ E: Enable / Disable all counters ▸ ARMv6: MRC/MCR p15, 0, <Rd>, c15, c12, 0 ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 0
  • 18.
    RECON 2016ARM DEBUGARCHITECTURE PMU REGISTERS - CONFIGURE COUNTERS ▸ PMCNTENSET - Enable Counter ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 1 ▸ PMCNTENCLR - Disable Counter ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 2 ▸ PMSELR - Counter Selection Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 5 Use this register prior to read/write
 of event type or counter registers
  • 19.
    RECON 2016ARM DEBUGARCHITECTURE PMU REGISTERS - CONFIGURE COUNTERS ▸ PMXEVTYPER - Counter Event Filter Register ▸ Selects event and modes to count ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c13, 1 ▸ PMXEVTCNTR - Event Counter Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c13, 2 EVENT CODE MODES INCLUDED 0x6800000C Branches in Secure PL1 and HYP 0x6000000C Branches in Secure PL1 0x9800000C Branches in Secure PL0 and HYP 0x9000000C Branches in Secure PL0 0x3800000C Branches in Secure PL0,PL1,HYP 0x4000000C Branches in non-secure PL1 0x8000000C Branches in non-secure PL0
  • 20.
    RECON 2016ARM DEBUGARCHITECTURE PMU REGISTERS - CONFIGURE COUNTERS //Enable armv7 PMU Counters
 MRC p15, 0, R1, c9, c12, 0
 ORR R1, R1, #1
 MCR p15, 0, R1, c9, c12, 0 //Set PMC1 to count Instructions Executed
 MOV R1, #1
 MCR p15, 0, R1, c9, c12, 5 //PMSELR
 MOV R1, #0x8
 MCR p15, 0, R1, c9, c13, 1 //PMXEVTYPER //Initialize PMC1 to -3
 MOV R1, #0xFFFFFFFD
 MCR p15, 0, R1, c9, c13, 2 //PMXEVTCNTR //Enable PMC1
 MOV R1, #1
 MCR p15, 0, R1, c9, c12, 1 //PMCNTENSET
  • 21.
    RECON 2016ARM DEBUGARCHITECTURE PMU REGISTERS - CONFIGURE INTERRUPTS ▸ PMINTENSET - Interrupt Enable Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c14, 1 ▸ PMINTENCLR - Interrupt Disable Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c14, 2 ▸ PMOVSR - Overflow Status Register
 PMOVSET - Overflow Status Set Register ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c12, 3 ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c9, c14, 3
  • 22.
    RECON 2016ARM DEBUGARCHITECTURE PMU REGISTERS - CONFIGURE INTERRUPTS //Enable Interrupts for PMC1 and PMC2
 MOV R1, #3
 MCR p15, 0, R1, c9, c14, 1 //PMINTENSET //Read and Clear Overflow on Interrupt
 MRC p15, 0, R0, c9, c12, 3 //PMOVSR
 MCR p15, 0, R0, c9, c12, 3 //PMOVSR
  • 23.
    RECON 2016ARM DEBUGARCHITECTURE DO YOU EVEN COUNT? ▸ DBGAUTHSTATUS ▸ Lists whether invasive/non-invasive debug are supported in secure and non-secure worlds ▸ ARMv7: MRC/MCR p14, 0, <Rd>, c7, c14, 6 ▸ ID_DFR0 ▸ Lists PMU version supported (if any) ▸ ARMv7: MRC/MCR p15, 0, <Rd>, c0, c1, 2
  • 24.
    RECON 2016ARM DEBUGARCHITECTURE THE CENTER FOR CHIPS WHO CAN COUNT GOOD
  • 25.
  • 26.
    RECON 2016CASE STUDY:PMU TRACING APPROACH ▸ Make the PMU more invasive with frequent PMC-based traps ▸ CoreSight Program Flow Trace (PFT) captures waypoints (i.e. branches) ▸ We can come pretty close to PFT Trace using the PMU: ▸ Count all branches: predicted and mispredicted ▸ Interrupt all the things: set our counter(s) to -1 ▸ Use our ISR as the instrumentation logic BX BL B
  • 27.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING func: error: -1 BL func
 
 STMFD SP!, {R0-R2,R4-R9,LR}
 MOV R8, R1
 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches)
  • 28.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 MOV R8, R1
 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 29.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 30.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 31.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING func: error: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 32.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 33.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 34.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) func: error: -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 -1 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 35.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT
 - RESET COUNTER func: error: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 -1 BEQ error
 0 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC overflow
  • 36.
    RECON 2016CASE STUDY:PMU TRACING APPROACH - BRANCH TRACING PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT
 - RESET COUNTER func: error: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) -1 BL func
 
 0 STMFD SP!, {R0-R2,R4-R9,LR}
 -1 MOV R8, R1
 -1 MOV R1, SP
 -1 MOV R2, R2
 -1 LDR R7, [SP]
 -1 CMP R7, #0
 -1 BEQ error
 0 MOV R4, #0xFFFFFFF7
 -1 ADD SP, SP, #0xC PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER
  • 37.
    RECON 2016CASE STUDY:PMU TRACING BUT WHAT ABOUT LINUX PERF? ▸ We want a custom ISR for instrumentation ▸ Too tightly coupled to Linux ▸ Invoking API’s != learning ▸ But perf source can be useful for understanding PMU interfaces
  • 38.
  • 39.
    RECON 2016CASE STUDY:PMU TRACING ARM GENERIC INTERRUPT CONTROLLER (GIC) SPECIFICATION ▸ SGI: Software Generated Interrupts
 PPI: Private Peripheral Interrupts
 SPI: Shared Peripheral Interrupts ▸ ARM GIC spec recommends PMU Overflows to use INTID 23 ARM GIC Architecture Specification
  • 40.
    RECON 2016 ▸ DeviceTree Source ▸ Brute Force ▸ Register all unused PPI’s & SPI’s, trigger PMIs, diff /proc/interrupts
 cpu-pmu { compatible = "qcom,krait-pmu"; qcom,irq-is-percpu; interrupts = <1 7 0xf00>; }; CASE STUDY: PMU TRACING CHALLENGE: FINDING PMU INTERRUPTS INT# = 16 + 7 = 23 ▸ Implementation: ▸ Android: request_percpu_irq(), request_threaded_irq() ▸ Embedded firmware: patch IRQ vector handler PPI
  • 41.
    RECON 2016CASE STUDY:PMU TRACING CHALLENGE: INTERRUPT SHADOW func: error: -1 BL func
 
 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches)
  • 42.
    RECON 2016CASE STUDY:PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW -1 BL func
 
 0 LDR R7, [SP]
 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC error:
  • 43.
    RECON 2016CASE STUDY:PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC error:
  • 44.
    RECON 2016CASE STUDY:PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC error:
  • 45.
    RECON 2016CASE STUDY:PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW error: -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 1 MOV R4, #0xFFFFFFF7
 ADD SP, SP, #0xC
  • 46.
    RECON 2016CASE STUDY:PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW error: PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 1 MOV R4, #0xFFFFFFF7
 1 ADD SP, SP, #0xC
  • 47.
    RECON 2016CASE STUDY:PMU TRACING func: PMC1: 0xFFFFFFFF (-1) Event: 0x0C (All Branches) overflow CHALLENGE: INTERRUPT SHADOW error: PMU ISR - CAPTURE PC - CAPTURE REGS
 - MEMORY SNAPSHOT - RESET COUNTER -1 BL func
 
 0 LDR R7, [SP]
 0 CMP R7, #0
 0 BEQ error
 1 MOV R4, #0xFFFFFFF7
 1 ADD SP, SP, #0xC } Interrupt Shadow
 Skid = 4 Instructions Causes miss of up to 15% covered basic blocks
  • 48.
    RECON 2016CASE STUDY:PMU TRACING OTHER CHALLENGES ▸ CPU Hot-Plugging — easy solution for Android: register_hotcpu_notifier() ▸ Lack of Last Branch Recording feature on ARM ▸ Complicated kernel mode instrumentation: use sampling period of -2
 ▸ Requires small patch to entry-armv.S (or hot patch) PMU ISR 2) RET from ISR causes overflow 3) PMI Infinite Interrupt Loop __IRQ_SVC Set PMC1 = -2Set PMC1 = -1 Sampling Period: 0xFFFFFFFE (-2) 2) RET from IRQ vector increments PMC1 Next branch triggers overflow Sampling Period: 0xFFFFFFFF (-1) 1) PMI 1) PMI
  • 49.
    RECON 2016CASE STUDY:PMU TRACING ANDROID PROTOTYPE PERFMON
 ISR PERFMON
 ISR PERFMON
 ISR … RELAY THREAD RELAY THREAD RELAY THREAD … KERNELUSER SPACE IDA Plugin CORE 1 CORE 2 CORE N … pmutrace.kopmu_server Captures PC at
 time of interrupt,
 buffered per core PC PC PC PC PC PC PC PC PC PC Visualize coverage and control pmu_server to select threads, mode, and start/stop
  • 50.
    RECON 2016CASE STUDY:PMU TRACING CONNECTING THE DOTS ▸ Use IDA to our advantage ▸ For each PMU waypoint: ▸ Color/count all instructions in Basic Block ▸ If only 1 xref from basic block: count/color it ▸ If only 1 xref to basic block: count/color it Example of a perfect PMU branch tracing run
  • 51.
    RECON 2016CASE STUDY:PMU TRACING CONNECTING THE DOTS ▸ Interrupt shadow ▸ Basic block xref algorithm helps fill in missed blocks ▸ Fuzzing / code coverage will eventually be interrupted in this block ▸ Could improve by adding 2nd counter to count instructions between interrupts Interrupt 
 Shadow Example of PMU trace missing basic block
  • 52.
    DEMO: PMU TRACING DEVICEREQUIREMENTS: ‣ ROOTED ‣ CONFIG_MODULES OPTION (NOT AS COMMON) ‣ CONFIG_PREEMPT OPTION (COMMON) ‣ IRQ HANDLER PATCH (PL1/EL1)
  • 53.
    RECON 2016CASE STUDY:PMU TRACING ANDROID INSTRUMENTATION. SO WHAT? ▸ Recall approach is hardware-assisted - not tied to a specific OS ▸ Less invasive than BKPT tracing ▸ Supports both user mode and kernel mode instrumentation ▸ Not limited to branch tracing, other potential instrumentation use-cases ▸ And these chips can count too: ▸ Broadcom WiFi; Intel/Infineon, MediaTek + other ARM Cellular Basebands ▸ Apple ARM SoCs ▸ PowerPC, MIPS
  • 54.
  • 55.
    RECON 2016CASE STUDY:PMU ROOTKITS PRIOR ART IN ARM ROOTKITS ▸ Traditional rootkits: modify syscall table or EVT [Phrack Issue 68] ▸ Suterusu performs hot patching of kernel functions [Coppola] ▸ Cloaker toggles SCTLR to move EVT [David et al] ▸ Clock Locking Beats explores using CPU governor for hiding cycles [Thomas] ▸ TrustZone based rootkit [Roth]
  • 56.
    RECON 2016CASE STUDY:PMU ROOTKITS INSPIRATION ARM Architecture Manual ARMv7-A&R - Appendix C AH AH AH
 very interesting….

  • 57.
    RECON 2016TEXT TRACING ROOTKITSTRUSTZONE HYPERVISOR EXPLOIT
 PREVENTION DEFENSENSTRUMENTATION PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED PMU
 ASSISTED
  • 58.
    RECON 2016CASE STUDY:PMU ROOTKITS QUICK NOTE ON ARM LICENSES ▸ ARM Core License ▸ Use core ARM designs ▸ ARM Architectural license ▸ Enables custom cores provided it implements an ARM instruction set ▸ Examples: Qualcomm Scorpion/Krait/Kryo, Apple A6/A7/etc.
  • 59.
    RECON 2016CASE STUDY:PMU ROOTKITS COUNTING THE EXCEPTION VECTOR TABLE EVENT Cortex-A7 Cortex-A53 Cortex-A57 Cortex-A72 Scorpion Krait Kryo Undefined Instruction √ √ √ √ ? SVC √ √ √ √ ? Prefetch Abort √ √ √ √ ? Data Abort √ √ √ √ ? IRQ √ √ √ √ √ √ ? FIQ √ √ √ √ √ √ ? SMC * * √ √ √ √ ? HVC √ √ ? ? ? ARM Design Custom ARM-based Design
  • 60.
    RECON 2016CASE STUDY:PMU ROOTKITS DOWN THE RABBIT HOLE ARM Architecture Manual ARMv7-A&R ▸ Chipset vendors with proprietary PMU implementations: ▸ Qualcomm ▸ Apple ▸ Likely others Covered in earlier slides
  • 61.
    RECON 2016CASE STUDY:PMU ROOTKITS SCORPION KRAIT KRYO 2008 2012 2015 ARMv7 ARMv7 ARMv8 1-2 Cores 2 or 4 cores 4 cores Snapdragon S1/S2/S3 Snapdragon S4/400/600/800/805 Snapdragon 818/820/823 BlackBerry Bold 9900
 Samsung Galaxy S2 (LTE)
 Nokia Lumia 900
 HTC Droid Incredible Nexus 4/5/6/7
 Samsung Galaxy S4/S5
 HTC One M8
 LG G3 LG G5
 Samsung Galaxy S7
 HTC 10
 Xiomi Mi 5
  • 62.
    RECON 2016CASE STUDY:PMU ROOTKITS QUALCOMM KRAIT PMU ▸ Adds 4 event select registers: 1 for Venum VFP, 3 for other components of CPU ▸ Krait event encoded using code + group + region => (code << 8 * group) ▸ ARM event select register (PMXEVTYPER) set to link to Krait region and group Krait Region 0 Krait Region 1 Krait Region 2 MRC/MCR p15, 1, <Rd>, c9, c15, 0 MRC/MCR p15, 1, <Rd>, c9, c15, 1 MRC/MCR p15, 1, <Rd>, c9, c15, 2 Interrupts/Exceptions + other ? ? ~100 event codes ~128 event codes ~156 event codes PMXEVTYPER = 0xCC | group PMXEVTYPER = 0xD0 | group PMXEVTYPER = 0xD4 | group Only a few documented in old Scorpion src. Black-box analysis used to determine # of events
  • 63.
    RECON 2016CASE STUDY:PMU ROOTKITS QUALCOMM KRAIT PMU ▸ Configure Krait + ARM PMU to count Prefetch Aborts: ▸ Krait Event Code: 0x0B group: 3 Region: 0 /*Set Krait Region 0 event selection register
 To count Prefetch Aborts*/
 MRC p15, 0, R1, c9, c15, 0
 ORR R1, R1, #0x8b000000
 MCR p15, 0, R1, c9, c15, 0 //Set PMXEVTYPER to point to krait region 0
 MOV R1, #0xCF
 MCR p15, 0, R1, c9, c13, 1

  • 64.
    RECON 2016CASE STUDY:PMU ROOTKITS PMU-ASSISTED ROOTKITS ▸ Trap SVC instructions via PMU ▸ Use ISR to filter system calls, and redirect code execution after servicing PMI ▸ Avoids patch protection* ▸ Installation: a few instructions to initialize PMU registers, and then register ISR for PMU interrupts 1 2 3 4
  • 65.
    RECON 2016CASE STUDY:PMU ROOTKITS CHALLENGE: DELAYED INSTRUCTION SKID ▸ PMI serviced at some point after IRQs enabled in vector_swi ▸ 3 cases we must deal with: 1. PMI before branch to syscall routine within vector_swi 2. PMI at entry point of syscall routine 3. PMI in middle of syscall routine IRQs enabled
  • 66.
    RECON 2016CASE STUDY:PMU ROOTKITS CASE 1: INTERRUPT BEFORE BRANCH TO SYSCALL ROUTINE #define CPSIE_ADDR 0xC01064D0
 …
 irq_regs = get_irq_regs(); //get SVC mode regs
 pregs = task_pt_regs(current); //get user mode regs
 …
 if (pregs->ARM_r7 == 0x3) //sys_read
 { 
 switch (irq_regs->ARM_pc - CPSIE_ADDR) //offset after CPSIE
 {
 //emulate remaining instructions up to LDRCC
 //can skip those involved in resolving syscall routine
 case 0x0:
 case 0x4:
 irq_regs->ARM_r9 = irq_regs->ARM_sp & 0xFFFFE000;
 …
 case 0x14:
 case 0x18:
 case 0x1C:
 case 0x20:
 irq_regs->ARM_lr = ret_fast_syscall;
 case 0x24:
 irq_regs->ARM_pc = (uint32_t)hook_sysread;

  • 67.
    RECON 2016CASE STUDY:PMU ROOTKITS CASE 2: SYSCALL ROUTINE ENTRY POINT ▸ Replace saved PC with address of hook 
 irq_regs = get_irq_regs();
 pregs = task_pt_regs(current);
 …
 if (pregs->ARM_r7 == 0x3) //sys_read
 { 
 //Check if PMU interrupted at entry point addr of sys_read 
 if (pregs->ARM_pc == orig_sys_read)
 {
 pregs->ARM_pc = (uint32_t)hook_sys_read; 
 

  • 68.
    RECON 2016CASE STUDY:PMU ROOTKITS CASE 3: MIDDLE OF SYSCALL ROUTINE ▸ We will let syscall routine complete ▸ Find address of ret_fast_syscall on the stack and replace with address of trampoline ▸ Trampoline loads LR with ret_fast_syscall, and branches to appropriate post_hook function ▸ post_hook can retrieve original params from saved user mode registers, and modify as necessary Case 3: Beyond entry point Find and replace on stack
  • 69.
    DEMO: PMU ROOTKITPROCESSAND FILE HIDING WITH SYS_GETDENTS64 PMU SVC TRAPS MOTOROLA NEXUS 6
 QCOM APQ8084 (KRAIT) CPU
  • 70.
    RECON 2016CASE STUDY:PMU ROOTKITS FUN WITH QMI ▸ Linux rootkits are boring. This is a phone… ▸ Hook sys_read in context of qmuxd in order to intercept all QMI comms from modem to Android (using only the PMU) QMUXD
 QMI PROXY KERNELUSER SPACE sys_write MODEM sys_read PMU Traps SMS APP PHONE APP …
  • 71.
    DEMO: PMU ROOTKITINTERCEPTINGQMI WITH SYS_READ PMU SVC TRAPS MOTOROLA NEXUS 6
 QCOM APQ8084 (KRAIT) CPU
  • 72.
    RECON 2016CASE STUDY:PMU ROOTKITS ANALYSIS AND LIMITATIONS ▸ PMU trap on SVC instructions adds less than 5% overhead (2-3%) ▸ Should evade current kernel integrity monitor algorithms ▸ PMU registers do not persist a core reset ▸ Any other code at PL1/EL1 or higher can read/write the registers
  • 73.
    RECON 2016CASE STUDY:PMU ROOTKITS DETECTION STRATEGIES ▸ /proc/interrupts —> easy to modify and cloak ▸ Reading PMU registers looking for someone counting SVCs ▸ Access to PMU registers can be trapped to HYP mode ▸ Not all usage of PMU in this way is malicious… ▸ irq_handler_entry/irq_handler_exit tracepoints ▸ Validate IRQ handler addresses by iterating radix tree structure ▸ PMU Traps on Data & Prefetch Aborts for ShadowWalker?
  • 74.
  • 75.
    RECON 2016CASE STUDY:PMU DEFENSE EXPLOIT DETECTION FROM THE KERNEL ▸ Trap SVC instructions to perform syscall monitoring ▸ Detect ROP behavior (e.g. EMET / ROPGuard checks) ▸ Doesn’t increase attack surface to protected user space binaries ▸ Much easier to implement than Rootkit since no re-direction required ▸ Protect COTS binaries (i.e no source/compiler required) ▸ No modifications to kernel image - just need ISR registered
  • 76.
    RECON 2016CASE STUDY:PMU DEFENSE ANDROID CVE’S IN MEDIA 0 6 12 18 24 30 INFORMATION 
 DISCLOSURE REMOTE CODE
 EXECUTION ELEVATION OF
 PRIVILEGE DENIAL OF
 SERVICE 4 25 18 10 13 23 4 libstagefright mediaserver Aug 2015 - Jun 2016
  • 77.
    DEMO: PMU DEFENSEBLOCKINGSTAGEFRIGHT ROP CHAIN FROM THE KERNEL LG NEXUS 5
 QCOM MSM8974 (KRAIT) CPU CVE-2015-3864
 POC’s courtesy Mark Brand, Google &
 NorthBit’s Metaphor
  • 78.
    RECON 2016 FUTURE WORK ▸Port instrumentation approach to basebands ▸ Analyze Apple hardware for PMU features and explore iOS kernel tracing
  • 79.
    RECON 2016 ACKNOWLEDGEMENTS ▸ CodyPierce, Endgame ▸ Eric Miller, Endgame ▸ Jamie Butler, Endgame ▸ Several others at Endgame ▸ Researchers that paved the way for PMU assisted security research
  • 80.
    QUESTIONS?OR FEEDBACK mspisak atendgame.com
 @matspisak
  • 81.
    RECON 2016 REFERENCES 
 S. Vogland C. Eckert, “Using Hardware Performance Events for Instruction-Level Monitoring on the x86 Architecture,” in Proceedings of EuroSec’12, 5th European Workshop on System Security, ACM Press, Apr. 2012. 
 
 G. Wicherski, “Taming ROP on Sandy Bridge: Using Performance Counters to Detect Kernel Return-Oriented Programming.” SyScan 2013.
 X. Li and M. Crouse, “Transparent ROP Detection using CPU Performance Counters.”https://www.trailofbits.com/threads/2014/ transparent_rop_detection_using_cpu_perfcounters.pdf . Threads 2014. 
 X. Wang and R. Karri, “NumChecker: detecting kernel control-flow modifying rootkits by using hardware performance counters,” in DAC, ACM, 2013. 
 Y. Xia, Y. Liu, H. Chen, and B. Zang, “CFIMon: Detecting violation of control flow integrity using performance counters,” in Proceedings of the 2012 42nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 1–12, IEEE Computer Society, 2012. 
 
 M. Coppola, “Suterusu Rootkit:Inline Kernel Function Hooking on x86 and ARM.” https://github.com/mncoppola/suterusu
 
 
 J. M. Thomas, “Clock Locking Beats: Exploring the Android Kernel and Processor Interactions.” https://github.com/monk-dot/ClockLockingBeats 
 
 
 T. Roth, “Next Generation Mobile Rootkits.” Hack In Paris 2013. https://hackinparis.com/data/slides/2013/Slidesthomasroth.pdf 
 F. M. David, E. M. Chan, J. C. Carlyle, and R. H. Campbell, “Cloaker: Hardware supported rootkit concealment,” in Proceedings - IEEE Symposium on Security and Privacy, pp. 296–310, 2008.