Inside Android's UI at AnDevCon VI
3 likes•695 views
This document provides a summary of Android's user interface architecture. It discusses the hardware components like the display, touch input, and keyboard. It then covers the main software layers including the kernel driver, HAL definition, HAL modules, SurfaceFlinger, Window Manager, and how they interact. It also summarizes OpenGL, the input layer, relevant apps and services, and the system startup process. The document recommends several references for learning more about Android's graphics and input subsystems.
![13
/frameworks/base/services/java/...
/frameworks/base/services/jni/
/hardware/libhardware/
/device/[MANUF.]/[DEVICE]
/sdk/emulator/
Kernel or module
/frameworks/base/core/...
AOSP-provided
ASL
Manuf.-provided
Manuf. license
Manuf.-provided
GPL-license](https://image.slidesharecdn.com/inside-androids-ui-131114-140516121653-phpapp02/85/Inside-Android-s-UI-at-AnDevCon-VI-13-320.jpg)
Inside Android's UI at AnDevCon VI
- 1. 1 Inside Android's UI AnDevCon 2013 Karim Yaghmour @karimyaghmour [email protected]
- 2. 2 These slides are made available to you under a Creative Commons Share- Alike 3.0 license. The full terms of this license are here: https://creativecommons.org/licenses/by-sa/3.0/ Attribution requirements and misc., PLEASE READ: ● This slide must remain as-is in this specific location (slide #2), everything else you are free to change; including the logo :-) ● Use of figures in other documents must feature the below “Originals at” URL immediately under that figure and the below copyright notice where appropriate. ● You are free to fill in the “Delivered and/or customized by” space on the right as you see fit. ● You are FORBIDEN from using the default “About” slide as-is or any of its contents. ● You are FORBIDEN from using any content provided by 3rd parties without the EXPLICIT consent from those parties. (C) Copyright 2013, Opersys inc. These slides created by: Karim Yaghmour Originals at: www.opersys.com/community/docs Delivered and/or customized by
- 3. 3 About â—Ź Author of: â—Ź Introduced Linux Trace Toolkit in 1999 â—Ź Originated Adeos and relayfs (kernel/relay.c) â—Ź Training, Custom Dev, Consulting, ...
- 4. 4 Agenda â—Ź Android's UI, what's that? â—Ź Architecture Basics â—Ź Display Core â—Ź OpenGL â—Ź Input Layer â—Ź Relevant Apps and Services â—Ź System Startup â—Ź References and Pointers
- 5. 5 1. Android's UI, what's that? â—ŹSoC / GPU â—ŹTouch input â—ŹLCD â—ŹKeyboard ???
- 6. 6 1.1. What I'm NOT covering â—Ź Media layer â—Ź StageFright â—Ź Video playback â—Ź Camera â—Ź DRM â—Ź Etc.
- 7. 7 2. Architecture Basics â—Ź Hardware used to run Android â—Ź AOSP â—Ź Binder â—Ź System Services â—Ź HAL
- 8. 8
- 9. 9
- 10. 10
- 11. 11
- 12. 12
- 13. 13 /frameworks/base/services/java/... /frameworks/base/services/jni/ /hardware/libhardware/ /device/[MANUF.]/[DEVICE] /sdk/emulator/ Kernel or module /frameworks/base/core/... AOSP-provided ASL Manuf.-provided Manuf. license Manuf.-provided GPL-license
- 14. 14 3. Display Core â—Ź Display Hardware â—Ź Classic Linux display stack â—Ź Display stack in Android â—Ź Kernel driver â—Ź HAL definition â—Ź HAL module â—Ź Surface Flinger â—Ź Window Manager â—Ź Walkthrough
- 16. 16 3.2. Classic Linux display stack
- 17. 17 3.3. Display stack in Android
- 18. 18
- 19. 19 3.4. Kernel driver â—Ź Video memory management â—Ź Mode setting â—Ź Checking of parameters â—Ź Motorola Xoom: â—Ź /dev/nvhdcp1 â—Ź /dev/nvhost-ctrl â—Ź /dev/nvhost-display â—Ź /dev/nvhost-dsi â—Ź /dev/nvhost-gr2d â—Ź /dev/nvhost-gr3d â—Ź /dev/nvhost-isp â—Ź /dev/nvhost-mpe â—Ź /dev/nvhost-vi â—Ź /dev/nvmap â—Ź /dev/tegra-crypto â—Ź /dev/tegra_avp â—Ź /dev/tegra_camera â—Ź /dev/tegra_fuse â—Ź /dev/tegra_rpc â—Ź /dev/tegra_sema â—Ź ... whatever hides in hwcomposer HAL module
- 20. 20 3.5. HAL Definition â—Ź /hardware/libhardware/include/hardware/hwcomposer.h â—Ź struct hwc_procs: â—Ź invalidate() â—Ź vsync() â—Ź struct hwc_composer_device: â—Ź prepare() â—Ź set() â—Ź dump() â—Ź registerProcs() â—Ź query() â—Ź *()
- 21. 21 3.6. HAL module ● Skeleton /hardware/libhardware/modules/hwcomposer.cpp ● /system/lib/hw/hwcomposer.BOARD.so ● /system/lib/hw/gralloc.BOARD.so ● Ex. - Mot Xoom: ● hwcomposer.tegra.so ● gralloc.tegra.so ● Surface Flinger hook: ● /frameworks/native/services/surfaceflinger/DisplayHardware – HWComposer.cpp – Provides fake vsync if none is provided in HW
- 22. 22 3.7. Surface Flinger â—Ź Actual server: â—Ź /frameworks/native/services/surfaceflinger â—Ź Client side: â—Ź /frameworks/native/libs/gui â—Ź Client / Server interface: â—Ź ISurfaceComposerClient.cpp â—Ź ISurfaceComposer.cpp â—Ź This is NOT an aidl'ed service â—Ź All communication is manually marshalled/unmarshalled
- 23. 23 3.8. Window Manager ● Server side: ● /frameworks/base/services/java/com/android/server/wm/ – WindowManagerService.java – Session.java ● Client side: ● /frameworks/base/core/java/android/view/ – WindowManager.java – WindowManagerImpl.java – ViewRootImpl.java ● Interfaces: ● IWindowManager.aidl ● IWindowSession.aidl ● Parameters (incl. z-order): ● See WindowManager.java
- 24. 24 3.9. Walkthrough â—Ź Activity Manager relies on Activity Thread â—Ź AT calls on attach() and makeVisible() â—Ź makeVisible does wm.addView() â—Ź wm.addView() - this also called by StatusBar to display itself â—Ź Creates a new ViewRootImpl â—Ź Call on its setView() â—Ź setView() calls on mWindowSession.addToDisplay(...) â—Ź This results in call to WM's addWindow() â—Ź ViewRootImpl's performTraversals() â—Ź Calls on relayoutWindow() â—Ź Calls to WM session's relayout() â—Ź Call to WM's relayoutWindow() â—Ź Call to createSurfaceLocked() â—Ź new Surface(...)
- 25. 25 frameworks/base/core/java/android/*/* LocalActivityManager.java: startActivity() - moveToState() - startActivityNow() ActivityThread.java: startActivityNow() - performLaunchActivity() - attach() -- gets AM handle and ties to it - handleResumeActivity() - makeVisible() Activity.java: makeVisible() - wm.addView() WindowManagerGlobal.java: addView() - root = new ViewRootImpl() - root.setView() ViewRootImpl.java: setView() - mWindowSession.addToDisplay() IWindowSession.aidl: addToDisplay()
- 26. 26 frameworks/base/services/java/com/android/server/wm/* Session.java: addToDisplay() - mService.addWindow() WindowManagerService.java: addWindow() ... frameworks/base/core/java/android/*/* ViewRootImpl.java: performTraversals() - relayoutWindow() - mWindowSession.relayout() frameworks/base/services/java/com/android/server/wm/* Session.java: relayoutWindow() - mService.relayoutWindow() WindowManagerService.java: relayoutWindow() - surface = winAnimator.createSurfaceLocked(); WindowStateAnimator.java: createSurfaceLocked() - new Surface();
- 27. 27 4. OpenGL â—Ź What's OpenGL? â—Ź What's in a modern-day GPU? â—Ź Software layers involved â—Ź Kernel driver â—Ź EGL libs â—Ź Native interface â—Ź Java interface â—Ź Software GL implementation
- 28. 28 4.1. What's OpenGL? ● It's just an API ... nothing but an API ... ● Check out Wikipedia ● Multiple versions out ● “ES” versions for embedded use ● Up to ES 3.0 ● Android support up to ES 2.0
- 29. 29 4.2. What's in a modern-day GPU? ● A tremendous amount of parallel processing units ● “SIMD”-like instruction set ● Video decoding/encoding capabilities ● ...
- 30. 30 4.3. Software layers involved â—Ź Kernel driver â—Ź GL libraries â—Ź Native GL API â—Ź Java GL API
- 32. 32 4.5. EGL libs â—Ź /frameworks/base/native/opengl/libs â—Ź Entry point: /system/lib/libEGL.so â—Ź Looks for /system/lib/egl/egl.cfg â—Ź /system/lib/egl - Mot Xoom: â—Ź egl.cfg â—Ź libEGL_perfhud.so â—Ź libEGL_tegra.so â—Ź libGLES_android.so â—Ź libGLESv1_CM_perfhud.so â—Ź libGLESv1_CM_tegra.so â—Ź libGLESv2_perfhud.so â—Ź libGLESv2_tegra.so â—Ź elg.cfg: 0 0 tegra
- 33. 33 4.6. Native interface â—Ź /frameworks/native/opengl/include â—Ź EGL â—Ź ETC1 â—Ź GLES â—Ź GLES2 â—Ź KHR
- 34. 34 4.7. Java interface â—Ź GL libs required by libandroid_runtime.so â—Ź /frameworks/base/opengl/java/android/opengl: â—Ź ...
- 35. 35 4.8. Software GL implementation â—Ź /frameworks/native/opengl/libagl
- 36. 36 5. Input Layer ● Kernel side - “std” Linux input layer: ● /dev/input/* ● No HAL use ● Native lib: ● libinput ● /frameworks/base/services/input ● Input Manager Service: ● /frameworks/base/services/java/com/android/server/input ● Started and directly tied to Window Manager ● Specific config files (see source.android.com) ● Soft keyboard: ● /frameworks/base/core/java/android/inputmethodservice ● Input methods: ● /packages/inputmehods ● http://developer.android.com/guide/topics/text/creating-input-method.html
- 37. 37 6. Relevant Apps and Services â—Ź Launcher â—Ź StatusBar â—Ź Wallpaper Manager Service â—Ź Notification Service â—Ź App Widgets
- 38. 38 6.1. Launcher â—Ź An app like any other â—Ź See /packages/app/Launcher2
- 39. 39 6.2. StatusBar â—Ź A unique app â—Ź See /frameworks/base/packages/SystemUI â—Ź Connects to Status Bar Manager and gives an interface it can use to call back into Status Bar â—Ź Can use setIcon() to display icons on the right â—Ź Provides a CPU usage add-on that renders straight on rest of display using higher z-order
- 40. 40 6.3. Wallpaper Manager Service â—Ź See /frameworks/base/services/java/com/android/se rver/WallpaperManagerService.java
- 41. 41 6.4. Notification Service â—Ź Toasts â—Ź Status bar notifications â—Ź Gets handle to Status Bar Service at instantiation â—Ź Uses handle to communicate with Status Bar
- 42. 42 6.5. App Widgets â—Ź See /frameworks/base/services/java/com/android/se rver/AppWidgetService.java
- 43. 43 7. System Startup â—Ź Kernel â—Ź Init â—Ź Boot animation â—Ź Launcher
- 44. 44 7.1. Boot animation ● Started by Surface Flinger ● “bootanim” binary ● /frameworks/base/cmds/bootanimation ● Relies on bootanimation.zip w/ PNGs (nothing but) ● See https://github.com/CyanogenMod/android_vendor_cm/tree/jellybean/pre built/common/bootanimatino ● Must contain a desc.txt: <width> <height> <fps> p <count> <pause> <path> p <count> <pause> <path>
- 45. 45 8. References and Pointers ● “Use the source, Luke” ● Jim Huang's “Android Graphics” ● Benjamin Zores' “Linux Magazine / France” articles ● MIPS article on graphics internals: http://developer.mips.com/2012/04/11/learning-about- android-graphics-subsystem/ ● Stéphane Marchesin's “Linux Graphics Drivers: an Introduction” http://source.android.com/tech/input/index.html