Project meeting: Android Graphics Architecture Overview
3 likes2,722 views
The Android graphics architecture uses SurfaceFlinger to composite surfaces from apps into the final display. Apps draw to surfaces using a canvas, which can target OpenGL or a software renderer. Views define a display list of drawing commands. SurfaceFlinger receives surfaces from apps and window status from WindowManager to composite surfaces in z-order before each VSYNC signal.
Project meeting: Android Graphics Architecture Overview
- 3. Android graphics stack overview ● App acquires its own Surface(s) from SurfaceFlinger ○ comes with a Canvas and a graphics buffer ● App uses Canvas to draw the screen ○ underlying implementations may use OpenGL or software render engine (e.g. SKIA) ● SurfaceFlinger composes final screen from Surfaces ○ get Activity status from ActivityManager ○ get z-order information from WindowManager ● Gralloc maps to the framebuffer device (fb0)
- 4. Android View component ● Button, TextView… are all Views ● A page layout is composed of a tree of Views ● Actual drawing behavior implemented in onDraw() method of each View ○ Display List: a set of graphic commands ○ commands can be mapped to OpenGL commands ● Create DeferredDisplayList in order to optimize the rendering order ○ analyze the overlapping… relationships ○ minimize GPU usage
- 5. Android View component ● An example of Display List (button) ○ Save 3 ○ DrawPatch ○ Save 3 ○ ClipRect 20.00, 4.00, 99.00, 44.00, 1 ○ Translate 20.00, 12.00 ○ DrawText 9, 18, 9, 0.00, 19.00, 0x17e898 ○ Restore ○ RestoreToCount 0 ● Can be mapped to gl-prefix OpenGL commands
- 6. 2D rendering path ● Can be rendered by HW or SW ○ HWUI: hardware accelerated using OpenGL ES 2.0 ○ Skia: software render engine
- 7. 3D rendering path ● App can instead create GLSurfaceView ● Use OpenGL ES bindings for Java ● Either use vendor GPU driver or software-based PixelFlinger to render the drawing to Surface
- 8. SurfaceFlinger Compose, but not render!
- 9. SurfaceFlinger ● Services connections from activities (client) via Binder interface ISurfaceComposer (server) ● Activity acquires Surface from SurfaceFlinger ○ created by ISurfaceComposer ● Receives activity status from ActivityManager ● Receives window status (visibility, z-order) from WindowManager ● Composes multiple Surfaces into a single image ○ by HWComposer: also generates VSYNC
- 10. SurfaceFlinger ● GraphicBuffer ○ allocated by Gralloc, accessed using mmap ○ pixel format: RGB/RGBA/RGBO ● Project Butter ○ VSYNC ○ Triple Buffer ○ Choreographer ○ http://www.androidpolice.com/2012/07/12/getting-to-know-andr oid-4-1-part-3-project-butter-how-it-works-and-what-it-added/
- 11. VSYNC Signal ● Generated by HWComposer ○ HW source: register callback of HAL ○ or SW source: VSyncThread ● Distributed by SurfaceFlinger ○ Client can create an event connection to SurfaceFlinger’s MessageQueue to receive the VSync event ○ SurfaceFlinger inherits HWComposer::EventHandler to handle onVSyncRecieved() and notify listeners
- 12. Application ● Each page has its View Tree ● Need to traverse the tree when ○ App or page being launched ○ External event: touch, key press ○ Internal event: UI control change, View.Visibility ● How to traverse ○ Post a request to Choreographer (if no request left) ○ Choreograpger calls perfromTraversals() when VSYNC signal triggered
- 13. Application ● Inside performTraversals() ○ performMeasure(): calculate size of UI controls ○ performLayout(): calculate UI layout ○ performDraw() ● Inside performDraw() ○ High level: draw to Canvas (based on Surface in Java level) ○ Low level ■ Lock Surface (native level) to dequeue a buffer from BufferQueue ■ Render to buffer by OpenGL ES or SKIA ■ Unlock Surface to enqueue the buffer
- 14. Surface (native level) ● Each app/page can acquire one or more Surface/Layer ● Done by using ISurfaceComposerClient to connect to SurfaceFlinger’s Binder service ● ISurfaceComposerClient will return a SurfaceControl, then use it to create Surface ● One of two native windows in Android graphic systems (another one is for SurfaceFlinger) ● OpenGL ES render to the buffer and enqueue to BufferQueue of Surface, act as a “producer”
- 15. BufferQueue ● A producer/consumer pattern ○ Producer: Application (Surface) ○ Consumer: SurfaceFlinger ● Buffer operation ○ Application: dequeue()->write buffer->enqueue() ○ SurfaceFlinger: acquire()->read buffer->release() ● When a buffer being queued ○ Layer listens to onFrameAvailable() callback ○ Call Layer’s onFrameQueued() to increment counter variable: mQueuedFrames
- 16. BufferQueue read buffer write buffer ApplicationSurfaceflinger
- 17. SurfaceFlinger ● When VSYNC signal triggered ○ handleMessageTransaction(): detect Layers’ changes (including size, z-order, alpha, add/removal…), save as mDrawingState ○ handleMessageInvalidate(): acquire a buffer from BufferQueue if available ○ signalRefresh() ---> call handleMessageRefresh()
- 18. SurfaceFlinger ● handleMessageRefresh() ○ preComposition(): check if there is any layer with available buffers (mQueuedFrames > 0), signal invalidate if true ○ rebuildLayerStacks(): by calculating opaque region, visible region, covered region and transparent region ○ …… ○ doComposition(): done by HWComposer or OpenGL ES ○ Swap buffers (front/back buffer)