OpenGL/ES and SGL: 2D/3D Graphics Accelerating Technologies
in Android Platform
Shao-Yi Chien (簡韶逸)
Media IC & System Lab
Graduate Institute of Electronics Engineering
National Taiwan University
Outline
• Introduction to Android
• Related 2D/3D graphics API
• SGL
• OpenGL/ES
• Integration of hardware accelerators
• Commercial products
• Our new developed GPU
Introduction to Android
• Android
– A software stack for mobile devices
– Includes an operating system, middleware and key applications
• The Android SDK
– Provides the tools and APIs necessary to begin developing applications on the Android platform using the Java programming language.
Architecture of Android
Related 2D/3D Graphics API
• 2D graphics library– SGL is the underlying 2D graphics engine
– The most common 2D graphics APIs can be found in the drawable package (android.graphics.drawable).
• 3D graphics library– An implementation based on OpenGL ES 1.0 APIs
– OpenGL APIs are available from the Khronos OpenGL ES package (javax.microedition.khronos.opengles, javax.microedition.khronos.egl)• Similar to the J2ME JSR239
– Plus some Android OpenGL utilities (android.opengl)
SGL
• Scalable Graphics Library (Skia Graphics Library)
• SGL is the low-level graphics library implemented in native code that handles rendering– Text, Geometries, and Images
• Developed by Skia Inc., which is acquired by Google in 2005.
• SGL is feature-set compatible with existing 2D standards, making it ideal to serve as a back-end for public formats such as SVG, PDF, and OpenVG
• Employed in Android and Chrome
Khronos Group
Khronos Group
OpenGL
OpenCL
OpenGL ES Family(OpenGL for Embedded Systems)
OpenGL 1.3
OpenGL ES 1.0
OpenGL ES 1.1
OpenGL 1.5
OpenGL 2.0
OpenGL ES 2.0
They will exist at the same time.
They won’t replace each other
Widely available
cross-platform 3D
graphics API
Enable software AND hardware implementations -
- including small-footprint, low-end fixed point
platform
What’s New in OpenGL ES 1.1
• Enhance effects– Multi-texture
– Vertex skinning
– Particle effect
– Bump shading…
• Enhance power– Buffer object – support render to texture
– Swap interval
– Power event
Roles for Each Family Members
• OpenGL ES 1.0: low cost devices, can be implemented with software
• OpenGL ES 1.1: must be implemented with some graphics accelerator
• OpenGL ES 2.0: support shading language. For high-end products
Integration of Hardware Accelerators:OpenGL as an Example
• If there is no 3D hardware acceleratorApplication Program
OS
Services
I/O
ServicesGDI OpenGL
Software
Rasterizer
Display
Device
Integration of Hardware Accelerators:OpenGL as an Example
• If there is 3D hardware acceleratorApplication Program
OS
Services
I/O
ServicesGDI OpenGL
Hardware
Driver
Display
Device
Commercial Products
• PowerVR MBX
– Support OpenGL ES 1.1 and OpenVG 1.0
• ARM Mali family
Commercial Products
Our New Developed GPU
• A unified architecture to support multimedia applications
• Quad-core stream processor SoC– 1 configurable filtering unit (CFU)– 2 stream processing units (SPUs)– 1 general-purpose RISC core
• Proposed techniques– Unified stream kernel– Configurable memory array (CMA)– Adaptive task scheduling– Adaptive multi-threading– Power aware frequency scaling (PAFS)– Configurable filtering unit (CFU)
System Architecture
AHB System Bus
Off-Chip
Memory
External
Memory
Bus
Unified Stream Processing Subsystem
RISC
CPU
Power
Aware
Frequency
Scaling
Memory
Controller
(MC)
Vertex
Fetch
Unit
Universal
Rasterizer
Rop
Unit
Configurable Memory Array (CMA)
Unified Shader 0
(US0)
Unified Shader 1
(US1)
I$ D$
T$Texture Unit with CFU Mobile
GPU
Coef.
Unified Stream Kernel (USK)
• Different level of scalability
19
Configurable Memory Array (CMA)
• Intra-level configuration
– Adjust different size of registers/cache depending on the application
• Inter-level configuration
– Assist to balance the workload in the two cores
20
Adaptive Task Scheduling (ATS)
• An application can always be partitioned into a series of cascaded tasks.
• If the workloads of the two stages are unbalance, the hardware utilization and performance will decrease– ATS technique is proposed
21
Adaptive Task Scheduling (ATS)
• USK0 is the bottleneck stage
• USK1 is the bottleneck stage
22
Adaptive Multi-Threading (AMT)
• Hide the instruction latency to increase hardware utilization and performance
23
Power Aware Frequency Scaling (PAFS)
• Level 1– Frequency scaling for low
battery state
– 30% Power Reduction
• Level 2– Clock gating for idle block
– 16% Power Reduction
• Nearly 50% power reduction
24