texture caching implementation for hardware accelerated · pdf filetexture caching...

12th Kandroid

Conference, 2013

Texture Caching Implementation

for Hardware Accelerated 2D Rendering

양정수 (Jeongsoo Yang)

yangjeongsoo at gmail.com

www.kandroid.org

“4 Featured Sessions” - Kandroid S/W Fundamentals Study Group

1. Terminology and History of Android Performance Features

2. Automated Historical Performance Analysis with kmemtracer

3. High Performance Apps Development with RenderScript

4. Texture Caching Implementation for Hardware Accelerated 2D Rendering

2 12th Kandroid Conference, 2013

CONTENT

1. What is Open GL Texture?

2. Understanding GPU Memory

3. Case Study :

a. Cocos2d-x Texture Cache

b. Android HWUI Texture Cache

c. Chrome Browser Texture Cache

d. Android SurfaceFlinger Texture Cache

4. Conclusion


The true worth of a man is not to be found in man himself, but in the

colours and textures that come alive in others. - Albert Schweitzer

Source


Bitmap mBitmapTexture

= BitmapFactory.decodeResource(mContext.getResources(), R.drawable.texture1);

nativeInit(mBitmapTexture);

mBitmapTexture.recycle();

nativeRender();

glGenTextures(1, &texture);

glBindTexture(GL_TEXTURE_2D, texture);

glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

AndroidBitmap_lockPixels(env, bitmap, &pixels);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA,

bitmapInfo.width, bitmapInfo.height,0,

GL_RGBA, GL_UNSIGNED_BYTE, pixels);

AndroidBitmap_unlockPixels(env, bitmap);

1

2

3

4

1. Load Bitmap

2. Load Texture from Bitmap

3. Free Bitmap

4. Draw Target with Texture

1 2 3 4

1 2 3 4


CONTENT



3. Case Study :





4. Conclusion


ISP

GPU

LPDDR3

CPU

~12GB/s

70GFLOPs ~20GFLOPs

DSP GDDR5

GPU

DDR3

CPU

200+GB/s 50GB/s

PCIe

~12GB/s

3+TFLOPs ~250GFLOPs

High Performance Applications with RenderScript

Google I/O 2013

Desktop/Server System Architecture Mobile System Architecture


Source : http://www.imgtec.com/powervr/insider/powervr_presentations/GDC2013_TheArchitectureOfHighEndMobileGraphicsHardware.pdf

http://www.imgtec.com/powervr/insider/powervr_presentations/GDC2013_TheArchitectureOfHighEndMobileGraphicsHardware.pdf




Source : http://www.nvidia.com/content/PDF/tegra_white_papers/Bringing_High-End_Graphics_to_Handheld_Devices.pdf

Geforce GPU Architecture in NVIDIA Tegra

http://www.nvidia.com/content/PDF/tegra_white_papers/Bringing_High-End_Graphics_to_Handheld_Devices.pdf






Source : http://www.many-core.group.cam.ac.uk/ukgpucc2/talks/Lokhmotov.pdf

http://www.many-core.group.cam.ac.uk/ukgpucc2/talks/Lokhmotov.pdf






Source : http://www.many-core.group.cam.ac.uk/ukgpucc2/talks/Lokhmotov.pdf







CONTENT



3. Case Study :





4. Conclusion


Beyond Android

With the current market saturation, is it possible to create a new

platform?

• B2G, Tizen, LG Web OS, Ubuntu Mobile

The successful development of a new platform could lead to

advances. Nevertheless, it would be just one more in an already

saturated market. So, we must streamline the market and focus

on cross-platform compatibility with the status quo.

• Web-based (e.g. PhoneGap)

• Native-based (e.g. Cocos2d-x)

• VM-based (e.g. Mono)


Source : http://www.cocos2d-x.org/projects/cocos2d-x/wiki/Relationships_in_Cocos2d_Family

Relationship in Cocos2d Family

http://www.cocos2d-x.org/projects/cocos2d-x/wiki/Relationships_in_Cocos2d_Family








Cocos2d-x Blank Screen Issue

We had some reports that was blank screen problems,

but we couldn’t reproduce that error.


Some Checkpoints for Cocos2d-x Blank Screen Issue

1. EGL Context related

• Cocos2d-x Architecture

2. Process and Thread termination related

• Binder Handle Recovery Mechanism

3. Texture Cache related

Cocos2d-x Android

Director ActivityManager

Scene Activity

Layer Surface

Sprite (Node) Widget (View)

Animations Frame, Tweened, Property

Texture Cache ?



4. Snapshot Analysis for Cocos2d-x Memory State


-

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000

140,000,000

160,000,000

180,000,000

1

16

31

46

61

76

91

106

121

136

151

166

181

196

211

226

241

256

271

286

301

316

331

346

361

376

391

406

421

436

451

466

481

496

511

Size

Alloc

Free

Native Pss

Other Pss

D/cocos2d-x debug info(697): OpenGL error 0x0505


4. Historical Analysis for Cocos2d-x Memory State


Cocos2d-x Android

Director ActivityManager

Scene Activity

Layer Surface

Sprite (Node) Widget (View)

Animations Frame, Tweened, Property

Texture Cache

• TextureAtlas (sprite image based)

• TextureCache (file name based) ?

D/cocos2d-x debug info(697): OpenGL error 0x0505




0

10000000

20000000

30000000

40000000

50000000

60000000

Total

Alloc

Free

NPss

OPss




CONTENT



3. Case Study :





4. Conclusion


OpenGLRenderer (libhwui) Properties

Source : http://source.android.com/devices/tuning.html

Property Type Default Value

ro.hwui.disable_scissor_opt boolean false

ro.hwui.texture_cache_size float 24

ro.hwui.layer_cache_size float 16

ro.hwui.gradient_cache_size Float 0.5

ro.hwui.path_cache_size float 4

ro.hwui.shape_cache_size float 1

ro.hwui.drop_shadow_cache_size float 2

ro.hwui.fbo_cache_size integer 16

ro.hwui.r_buffer_cache_size float 2

ro.hwui.texture_cache_flush_rate float 0.6

ro.hwui.text_small_cache_width integer 1024

ro.hwui.text_small_cache_height integer 256

ro.hwui.text_large_cache_width integer 2048

ro.hwui.text_large_cache_height integer 512

hwui.text_gamma_correction string lookup

hwui.text_gamma float 1.4

hwui.text_gamma.black_threshold integer 64

hwui.text_gamma.white_threshold integer 192

hwui.use_gpu_pixel_buffers boolean true

http://source.android.com/devices/tuning.html




libhwui : frameworks/base/libs/hwui/TextureCache.cpp

mCache GenerationCache<SkBitmap*, Texture*> hwui

Cache

TextureCache

LayerCache

GradientCache

PathCache

Texture* TextureCache::get(SkBitmap* bitmap) {

Texture* texture = mCache.get(bitmap);

if (!texture) {

generateTexture(bitmap, texture, false);

mCache.put(bitmap, texture);

}

else if (bitmap->getGenerationID()

!= texture->generation) {

generateTexture(bitmap, texture, true);

}

return texture;

}


Android Animations : Frame, Tweened, Property


Caches: Current memory usage / total memory usage (bytes): TextureCache 24576000 / 25165824 LayerCache 0 / 16777216 RenderBufferCache 0 / 2097152 GradientCache 0 / 524288 PathCache 0 / 10485760 TextDropShadowCache 0 / 2097152 FontRenderer 0 524288 / 524288 Other: FboCache 0 / 16 PatchCache 1 / 512 Total memory usage: 25100288 bytes, 23.94 MB

ImageView logoImage = (ImageView) findViewById(R.id.iv1);

logoImage.setBackgroundResource(R.drawable.logo_animation);

logoAnimation = (AnimationDrawable) logoImage.getBackground();

logoAnimation.start();

Android Animations : Frame, Tweened, Property


CONTENT



3. Case Study :





4. Conclusion


Google I/O 2012 Keynote

Android vs. Chrome


Google I/O 2013 Keynote

Sundar Pichai (SVP, Android, Chrome & Apps)


Chrome’s Multi-process Architecture on Android

Main

Thread

(UI)

I/O

Thread

Main

Thread

Render

Thread

Browser Process Render Process

IPC

android:process=":sandboxed_process0" android:isolatedProcess="true“

android:process=":privileged_process2" android:isolatedProcess="false"

android:isolatedProcess

If set to true, this service will run under a special process that is isolated from the rest of

the system and has no permissions of its own. The only communication with it is

through the Service API (binding and starting).


Render Process

WebKit / Skia

Shared

Memory

Bitmaps

& Arrays

IPC

Compositor

Commands

Browser

Process

GPU Thread

(server)

Compositor

Context

GL/D3D

Chrome’s Hardware Acceleration on Android

Compositing with the GPU Thread


One challenge with all these textures is that we rasterize them on

the main thread of the renderer process, but need to actually get

them into the GPU memory. This requires handing information about these textures (and their

contents) to the impl thread, then to the GPU process, and once there, into the GL/D3D driver. Done naively, this

causes us to copy a single texture over and over again, something we definitely don't want to do.

We have two tricks that we use right now to make this a bit faster. To understand them, an aside on

“painting” versus “rasterization.”

• Painting is the word we use for telling webkit to dump a part of its RenderObject tree to a GraphicsContext. We

can pass the painting routine a GraphicsContext implementation that executes the commands as it receives them,

or we can pass it a recording context that simply writes down the commands as it receives them.

• Rasterization is the word we use for actually executing graphics context commands. We typically execute the

rasterization commands with the CPU (software rendering) but could also execute them directly with the GPU using

Ganesh.

• Upload: this is us actually taking the contents of a rasterized bitmap in main memory and sending it to the GPU as

a texture.

With these definitions in mind, we deal with texture upload with the following tricks:

• Per-tile painting: we pass WebKit paint a recording context that simply records the GraphicsContext

operations into an SkPicture data structure. We can then rasterize several texture tiles from that one picture.

• SHM upload: instead of rasterizing into a void* from the renderer heap, we allocate a shared memory

buffer and upload into that instead. The GPU process then issues its glTex* operations using that shared memory,

avoiding one texture copy.

Compositor Thread Architecture & Texture Upload

Source : http://dev.chromium.org/developers/design-documents/compositor-thread-architecture

http://dev.chromium.org/developers/design-documents/compositor-thread-architecture









CONTENT



3. Case Study :



c. Android Animation & Texture Cache

d. Chrome Browser Texture Cache

e. Android SurfaceFlinger Texture Cache

4. Conclusion


Beyond Android

With the current market saturation, is it possible to create a new

platform?

• B2G, Tizen, LG Web OS, Ubuntu Mobile

The successful development of a new platform could lead to

advances. Nevertheless, it would be just one more in an already

saturated market. So, we must streamline the market and focus

on cross-platform compatibility with the status quo.

• Web-based (e.g. PhoneGap)

• Native-based (e.g. Cocos2d-x)

• VM-based (e.g. Mono)


How Android is using Gralloc

The magic part is that SurfaceFlinger does not need to do the equivalent of

GrallocTextureHostOGL::Unlock. In our case, we have a separate OpenGL

texture object for each TextureHost, which typically (at least in the case of a

ContentHost) represent one buffer each (so a double-buffered ContentHost

has two TextureHost's). So we have to unbind the GraphicBuffer from the

OpenGL texture before we can hand it back to the content side --- otherwise

it would remain locked for read and couldn't be locked for write for content

drawing.

By contrast, SurfaceFlinger does not need to worry about this because it

uses only one OpenGL texture, so that when it binds a new

GraphicBuffer to it for compositing, that automatically unbinds the

previous one!

Source : https://wiki.mozilla.org/Platform/GFX/Gralloc

SurfaceFlinger and OpenGL Texture


FrameBuffer SurfaceFlinger

Surface

Surface

Surface

APP

APP

Google I/O 2008 : Anatomy & Physiology of an Android


Frame Buffer

Surface Flinger

APP

APP

Surface

Surface

Surface

SurfaceFlinger::instantiate()

- AddSevice(“Surface Flinger”..)

SurfaceFlinger::readyToRun()

- Gather EGL extensions

- Create EGL Surface and Map Frame Buffer

- Create our OpenGL ES context

- Gather OpenGL ES extensions

- Init Display Hardware for GPU

SurfaceFlinger::threadLoop()

- Wait for Event

- Check for tranaction - Post Surface (if needed) - Post FrameBuffer …

Source : 4th kandroid conference, 2009


Draw

DisplayList

Swap

Buffers

Composite

Windows

Post

Buffer

Buffer

Overlay

Overlay

Overlay

Overlay

FB

FB

FB

GPU

Google I/O 2012 : For Butter or Worse


CONTENT



3. Case Study :





4. Conclusion


Developer Event Agenda

13:00 - 13:20 Registration and demos

13:20 - 13:40 Introduction to Imagination

Bryce Johnstone, Senior Manager Ecosystems Marketing

13:40 - 14:40 Imagination Hardware Architecture

Kristof Beets, Senior Business Development Manager, Imagination

14:40 - 15:00 Coffee Break & Demos

15:00 - 16:00 PowerVR SDK and Utilities

Bob Gardner, Developer Technology Engineer, Imagination

16:00 - 17:00 PowerVR Tips and tricks : Maximise your performance

Bob Gardner, Developer Technology Engineer, Imagination

17:00 - 17:15 Q+A, wrap up and prize draw

Bryce Johnstone, Senior Manager Ecosystems Marketing

17:15 - 19:15 Networking session - beer and hors d'oeuvres.

IDC 2013 agenda – Seoul, Korea - 12th November 2013


ARM Technology Symposia 2013 : 11월19일


13th Kandroid Conference, 2014

“Graphics & GPU”

• Morning Sessions : Open Sessions

• Afternoon Sessions : Featured Sessions

texture caching implementation for hardware accelerated · pdf filetexture caching...

Documents