Android Benchmarking and Optimization Opportunities

● Talk about what was planned in Hong Kong and give a summary

● Present current benchmark methodology and hotspot benchmark results

● Discuss methodology● Suggest specific optimization opportunities and

additional measurements and benchmarks● Explore additional benchmarks

Recap

● Linaro Android as a Benchmarking Platform● Integrate ARM's system benchmarks into LAVA● Perform hotspot analysis of benchmarking APKs and

browser benchmarks

Benchmarks Integrated● AndEBench● Antutu● Linpack● CaffeineMark● GLBenchmark● GeekBench 2● Quadrant● Vellamo● NBench● Methanol

Benchmark methodology

For Low Variance:● Governance policy to performance.● Disable data syncing for other apps.● Kill Background processes.● Avoid installing other apps.● Wifi enabled for network access.

Characterization:● DS-5 streamline.

Benchmark GainsAOSP + Linaro Patches build on Panda 4460 Measured with 3.2 TILT stable kernel vs AOSP build with 3.2 kernel:● Antutu RAM 21 %● CaffeineMark String 24 %● Quadrant Mem 81 %● Antutu Float 10 %● 0xBench Sunspider 10 %● 0xBench DrawArc,DrawCircle,DrawRect >40 %● 0xBench Garbage Collection 11 %● Vellamo Sunspider 14 %● Vellamo V8 Benchmark 8 %

Benchmarks Compiler libc JIT libdvm libskia libm kernel webkit V8 Engine Other libraries

AndEBench Native High None None None None None None None None None

AndEBench Java None None High Low None None None None None None

Linpack None None High None None None None None None None

Caffeinemark None None High Low None None None None None None

Antutu CPU & Memory High Medium None None None Low Low None None None

Antutu 2D Graphics None Medium None None Low None Medium None None None

Antutu 3D Graphics Low Low None None None None Low None None None

Antutu DB IO None Medium None None None None Medium None None None

nbench High Low None None None None None None None None

Quadrant CPU Benchmark None None High None None None None None None None

Quadrant Memory Benchmark

None None High None None Medium None None None

Quadrant IO Benchmark None None None None None None Low None None None

High(>50%) Medium(50%>x>30%) Low(<10%)Benchmark Characterization

Benchmarks Compiler libc JIT libdvm libskia libm kernel webkit V8 Engine

Other libraries

Quadrant 2D Benchmark

None None None Low Medium None Medium None None None

Quadrant 3D Benchmark

None None None Low None None Low None None Low(libgfx)

Vellamo None Low None None Low None None Medium Medium None

Geekbench High Low None None None None None None None None

GlBenchmark Low None None None None None Low None None Low(libgfx)

Bbench Low Low None Low Low None Medium High Very Low

Low(libgfx)

Benchmarks >50% 30%< X <50% 10%< X < 30% <10% <10% <10%

AndEBench Native

libcore_ARM 86% (ndk Toolchain)

AndEBench Java Dalvik-jit-cache-code (52%)

libdvm(11%)

Linpack Dalvik-jit-cache-code (90%)

Caffeinemark Dalvik-jit-cache-code (40%)

libdvm(4.5%)

Antutu CPU & Memory

libabenchmark 53%(NDK Toolchain)

libc(memcpy 14.25,bcopy 10.8)

kernel(10%) Libm(6.25%)

Antutu 2D Graphics

kernel(30.9%) libc(memcpy 12.28) libskia(3.75%)

Antutu 3D Graphics

libabenchmark(7.3%) libc(7.7%) kernel(7.3%)

Antutu DB IO kernel(31.5%) libc(memcpy 21.5)

nbench nbench(40%) libc(bcopy3.9%, memcpy 1.19%)

Quadrant CPU Benchmark

Dalvik-jit-cache-code(94.11)

Quadrant Memory Benchmark

libc(memcpy 38%) kernel(13%)

Quadrant IO Benchmark

kernel(6.28%)

Quadrant 2D Benchmark

Libskia(22.3%),kernel(15%) libdvm(7%)

Quadrant 3D Benchmark

libgfx(9%) kernel(7.11%) libdvm(5.78%)

Vellamo libwebcore(12.44%) Unresolved(8.50%) Libskia(8.33%)

Geekbench Libgeekbench(58%) Libc(1.54%)

GlBenchmark Kernel(3.35%) Libglbenchmark(3.01%) Libgfx (2.74%)

Optimization Opportunities

● Some of the gains already achieved from libc,libm optimization which reflect in benchmark gains.

● Scope for more optimizations in bionic.● A15 optimization for the same routines.● Scope for Dalvik JIT,V8 optimizations.● Add real world use cases for analysis.