specialization of generics array accesses after inlining

Specialization of Generic Array Accesses After Inlining

Oleg勉強会@no_maddo

2/22

What We DidType specialization of generic

(dynamically typed) array accesses after function inlining

• Eliminates dispatch on type tags in arrays

⇒ Achieves up to 21% speed-up for numerical computation

3/22

Background:Representation of Arrays in OCaml• Basically boxed (for polymorphism)• Exception: int and float arrays are flat⇒ requires array_kind on each operation

let i = int_arr.{I}(1)let f = float_arr.{F}(2)

let get0 a = a.{Gen}(0)(* get0: 'a array -> 'a *)

type array_kind = I (* integer *)| F (* float *)| P (* pointer *) | Gen (* dynamic *)

4/22

Problem

• The dynamic type checks remain even after monomorphised by inlining – due to the (basically) untyped intermediate

language⇒ large overheads for polymorphic array operations such as Array.map and Array.fold_left

Inlining

let get0 a = a.{Gen}(0)(* get0: 'a array -> 'a *)let i = get0 int_arrlet f = get0 float_arr

let i = int_arr.{Gen}(0)let f = float_arr.{Gen}(0)

5/22

Our Approach

• Add more type information like System F (type abstraction and instantiation)to the intermediate language

Inlining

let get0 {'a} a = a.{'a}(0)let i = get0{I} int_arr let f = get0{F} float_arr

let i = int_arr.{I}(0)

let f = float_arr.{F}(0)

6/22

Gap between Theory and Implementation

The IL of OCaml compiler has:• No type abstraction

⇒ Introduce type variables represented by global id (as usual in OCaml)

• No type instantiation⇒ Extend the IL

7/22

Our Solution

Inlining

let get0 (*{'a}*) a = a.{'a}(0)(* get0: 'a array -> 'a *)let i = get0{'a↦I} int_arr let f = get0{'a↦F} float_arr

let i = int_arr.{I}(0)let f = float_arr.{F}(0)

8/20

Implementation

9/22

type array_kind = I (* integer *)| F (* float *)| P (* pointer *) | Gen (* dynamic *)

Type Abstraction• Replace Gen with Kvar

let get0 a = a.{Gen}(0)(* get0: ‘a array -> ‘a *)

‘a

Kvar of int (* kind variable id *)

10/22

Type Application

• Add Lspecialized to lambda

• Insert Lspecialized for every use of polymorphic value during translation from typedtree to lambda

type lambda = ... | Lspecialized of lambda * kind_mapand kind_map = (int * type_kind) list (* eg. 'a ↦I *)

11/22

How to Construct kind_map

Compare• the polymorphic type stored in the type

environment– eg. get0: 'a array -> 'a

and• the monomorphic type attached in the

AST – eg. (get0: float array -> float) [|3.14|]

to construct {'a ↦ F}

12/22

Follow α-renaming of type variables

val get0: ‘b array -> ‘ba.cmi

get0 a = a.{‘a}(0)a.cmx

let get0 = a.{‘a}(0)(* get0:‘a array -> ‘a *)

a.ml

‘b

13/22



get0 a = a.{'b}(0)a.cmx

(* import A.get0 as A.get0:‘c array -> ‘c *)

A.get0{‘c ↦ F} float_arr

b.mllet get0 = a.{‘a}(0)(* get0:‘a array -> ‘a *)

a.ml ‘c ↦‘b

14/22





float_arr. {(‘b ↦ F)(‘c ↦ ‘b)(‘c)}(0)


a.ml ‘c ↦‘b

15/22





float_arr.{F}(0)


a.ml ‘c ↦‘b

16/20

Experiments

17/22

Experiments• Performed numerical computations– written in a functional style using

Array.map, Array.fold_left, etc.• Modified 4.02 branch of OCaml

compiler as of May 6, 2015• Compiled with ocamlopt -inline

10000000 –unsafe including stdlib

– Run on Ubuntu Linux 14.04 i7-2677M 1.80 GHz (fixed at 1.0 GHz) and 4 GiB DDR3 SDRAM

18/22

As an aside ...• A modern CPU is complex, so it is

difficult to get precise result because of– variable frequency (Turbo-boost)

19/22

ResultsProgram Generic Array Access

RemovedExecution Time Reduced

Simple 100.0% 5%Random 50.0% 15%DKA 33.6% 2%FFT 2.8% 0%K-Means 8.6% 1%LD 15.8% 3%LU 0.0% 0%NN 22.9% 4%QR 98.9% 21%

20/22

Summary• Relatively simple modification to

OCaml compiler for faster polymorphic array operations

• Non-trivial speed-ups for numerical computations

21/22

Related Work ???• Type directed compilation– TILT, Intentional analysis (Morrisett 95)

• Scala (specialized annotations, mini-boxing)

• Type specialization is common in Just-In-Time compilation– In latter half of 2000

22/22

Future Work• Relax a restriction on inlining

due to extra closures – eg.

let foldi f init x = snd (Array.fold_left (fun (i, acc) xi -> (i+1, f i acc xi)) (0, init) x) (from K-Means)

• Apply our technique to generic comparisons

23/22

Misc ...• Oleg-san made a

documentation of type inference algorithm of OCaml

http://okmij.org/ftp/ML/generalization.html




24/22

宣伝• MLアドベントカレンダー人少ないので是非記事をお願いします！– http://www.adventar.org/calendars/848

specialization of generics array accesses after inlining

Technology