deep dive into deeplearn.js
Post on 21-Jan-2018
1.050 Views
Preview:
TRANSCRIPT
Deep Dive into deeplearn.jsKai Sasaki
Treasure Data Inc
About me
Kai Sasaki - 佐々木 海 (@Lewuathe)
Software Engineer at Treasure Data Apache Hivemall Committer Hadoop, Spark, deeplearnjs Contributor
What is deeplearn.js?
deeplearn.js is a deep learning library running on your browser accelerated by WebGL. Mainly developed by Google PAIR project.
https://research.googleblog.com/2017/08/harness-power-of-machine-learning-in.html
Similar Products
WebDNN - https://mil-tokyo.github.io/webdnn/ Keras.js - https://github.com/transcranial/keras-js
Technology
- Written in TypeScript - Available in TypeScript and ES6 project - Accelerated by WebGL - Tested on Chrome and Firefox - Can run both training and inference
https://www.npmjs.com/~types
Architecture
Graph: Representing computation data flow much like graph in TensorFlow etc Op: A graph node that represents a computation on tensor NDArray: Real data copied from/to CPU and GPU backed by Typed Array
Computation Graph
Graph
Node
Node
Node Tensor
Tensor
Tensor
Graph
Node
Node
Node
Node
Node Tensor
Tensor
Tensor
Graph
Node
Node
Node
Node
Node
Node
Tensor
Tensor
Tensor
Tensor
Tensor
add
multiply
conv2d
reluetc
Operation
Op
Op
Op
Op
Op
Op
NDArray
NDArray
NDArray
NDArray
NDArray
Kernel Implementation
math
Web Browser
WebGL
math
NDArrayMathCPUNDArrayMathGPU
Web Browser
WebGL
math
math
NDArrayMathCPUNDArrayMathGPU
Web Browser
WebGL
kernelNDArray
math
math
NDArrayMathCPUNDArrayMathGPU
Web Browser
WebGL
kernelNDArray
Conv2d softmax etc…
Graph
matmul
Kernel Exampleimport {Array1D, NDArrayMathGPU, Scalar} from 'deeplearn';
const math = new NDArrayMathGPU();const a = Array1D.new([1, 2, 3]);const b = Scalar.new(2);
const result = math.add(a, b);
// Option 1: With async/await.// Caveat: in non-Chrome browsers you need to put this in an async function.console.log(await result.data()); // Float32Array([3, 4, 5])
// Option 2: With a Promise.result.data().then(data => console.log(data));
// Option 3: Synchronous download of data.// This is simpler, but blocks the UI until the GPU is done.console.log(result.dataSync());
TensorFlow like Exampleconst graph = new Graph();// Make a new input in the graph, called 'x', with shape [] (a Scalar).const x: Tensor = graph.placeholder('x', []);// Make new variables in the graph, 'a', 'b', 'c' with shape [] and random// initial values.const a: Tensor = graph.variable('a', Scalar.new(Math.random()));const b: Tensor = graph.variable('b', Scalar.new(Math.random()));const c: Tensor = graph.variable('c', Scalar.new(Math.random()));
//…
// At this point the graph is set up, but has not yet been evaluated.// **deeplearn.js** needs a Session object to evaluate a graph.const math = new NDArrayMathGPU();const session = new Session(graph, math);
TensorFlow Integration
Port TensorFlow model
deeplearn.js can import TensorFlow model from checkpoint. Need to port the weights from checkpoint files.
https://deeplearnjs.org/demos/mnist/mnist.html
Porting weights
$ python scripts/dump_checkpoints/dump_checkpoint_vars.py \ —model_type=tensorflow \ —output_dir=demos/mnist/ \ —checkpoint_file=/tmp/tensorflow/mnist/logs/fully_connected_feed/model.ckpt-1999
# Will save a lot of files # (one file per variable and manifest.json)
Porting weights
import {CheckpointLoader} from 'deeplearn';
const varLoader = new CheckpointLoader('.');varLoader.getAllVariables().then(async vars => { const math = new NDArrayMathGPU(); const hidden1W = vars[‘hidden1/weights’] as Array2D; const hidden1b = vars[‘hidden1/biases’] as Array1D; //…
}
GPU Acceleration
WebGL
WebGL (Web Graphics Library) is a JavaScript API for rendering interactive 3D and 2D graphics within any compatible web browser without the use of plug-ins.
- Mozilla Developer Network
WebGL
- Provides the standard API - Implemented by major web browsers such as Chrome, Firefox, Safari and Opera - Hardware Agnostic
WebGL Graphic Pipeline
https://www.tutorialspoint.com/webgl/webgl_graphics_pipeline.htm
WebGL Graphic Pipeline
http://math.hws.edu/graphicsbook/c6/s4.html
WebGL
1. deeplearnjs compiles GLSL shader 2. Send input NDArray as texture into GPU 3. Run a fragment shader 4. Fetch result NDArray from GPU
1. Compile Shader
float unaryOperation(float x) {return (x < 0.0) ? 0.0 : x;
}
void main() {float x = getAAtCoords(); // Sample a value from a texturefloat y = unaryOperation(x);
setOutput(y);}
2. Upload NDArray to GPU
gl.bindTexture(gl.TEXTURE_2D, texture);gl.texSubImage2D( gl.TEXTURE_2D, 0, 0, 0, width, height, textureFormat, getTextureType(gl), data);
// Texture format is RGBA// Texture type is gl.UNSIGNED_BYTE or gl.FLOAT
gl.bindTexture(gl.TEXTURE_2D, null);
3. Run shader program
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
// Map buffer data to texture coordinationgl.vertexAttribPointer( loc, arrayEntriesPerItem, gl.FLOAT, false, itemStrideInBytes, itemOffsetInBytes);
// Draw somethinggl.drawElements(gl.TRIANGLES, 6, gl.UNSIGNED_SHORT, 0);
4. Download the result
getBufferSubDataAsyncExtension.getBufferSubDataAsync( gl2.PIXEL_PACK_BUFFER, 0, downloadTarget);
// Download as Typed Array -> NDArray// Typed Array is backed by raw buffer memory (ArrayBuffer) // accessed through views (e.g. Float32Array)
https://www.khronos.org/registry/webgl/extensions/WEBGL_get_buffer_sub_data_async/
Future Works
Automatic TensorFlow to deeplearn.js
Direct importing from TensorFlow GraphDef format makes model importing easier.
https://deeplearnjs.org/docs/roadmap.html
Eager Mode
Eager execution mode enables us to run computation without Graph definition.
Similar to TensorFlow eager mode.
Decoupling NDArray from storage mechanism
NDArray is now tightly coupled with storage layer (GPU texture)
Tracking NDArray to avoid memory leak often becomes hard work.
Great Demos
One of the reason we use deeplearn.js is that it enables us to create interesting application by using deep learning easily.
deeplearn.js is now enhancing demos to show the potential of deep learning on Web Browser.
Patches Welcome!
top related