in-memory computing in modern data architecture

©2015Slide 1

In-­Memory Computing in Modern Data Architecture

Milind Bhandarkar, PhDFounder & CEOAmpool Inc.

(http://ampool.io)(@techmilind)

About Mehttp://www.linkedin.com/in/milindb

Founding member of Hadoop team at Yahoo! [2005-­2010]

Contributor to Apache Hadoop since v0.1

Built and led Grid Solutions Team at Yahoo! [2007-­2010]

Parallel Programming Paradigms [1989-­today] (PhD cs.illinois.edu)

Center for Development of Advanced Computing (C-­DAC), National Center for Supercomputing Applications (NCSA), Center for Simulation of Advanced Rockets, Siebel Systems (acquired by Oracle), Pathscale Inc. (acquired by QLogic), Yahoo!, LinkedIn, Pivotal (formerly Greenplum), and Ampool Inc.

Hadoop Midwife ;;-­)

About AmpoolEarly stage stealth mode startup based in Silicon Valley

Ampool India Pvt Ltd started August 3

Building next generation Big Data infrastructure products

Focus on Open Source Software & Modern Hardware

Co-­Founders: Robert Geiger, Nitin Lamba, & Dr. R. K. Singh

http://ampool.io

©2015Slide 5

Once upon a time, in a land far far away…

©2015Slide 6

©2015Slide 7

Fast forward 15 years..

©2015Slide 8

©2015Slide 9

What Happened ?

©2015Slide 10

©2015Slide 11

©2015Slide 12

©2015Slide 13

©2015Slide 14

©2015Slide 15

©2015Slide 16

And, then…

HDFS

MapReduce

©2015Slide 17

In a blink of an eye…

HDFS

Pig

Sqoop Flume

Coordination and workflow management

Zookeeper AmbariGemFire XDOozie

MapReduce

Hive

TezGiraph

Hadoop UI

Hue

SolrCloudPhoenix

HBase

Crunch Mahout

Spark

Shark

Streaming

MLib

GraphX

HAW

Q

SpringXD

MADlibHam

ster

YARN

©2015Slide 18

Technology Trends

Infrastructure is increasingly Scale-­Out

Memory throughput growth

Memory hierarchy getting deeper

One-­size-­fits-­all Data Platform Era is Over

Analytics moving from Batch to Real-­Time

Then…

… and Now

©2015Slide 26

©2015Slide 29

Questions ?

We Are Hiring in Pune!(Send CV to jobs@ampool.io)