Download - eHarmony @ Phoenix Con 2016
![Page 1: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/1.jpg)
![Page 2: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/2.jpg)
V I JAY VA N G A PA N D U
@ v i j a y v a ng a p a nd uv i j ay ku marv a ng a p a nd u
![Page 3: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/3.jpg)
W H O A R E W E ?
![Page 4: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/4.jpg)
EH AR M O NY CR EAT E STH E H A PP I ES T ,M O S T PAS S I O NAT EAND M O S T FU L F I L L I NG R EL AT I O NS H I PS *
*ACC OR D I NG TO A R E C E N T ST U DY
![Page 5: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/5.jpg)
4 3 8M A R R I AG E S P E R DAY
![Page 6: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/6.jpg)
![Page 7: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/7.jpg)
150 questions
PersonalityValuesAttributesBeliefs
![Page 8: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/8.jpg)
M ATC H I N G S Y S T E M
Compatibility Matching System®
C OM PAT IB I L I T Y MATC H ING A F F IN I T Y MATC H ING MATC H
D IS T R I B U T I ON
![Page 9: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/9.jpg)
IntellectEnergy
SociabilityAmbition
KindnessCuriosity
HumorSpiritualit
y
![Page 10: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/10.jpg)
![Page 11: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/11.jpg)
U S E R U P DAT E S
![Page 12: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/12.jpg)
M ATC H D E L I V E RY (V 1 )
M A P-S I DE J O I N S( TB ) SC OR I N G
VOL DE M ORT
MATC H DATA S E RV I CE
MATC H I NG S YS TE M
30+ M I LL I ON E VE NT S
65+ M I L L I ON U SE RS30+ B I L L I ON R E C ORD S
![Page 13: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/13.jpg)
VO L D E M O RT ?T H AT N A M E
T H AT N A M ES O U N D S FAM I L I A R
![Page 14: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/14.jpg)
VO L DE M O RT
AUT OPART I T I O N I NG
P LU G GAB L E SE R IAL I ZAT IO N
AU T OR EP L IC AT I O N
KE Y-VA LUE DY NAM O
G O SS I P
![Page 15: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/15.jpg)
U S E R U P DAT E S O ( N ^ 2 )
* USE R UPDAT ED T HE AD DR E SS
![Page 16: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/16.jpg)
N E E D F O R S C A L AB I L I T Y
VOL DE MORT
30+ Million Match Events / Day
30+ Billion Match Records
Millions of user generated Events / Day
Low latency user requests
1 . 4G B / M IN (14 )
![Page 17: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/17.jpg)
N E E D F O R S C A L AB I L I T YG E T M ATC H E S R E S P ON S E T I M E S
![Page 18: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/18.jpg)
DATA STORE NEEDS
QU E R I E S
LOW L AT E N CY
C R U DOP E RAT I ON S F I LT E R I N G
T H R O U G H P UT40+ M I LL I O N
W R I T E S
30+ B I LL I ONR E C OR D S
![Page 19: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/19.jpg)
DATA STORE NEEDS
E AS Y TOM A I NTA I N
C ON S IS T E N CY AVA I L AB LEPA RT I T I ON
T O LE RA N C E
![Page 20: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/20.jpg)
BREAKING CAP ?
Consistency
Availability PartitionTolerance
CA CP
AP
MongoDBHBaseRedis
CassandraDynamoDB
Riak
RDBMSKAFKA
![Page 21: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/21.jpg)
L A M B D
• Robust and fault-tolerant system• Serves a wide range of workloads and use cases• linearly scalable• Layered Architecture Batch Layer Query Layer Speed Layer
- Nathan Marz
![Page 22: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/22.jpg)
L A M B D
BATCH L AY ER
QU
ERY
LAYE
R
S P E E D / S AV E L AY E R
M A P - S I D E J O I N S( T B ) SCOR I N G
M ATCH I N G S Y S T E M
M ES S AG EBROK ER
BATC H S TORAG E
S P EE D S TORAG E
MER
GE
![Page 23: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/23.jpg)
DATA STORE EVALUATION
![Page 24: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/24.jpg)
C R UDOP ERAT I ON S
THR O UGH PUT40 + M I LL IO N
W R IT ES30+ B I LL I ON
R ECO RDSEA SY T O
M A I NTA I N
CONS IS TE NCY PART IT IO NTO LERA N CE
AVA I L AB I L I T Y
HBASE AS BATCH STORE
![Page 25: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/25.jpg)
T H R O UG H P UT40+ M I LL I O N
W R I T E SC ON S IS T E N CY
AVA I L AB LE
PA RT I T I ONT OL E RA N C E
KAFKA AS BROKER
![Page 26: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/26.jpg)
C ON S IS T E N CY
PA RT I T I ONT O LE RA N C E
AVA I L AB LE
REDIS AS SPEED STORAGE
LO W L AT E N CY
![Page 27: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/27.jpg)
C R U DOP E RAT I ON S
E AS Y TOM A I NTA I N
AS SQL LAYER
QU ER IESI N DE X I N G
T RAN S ACT I O N SM ULT I T E N AN CY
![Page 28: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/28.jpg)
C R U DOP E RAT I ON SE AS Y TO
M A I NTA I N
PHO LIBRARY
QU ER IES
F I LT E R I N G
![Page 29: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/29.jpg)
PHO LIBRARYCONFIGURATION
ANNOTATE THE ENTITY BEAN
@Entity(value="user_matches")public class MatchDataFeedItemDto implements Serializable {
@Embedded private MatchCommunicationElement communication;@Embedded private MatchElement match;
@Property(value = "UID") private long storeUserIdKey; @Property(value = "MID") private long matchId;}
REGISTER THE BEAN
<util:list id="entityPropertiesMappings"> <value>com.eharmony.datastore.model.MatchDataFeedItemDto</value>
</util:list><bean id="entityPropertiesMappingContext" class="com.eharmony.datastore.mapper.EntityPropertiesMappingContext">
<constructor-arg ref="entityPropertiesMappings"/></bean><bean id="entityPropertiesResolver" class="com.eharmony.datastore.mapper.EntityPropertiesResolver">
<constructor-arg ref="entityPropertiesMappingContext"/></bean><bean id="phoenixHBaseQueryTranslator" class="com.eharmony.datastore.hbase.translator.PhoenixHBaseQueryTranslator">
<constructor-arg name="propertyResolver" ref="entityPropertiesResolver" /></bean><bean id="phoenixHBaseQueryExecutor" class="com.eharmony.datastore.hbase.query.executor.PhoenixHBaseQueryExecutor">
<constructor-arg name="queryTranslator" ref="phoenixHBaseQueryTranslator"/> <constructor-arg name="resultMapper" ref="phoenixProjectedResultMapper" />
</bean>
![Page 30: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/30.jpg)
PHO LIBRARYQUERY BUILDING
Disjunction disjunction = new Disjunction(); for (int statusFilter : statusFilters) { disjunction.add(Restrictions.eq("status", statusFilter)); } QueryBuilder.builderFor(FeedItemDto.class).select() .add(Restrictions.eq("userId", userId)) .add(Restrictions.gte("spotlightEnd", spotlightEndDate)) .add(disjunction) .setReturnFields(projection) .addOrder(orderings) .setMaxResults(maxResults) .build();
![Page 31: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/31.jpg)
http://eharmony.github.io/
![Page 32: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/32.jpg)
L A M B D
KA F KA
M A P - S I D E J O I N S( T B ) SCOR I N G BATCH L AY ER
QU E RY L AY E R
S P E E D / S AV E L AY E R
M ATCH I N G S Y S T E M
![Page 33: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/33.jpg)
P E R F O R M A N C E
HBA SE CU TOVE R
S AV E M ATC H R ES P ON S E T I M ES
50% 10 0%
G E T M ATC H E S R E S PON SE T I ME S
HBA SE CUTOV ER 10 0%
![Page 34: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/34.jpg)
SO M E IN S IGH T
![Page 35: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/35.jpg)
C H A L L E N G E S
H O T R E G I O N M I G RAT I O N
![Page 37: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/37.jpg)
http://www.eharmony.com/about/careers/
![Page 38: eHarmony @ Phoenix Con 2016](https://reader035.vdocuments.pub/reader035/viewer/2022081507/587065971a28ab48378b4e93/html5/thumbnails/38.jpg)
T H A N K YO UQ U E S T I O N S ?
@ v i j a y v a ng a p a ndu