NoSQL Solutions

Byeongweon Moontasyblue@gmail.com

2012.01.26

Redis, Couchbase, MongoDB, Mem-base

Memory Base Cache vs. Data-base

NoSQL

• Collections (versus tables)• Documents (versus rows)• Loosely defined fields (versus columns)

• Scale out (versus scale up)• Denormalization (versus normal-ization)

Data Model

• Relational • Key-value • Column-oriented• Document-oriented

Relational

Key-value

Column-oriented

Document-oriented

KEY

FirstName="Jonathan", Address="15 Wanamassa Point Road", Children=[

{Name:"Michael",Age:10}, {Name:"Jennifer", Age:8}, {Name:"Samantha", Age:5}, {Name:"Elena", Age:2}]

Memory Base Cache

• Weak Persistence• Weak Consistency• Strong Performance• Low Latency

Membase

• Based on Memcached• Written by C++(Memcached), Er-lang(Membase)

• Distributed, in-memory key-value database management sys-tem

• Optimized for storing data be-hind web applications

Membase (cont’)

• Persistence– Asynchronously writes data to disk after ac-knowledging write to client

– Guarantees data consistency

• Replication and failover (server failures recoverable in under 100ms)

• Scalability and performance– Distributed object store– Dynamic cluster resizing and rebalancing– Guaranteed data consistency– High sustained throughput– Low, predictable latency

Redis

• In-memory, key-value data store• Written by ANSI C, support var-ious client

Redis (cont’)

• Various Data Models– List, Set, Sorted Set, Hash– Support atomic operation about data types

• Persistence– Data is held in memory but written to disk asynchronously

• Replication– Master-Slave replication

• Performance– Non-blocking I/O. Single threaded

• Publish/Subscribe

Membase vs. Redis

Membase Redis

String Set, List, Sorted List, Hash..

Master-Master Master-Slave

Storing, inc/dec API Various operations includes pop, push, extract …

Web management UI Console management tool

How to use…

• Normally use …

$key = md5('SELECT * FROM rest_of_sql_statement_goes_here');if ($memcache->get($key)) { return $memcache->get($key);}else { $result = $query_results_mangled_into_most_likely_an_array $memcache->set($key, $result, TRUE, 86400); return $result;}

How to use … (cont’)

• Structured Data (array, struct…)– Serialize

– Normalization

user:$user_id

name:문병원 |call:하겐다즈 |…

KEY VALUE

user:$user_id:name 문병원

KEY VALUE

user:$user_id:call 하겐다즈

Application Design using Mem-base

• Cache result other than SQL data!• Use a cache hierarchy• Update membase as your data up-dates

• Race conditions and stale data• Pre warm your cache• Storing lists with keys• Batch your requests with get_multi

From memcached FAQ

Database

• Persistence• Reliable• Scalable• Distributed• Clustered

MongoDB

• Document-oriented Storage• High Write Performance• Full index support• Master/Slave Replication• Support Map/Reduce• Auto-Sharding• Querying• GridFS• Written in C++

CouchDB

• Document-oriented Storage• High Read Performance• ACID Semantics• Map/Reduce View and Indexes• Distributed Architecture with Replication

• REST API• Eventual Consistency• Written in Erlang

CASE STUDY

Twitter

Facebook Timeline

APPENDIX

CouchDB MongoDB  MySQL

Data Model Document-Oriented (JSON)

Document-Oriented (BSON) Relational

Data Types string, number, boolean, array, ob-ject

string, int, dou-ble, boolean, date, byte array, object, array, others 

Various Types Link

Large Ob-jects (Files) 

Yes (attachments)  Yes (GridFS)  BlobZ

Horizontal partition-ing scheme 

CouchDB Lounge  Auto-sharding Partitioning 

Replica-tion 

Master-master (with developer supplied conflict resolu-tion) 

Master-slave and replica sets

Master-slave, multi-master, and circular replication

Object(row) Storage 

One large reposi-tory  Collection-based  Table-based 

Query Method 

Map/reduce of javascript functions to lazily build an index per query 

Dynamic; ob-ject-based query language 

Dynamic; SQL 

Secondary Indexes  Yes Yes Yes

Atomicity Single docu-ment 

Single docu-ment 

Yes - ad-vanced 

Interface  REST  Native drivers ; REST add-on 

Native driv-ers 

Server-side batch data manipulation  Map/Reduce

Map/Reduce, server-side javascript 

Yes (SQL) 

Written in  Erlang C++ C++ 

Distributed Consistency Model

Eventually consistent (master-master replication with version-ing and ver-sion reconcil-iation)

Strong consis-tency.  Eventu-ally consistent reads from sec-ondaries are available. 

Strong consis-tency.  Even-tually consis-tent reads from secon-daries are available.

References• NoSQL solutions: Membase, Redis, CouchDB and MongoDB : http

://blog.fedecarg.com/2011/01/25/nosql-solutions-membase-redis-couchdb-and-mongodb/

• Visual Guide to NoSQL Systems : http://blog.nahurst.com/visual-guide-to-nosql-systems

• MongoDB, CouchDB, MySQL Compare Grid : http://www.mongodb.org/display/DOCS/MongoDB,+CouchDB,+MySQL+Compare+Grid

• SQL to Mongo Mapping Chart : http://www.mongodb.org/display/DOCS/SQL+to+Mongo+Mapping+Chart

• Memcached FAQ : http://code.google.com/p/memcached/wiki/FAQ#Simple_query_result_caching

• Couchbase 2.0 Manual : http://docs.couchbase.org/couchbase-manual-2.0.pdf

• Building Timeline : Facebook http://www.facebook.com/notes/facebook-engineering/building-timeline-scaling-up-to-hold-your-life-story/10150468255628920