nosql y mongodb

Bases de datos NoSQL y MongoDBDavid Gómez

miércoles 25 de enero de 12

2

David Gómez

[email protected] & Trainer

@dgomezg


mailto:[email protected]





















Un vistazo a las soluciones NoSQL

3


¿Qué es NoSQL?

4

(cc) Photo by mwin - http://www.flickr.com/photos/wingler/3429634150


http://www.flickr.com/photos/wingler/3429634150


¿Qué es NoSQL?

4

(cc) Photo by mwin - http://www.flickr.com/photos/wingler/3429634150

NoSQL !== no SQL

Not only SQL




¿Qué es NoSQL?

Non Relational

Distributed

Horizontally scalable

5


Motivaciones

6


Motivaciones

Aplicaciones web ultra-escalables

Gran cantidad de datos

Velocidad de guardado

Alto número de peticiones

Máxima disponibilidad

6


Características

Replicado (particionado horizontal)

Sin esquemas (versatibilidad, adaptabilidad)

BASE (not ACID)

7


Características: BASE vs ACID

Atomicity, Consistency, Isolation, Durability

Basically Available, Soft state, Eventual consistency

8

(cc) Photo by zhouxuan12345678 - http://www.flickr.com/photos/53921113@N02/5645102295/


http://www.flickr.com/photos/53921113@N02/5645102295/

http://www.flickr.com/photos/53921113@N02/5645102295/

9

orientadas a Columna orientadas a Documento

orientadas a GrafosKey-value

Tipos


Orientadas a columna

10

(cc) Photo by profzuckerk - http://www.flickr.com/photos/profzucker/4630958694


http://www.flickr.com/photos/profzucker/4630958694



10

(cc) Photo by profzuckerk - http://www.flickr.com/photos/profzucker/4630958694

Configuración individual de columnas por fila





11

(cc) Photo by Ash Brian - http://www.flickr.com/photos/ashbrian/2977673855


También conocidas como BigTable (google paper)


http://www.flickr.com/photos/ashbrian/2977673855



11

(cc) Photo by Ash Brian - http://www.flickr.com/photos/ashbrian/2977673855


También conocidas como BigTable (google paper)

Tamaño

Alta carga de escrituras

Disponibilidad

MapReduce




Orientadas a DocumentoEl documento es lo que importa

Almacenados en Colecciones

(bajo una clave)

Los documentos pueden ser heterogéneos

Formatos textuales: XML, YAML, JSON & BSON

Formatos binarios: P ej: PDFs

12(cc) Photo by Ampersand Duck - http://www.flickr.com/photos/ampersandduck/4941185476


http://www.flickr.com/photos/ampersandduck/4941185476

http://www.flickr.com/photos/ampersandduck/4941185476

Key-Value

13

Basadas en:

Dynamo (paper de Amazon)

y Tablas Hash distribuidas

Colleción de pares clave-valor

Almacenados de forma distribuida

Útiles con:

muchas escrituras y lecturas pequeñas.


Orientadas a Grafos

14

Almacenan:

Nodos y relaciones

Ambos pueden contener pares clave-valor

Muy rápidos para consultas con operaciones sobre el gráfo(p. Ej: personas con un grado de afinidad 3)


¿Cuándo utilizar NoSQL?Teorema CAP (Eric Brewer)

15Source: Nathan Hurst’s Blog



15

CConsistency

Source: Nathan Hurst’s Blog



15

C AConsistency Availability




15

C A

P

Consistency Availability

Partition Tolerance




15

C A

P


Partition Tolerance

Sólo 2




15

C A

P


Partition Tolerance

Sólo 2

Oracle, MySQL,




15

C A

P


Partition Tolerance

Sólo 2

Oracle, MySQL,

MongoDB, DataStore,

HyperTable, HBase

Redis, MemcacheDB




15

C A

P


Partition Tolerance

Sólo 2

Oracle, MySQL,

MongoDB, DataStore,

HyperTable, HBase

Redis, MemcacheDBDyn

amo,

Volde

mort,

Cas

sand

ra,

Sim

pleDB, C

ouch

DB, RiaK

RDBMSOrientadas a documento

orientadas a columnaKey-Value



NoSQL Document DB:MongoDB

16


MongoDB

17

Escrita en C++

Orientada a documento

Formato JSON (o BSON)

Un poco SQL (queries, índices, Referencias externas)

Particionado horizontal (sharding)

Consultas Javascript

Almacenamiento en GridFS


Orientadas a Documento

Documentos formateados en JSON

18


Orientadas a Documento

Documentos formateados en JSON

18

{ "_id" : { "$oid" : "4f13f2b98de23dc3d58f959a" }, "name" : "NoSQL y MongoDB", "presenter" : "David Gomez", "Event" : "decharlas", "duration" : 60, "timing" : { "start" : 17, "end" : 19 } }


Consultas

Utilizando un API y funciones JavaScript

19


beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test>

Consultas


19



Consultas


19

beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground>



Consultas


19


beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground> db.talks.insert({'name' : 'codemotion', 'presenter' : 'David Gomez', 'Event' : 'decharlas' });>



Consultas


19


beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground> db.talks.insert({'name' : 'codemotion', 'presenter' : 'David Gomez', 'Event' : 'decharlas' });>

beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground> db.talks.insert({'name' : 'codemotion', 'presenter' : 'David Gomez', 'Event' : 'decharlas' });> db.talks.find();{ "_id" : ObjectId("4f13f7e5a00e6496e7a9541b"), "name" : "codemotion", "presenter" : "David Gomez", "Event" : "decharlas" }>


Colecciones

Equivalente a la tabla

Guardan documentos

No todos iguales

Pueden definir índices sobre atributos

Creadas con la primera inserción

Persistidas sobre un GridFS

20


Documentos

Identificados por un “_id” (generado o asignado)

Textual (JSON)

Binario (BSON)*

Documentos grandes divididos en chunks

21


Programación de clientes

Multitud de lenguajes (drivers)

Transformación a JSON realizada por el driver

22





22

{ "name" : "NoSQL y MongoDB", "presenter" : "David Gomez", "Event" : "decharlas", "duration" : 60, "timing" : { "start" : 17, "end" : 19 } }

sample.py





22


sample.py

{ "name" => "NoSQL y MongoDB", "presenter" => "David Gomez", "Event" => "decharlas", "duration" => 60, "timing" => { "start" => 17, "end" => 19 } }

Ruby





22


sample.py


Ruby

array( "name" => "NoSQL y MongoDB", "presenter" => "David Gomez", "Event" => "decharlas", "duration" => 60, "timing" => array( "start" => 17, "end" => 19 )}

sample.php





22


sample.py


Ruby

array( "name" => "NoSQL y MongoDB", "presenter" => "David Gomez", "Event" => "decharlas", "duration" => 60, "timing" => array( "start" => 17, "end" => 19 )}

sample.php

BasicDBObject doc = new BasicDBObject(); doc.put("name", "NoSQL y MongoDB"); doc.put("presenter", "David Gomez"); doc.put("Event", "decharlas"); doc.put("duration", 60);

BasicDBObject timing = new BasicDBObject(); timing.put("start", 17); timing.put("end", 19); doc.put("timing", timing);

sample.java


1. Obtener el Driver

Importar el driver

23


1. Obtener el Driver

Importar el driver

23

<dependency><groupId>org.mongodb</groupId><artifactId>mongo-java-driver</artifactId><version>2.7.0-rc1</version>

</dependency>


2. ConexiónObtener la referencia a la BD y la conexión

24



24

Mongo m = new Mongo( "127.0.0.1" , 27017 );

DB db = m.getDB( "playground" );



24

Mongo m = new Mongo( "127.0.0.1" , 27017 );


Sobre la BD se puede:



24

Mongo m = new Mongo( "127.0.0.1" , 27017 );


Sobre la BD se puede:Boolean authorized = db.authenticate(user, passwd);if (db.collectionExists("talks")) {

CommandResult cmdResult = db.command("{ collStats:"db.talks" , scale : 1 }");

} Set<String> colls = db.getCollectionNames();for (String s : colls) { System.out.println(s);}


3. Inserciones

Crear e insertar un objeto

25


3. Inserciones

Crear e insertar un objeto

25

BasicDBObject doc = new BasicDBObject(); doc.put("name", "NoSQL y MongoDB"); doc.put("presenter", "David Gomez"); doc.put("Event", "decharlas"); doc.put("duration", 60);

BasicDBObject timing = new BasicDBObject(); timing.put("start", 17); timing.put("end", 19); doc.put("timing", timing);

DBCollection coll = db.getCollection("talks"); coll.insert(doc);


4. Consultas (I)

Obtener todos los documentos de una colección

26


4. Consultas (I)


26

DBCollection talks = db.getCollection("talks");

DBCursor cursor = talks.find();

while (cursor.hasNext()) { System.out.println(cursor.next()); }


4. Consultas (I)


26

DBCollection talks = db.getCollection("talks");

DBCursor cursor = talks.find();

while (cursor.hasNext()) { System.out.println(cursor.next()); }

public abstract class DBCollection {

public final DBCursor find(){ public final DBCursor find( DBObject ref ) public final DBCursor find( DBObject ref , DBObject keys )

public final DBObject findOne( Object obj ) public final DBObject findOne( Object obj, DBObject fields )

public DBObject findAndModify( DBObject query , DBObject update )

}


4. Consultas (II)

Consultas “SQL-like”

27


4. Consultas (II)


27

SELECT * FROM TALKS WHERE PRESENTER = “David Gomez”;

Las consultas se hacen con Documentos JSON


4. Consultas (II)


27


{ “presenter” : “David Gomez” }



4. Consultas (II)


27




beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground> db.talks.find( { "presenter" : "David Gomez" });{ "_id" : ObjectId("4f13f7e5a00e6496e7a9541b"), "name" : "codemotion", "presenter" : "David Gomez", "Event" : "decharlas" }>


4. Consultas (II)


27




beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground> db.talks.find( { "presenter" : "David Gomez" });{ "_id" : ObjectId("4f13f7e5a00e6496e7a9541b"), "name" : "codemotion", "presenter" : "David Gomez", "Event" : "decharlas" }>

DBObject query = new BasicDBObject();query.put("presenter", "David Gomez"); DBCursor davidTalks = talks.find(query);while (davidTalks.hasNext()) { System.out.println(davidTalks.next());} }


4. Consultas (III)

Comparadores <, <=, >=, >, !=

28


4. Consultas (III)


28

SELECT * FROM TALKS WHERE duration < 60;


4. Consultas (III)


28


{ “duration” : { $lt : 60} }{ “duration” : { $lte : 60} }{ “duration” : { $gt : 60} }{ “duration” : { $gte : 60} }{ “duration” : { $ne : 60} }


4. Consultas (III)


28



beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground> > db.talks.find({"duration" : { $lt : 90 } }){ "_id" : ObjectId("4f1402f28de2930293c7cf8e"), "name" : "NoSQL y MongoDB", "presenter" : "David Gomez", "Event" : "decharlas", "duration" : 60, "timing" : { "start" : 17, "end" : 19 } }>


4. Consultas (III)


28



beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use playgroundswitched to db playground> > db.talks.find({"duration" : { $lt : 90 } }){ "_id" : ObjectId("4f1402f28de2930293c7cf8e"), "name" : "NoSQL y MongoDB", "presenter" : "David Gomez", "Event" : "decharlas", "duration" : 60, "timing" : { "start" : 17, "end" : 19 } }>

DBObject query = new BasicDBObject();query.put("duration", new BasicDBObject("$lt","90")); DBCursor shortTalks = talks.find(query);while (shortTalks.hasNext()) {

System.out.println(shortTalks.next());}


4. Consultas (IV)

Concatenación de consultas

29


4. Consultas (IV)


29

SELECT * FROM TALKS WHERE duration < 90 AND duration > 30;


4. Consultas (IV)


29


{ “duration” : { $lt : 90, $gt : 60} }}


4. Consultas (IV)


29


{ “duration” : { $lt : 90, $gt : 60} }}

DBObject query = new BasicDBObject();DBObject condition = new BasicDBObject("$lt", 90);condition.put("$gt",30);query.put("duration", condition); DBCursor shortTalks = talks.find(query);while (shortTalks.hasNext()) {

System.out.println(shortTalks.next());}


4. Consultas (V)

Otros operadores

30


4. Consultas (V)

Otros operadores

30

$all$exists$mod$ne$in$nin$nor$or$and$size$type


http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24all

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24all

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24exists

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24exists

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24mod

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24mod

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24ne

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24ne

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24in

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24in

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24nin

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24nin

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24nor

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24nor

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24or

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24or

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24and

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24and

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24size

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24size

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24type

http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24type

4. Consultas (y VI)


31


4. Consultas (y VI)


31

SELECT * FROM TALKS WHERE duration < 90 AND duration > 30AND (presenter = “David Gomez” or presenter=”Ricardo Borillo”);


4. Consultas (y VI)


31


{ “duration” : { $lt : 90, $gt : 60}, $or : [ { “presenter” : “David Gomez”}, { “presenter” : “Ricardo Borillo”} ]}


4. Consultas (y VI)


31


{ “duration” : { $lt : 90, $gt : 60}, $or : [ { “presenter” : “David Gomez”}, { “presenter” : “Ricardo Borillo”} ]}

DBObject query = new BasicDBObject();DBObject condition = new BasicDBObject("$lt",90);condition.put("$gt",30);query.put("duration", condition);

DBObject [] speakerCond = {new BasicDBObject("presenter", "David Gomez"), new BasicDBObject("presenter", "Ricardo Borillo")

};query.put("$or", speakerCond);

talks.find(query);


DBObjects desde JSON

32

com.mongo.util.JSON


DBObjects desde JSON

32

Object query = JSON.parse("{ 'duration' : { $lt : 90, $gt : 60}, " + " '$or' : [ { 'presenter' : 'David Gomez'}, " + " { 'presenter' : 'Ricardo Borillo'} ] }");

DBCursor cursor = db.find((DBObject) query);

com.mongo.util.JSON


Spring Data MongoDB

33


Spring Data

Objetivo:- Proporcionar un mecanismo homogeneo y

completo para el acceso a BD NoSQL

Multiples proyectos:- Soportados:

‣ MongoDB, Redis, Neo4j, Hadoop, GemFire, RiaK

- En proceso:‣ CouchDB, Hbase, Cassandra

34


Spring Data y MongoDB

XML namespace para configurar driver Mongo

MongoTemplate

Conversión automática de excepciones

OXM Configurable

JMX monitoring

35


Importando Spring Data

36


Importando Spring Data

36

<repository> <id>spring-milestone</id> <name>Spring Maven MILESTONE Repository</name> <url>http://maven.springframework.org/milestone</url></repository>

<dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-mongodb</artifactId> <version>1.0.0.RC1</version></dependency>


http://maven.springframework.org/milestone

http://maven.springframework.org/milestone

Namespace

37


Namespace

37

<?xml version="1.0" encoding="UTF-‐8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-‐instance" xmlns:mongo="http://www.springframework.org/schema/data/mongo" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-‐beans.xsd http://www.springframework.org/schema/data/mongo http://www.springframework.org/schema/data/mongo/spring-‐mongo-‐1.0.xsd

http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-‐context.xsd">

</beans>


http://www.springframework.org/schema/beans


http://www.w3.org/2001/XMLSchema-instance


http://www.springframework.org/schema/data/mongo


http://www.springframework.org/schema/context




http://www.springframework.org/schema/beans/spring-beans.xsd




http://www.springframework.org/schema/data/mongo/spring-mongo-1.0.xsd




http://www.springframework.org/schema/context/spring-context.xsd


Namespace

37




</beans>

<mongo:db-‐factory dbname="vts"/>






















Namespace

37




</beans>

<mongo:db-‐factory dbname="vts"/>

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> <constructor-‐arg name="mongoDbFactory" ref="mongoDbFactory" /> </bean>






















MongoTemplate

38

Convierte query a JSON

Convierte respuesta (“_type”)

Gestiona las conexiones

Maneja las excepciones

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> <constructor-‐arg name="mongoDbFactory" ref="mongoDbFactory" /> </bean>


Repository

39


Repository

39

public abstract class AbstractRepository<T> {

@Inject protected MongoTemplate mongoTemplate; /** Clase que utiliza Mongo para identificar el nombre de la coleccion */ private Class<T> persistentClass;

protected AbstractService(Class<T> persistentClass) { this.persistentClass = persistentClass; }

public List<T> findAll(){ return mongoTemplate.findAll(persistentClass); }

public void save(T e) { mongoTemplate.save(e); }

public T findById(String id) { T t = mongoTemplate.findById(id, persistentClass); if (t == null) { logger.debug(persistentClass.getSimpleName() + " with id " + id + " could not be found"); throw new ObjectNotFoundException(persistentClass, id); } return t; }

}


Consultas más sencillas

40


Consultas más sencillas

40

public abstract class AbstractRepository<T> {

public List<T> getByType(String type) { return mongoTemplate.find(

new Query(Criteria.where("type").is(type)),T.class, persistentClass);

}

}


Conversión Manual

41

Porque a veces lo automático no funciona


Conversión Manual

41

Porque a veces lo automático no funcionapublic class BitSetReadConverter implements Converter<DBObject, BitSet> {

@Override public BitSet convert(DBObject source) { BasicDBList words = (BasicDBList)source.get("words"); BitSet bitset = new BitSet(); int index = 0; for (Object word : words) { long value = (Long)word; while (value != 0L) {

if ((value & 1L) != 0) { bitset.set(index); } ++index; value = value >>> 1; } } return bitset; }}


Conversión Manual

42



Conversión Manual

42


<?xml version="1.0" encoding="UTF-‐8"?><beans>

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> <constructor-‐arg name="mongoDbFactory" ref="mongoDbFactory" /> <constructor-‐arg name="mongoConverter" ref="mappingConverter" /> </bean> <mongo:db-‐factory dbname="vts"/> <mongo:mapping-‐converter> <mongo:custom-‐converters> <mongo:converter> <bean class="com.vts.db.BitSetReadConverter"/> </mongo:converter> </mongo:custom-‐converters> </mongo:mapping-‐converter>

</beans>


Conclusiones

43

Reducción del número de tablas

Optimización en el rendimiento de escrituras

Simplificación de los DAOs con Spring-Data

Otras ventajas:

Documentos en formato de Respuesta REST


Q&A

44


45


nosql y mongodb

Business