[2d1]elasticsearch 성능 최적화

Download [2D1]Elasticsearch 성능 최적화

Post on 29-Nov-2014

1.460 views

Category:

Technology

2 download

Embed Size (px)

DESCRIPTION

DEVIEW 2014 [2D1]Elasticsearch 성능 최적화

TRANSCRIPT

  • 1. / BigDataPlatform Team ElasticSearch
  • 2. BigdataPlatform, GruterCorp hwjeong@gruter.com http://jjeong.tistory.com E-book: Elasticsearch-
  • 3. 1.ElasticSearch 2.ElasticSearch 3.ElasticSearch CONTENTS
  • 4. 1.ElasticSearch 1.1.ElasticSearch 1.2. 1.3.Modeling
  • 5. ElasticSearch? Lucene 1.1.ElasticSearch ElasticSearch Easy Real time search & analytics Distributed & highly available search engine
  • 6. ElasticSearch Physical Logical 1.1.ElasticSearch Cluster Index Node Node Node Indice Indice Indice Shard Shard Shard Shard Shard Shard Shard Shard Shard Type Type Type Document Document Document field:value field:value field:value field:value field:value field:value field:value field:value field:value [Physical ] [Logical ]
  • 7. ElasticSearchNodes Master node Data node Search load balancer node Client node 1.1.ElasticSearch Master node.master: true Data node.data: true Search LB node.master: false node.data: false Client node.client: true
  • 8. ElasticSearchNodes 1.1.ElasticSearch Case 1) All round player node.master: true node.data: true node.master: true node.data: true node.master: true node.data: true Case 2) Master Data node.master: true node.data: false node.master: true node.data: false node.master: false node.data: true node.master: false node.data: true Case 3) Master Data Search LB node.master: true node.data: false node.master: true node.data: false node.master: false node.data: true node.master: false node.data: true node.master: false node.data: false node.master: false node.data: false
  • 9. ElasticSearchvs RDBMS 1.1.ElasticSearch Relational Database ElasticSearch Database Index Table Type Row Document Column Field Index Analyze Primary key _id Schema Mapping Physical partition Shard Logical partition Route Relational Parent/Child, Nested SQL Query DSL
  • 10. ElasticSearchshard replication 1.1.ElasticSearch POST /my_index/_settings{ "number_of_replicas":1} POST /my_index/_settings{ "number_of_replicas":2} http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/replica-shards
  • 11. Creating, indexing and deleting a document 1.1.ElasticSearch http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/distrib-write.html
  • 12. Retrieve, query and fetch a document 1.1.ElasticSearch http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/distrib-read.html http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_query_phase.html http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_fetch_phase.html
  • 13. 1.2. Create index Add document Get document Search document
  • 14. Indice/type design Time-based/User-based data Relational data 1TB 1.3.Modeling Field design Primary key
  • 15. Modeling 1.3.Modeling Indice1 Indice2 Indice3 IndiceA IndiceB IndiceC Type Parent Type Child Type Parent Type Child Type Child Type 1 : N 1 : N 1 : N
  • 16. Shard design number_of_shards>= number_of_data_nodes number_of_replica 20140624235959 ORDER BY search_fieldDESC "range": { "search_field": { "gt": "20140624235959" } } "sort": [ { "search_field": { "order": "desc" } } ]
  • 47. SQL on ElasticSearch Demo
  • 48. ElasticSearch Lucene ElasticSearch . modelingsizing. . Queryfilter. Bulk API. ElasticSearch HadoopSQL.
  • 49. Q&A E-mail : sophistlv@gmail.com
  • 50. THANK YOU