zentrales logging mit dem elastic stack - froscon€¦ · zentrales logging mit dem elastic stack...
TRANSCRIPT
Zentrales Logging mit dem Elastic Stack
Simon SchneiderFroscon 2019
Simon Schneider
Zentrales vs Dezentrales Loggingi
Zentrales LoggingDezentrales Logging
Elastic Stacki
Beats
Logstash
Beats Logstash Ingest
Elasticsearch
Kibana
Elasticsearch
Beats Logstash
SaaS Self Managed
Elastic Stack
Visualize & Manage
Store, Search & Analyze
Ingest
DeploymentElastic Cloud Elastic CloudEnterprise
Standalone
Hot Node Warm NodeMaster Node
● Nur Management ● Keine Datenhaltung
● SSD (RAID 0)● Query● Ingest● Letzten 1-3 Tage
● HDD● Query● > 3 Tage
Hot-Warm Architektur
Docker NodesIngest Nodes
Data NodesKibanaNodes
REWE Digital Ecom - Elastic Stack
REWE Digital Big Data - Elastic Stack
https://github.com/elastic/cloud-on-k8s
Live Coding
Web Server● Static HTML● Proxy Route
Such-Backend● Such-Endpunkt● Bleve Search
Filebeat Logstash Elasticsearch
● Access Log Aufbereitung● Geo IP Parser● User Agent Parser
➔ Großer Initialaufwand / Kosten
➔ Hot-Warm Architektur zur Kostensenkung
➔ JSON Logs sind einfach zu verarbeiten
➔ Filebeat + Elasticsearch kann ausreichend sein
➔ Sicherheit, Datenschutz und Schnelligkeit gewonnen
Wrap Up
Thank youSpecial Thanks to:
- Peter Larsen- Paul Puschmann
twitter.com/raynigongithub.com/raynigon
● Verschiedene Logos de.wikipedia.org● RRZE Icon Set● Morning Coffee Photo by David Mao on Unsplash● Shock Photo by David Mao on Unsplash● Work Photo by Glenn Carstens-Peters on Unsplash● Office Photo by Tim van der Kuip on Unsplash● Folders Photo by Samuel Zeller on Unsplash● Light Building Photo by Lysander Yuen on Unsplash● Building a House Photo by Randy Fath on Unsplash● Trophy Photo by Fauzan Saari on Unsplash● Ecommerce by Mein Deal on Unsplash● Big Data by ev on Unsplash● Burning Money by Jp Valery on Unsplash● Rocket by Bill Jelen on Unsplash
Credits