手把手教你如何串接 log 到各種網路服務
Post on 09-Jan-2017
1.047 Views
Preview:
TRANSCRIPT
手把手教你如何串接 log 到各種網路服務Kewang
Who I am
● 王慕羣● Java / Node.js / AngularJS● HBase / SQL-like● Git / DevOps
kewang @ GitHubkewang.information @ Facebookkewangtw @ Linkedinkewang @ SlideSharecpckewang @ Gmail
HadoopCon '14, '15MOPCON '14
What Mitake is
三竹資訊● Qmi：企業內部即時通訊軟體● 全國最大的企業簡訊平台● 市佔近 100% 的行動下單● 市佔近 70% 的行動銀行● 企業內部應用、產壽險、金融相關政府機關● 其他：兌彩券、台灣匯率、三竹小股王、行動股市 ... 等
4
Agenda
5
Agenda
● 各 logger framework 簡單介紹
6
Agenda
● 各 logger framework 簡單介紹● logback 內部架構
7
Agenda
● 各 logger framework 簡單介紹● logback 內部架構● 如何用 logback 串接網路服務
8
各 logger framework 簡單介紹interface & implementation
9
logger framework
10
logger framework
● slf4j
11
logger framework
● slf4j● jcl
12
logger framework
● slf4j● jcl● jul
13
logger framework
● slf4j● jcl● jul● Log4j 1
14
logger framework
● slf4j● jcl● jul● Log4j 1● Log4j 2
15
logger framework
● slf4j● jcl● jul● Log4j 1● Log4j 2● Logback
16
17
logger 分類 - Interface
18
logger 分類 - Interface
● slf4j - Simple Logging Facade for Java– 利用 Facade Pattern 提供一些 logging API ，可以轉換各種不同的 logger implementation
19
logger 分類 - Interface
● slf4j - Simple Logging Facade for Java– 利用 Facade Pattern 提供一些 logging API ，可以轉換各種不同的 logger implementation
● jcl - Jakarta Commons Logging– Apache 開發出與 slf4j 類似功用的 library
20
logger 分類 - Implementation
21
logger 分類 - Implementation
● Logback - 今日主角
22
logger 分類 - Implementation
● Logback - 今日主角● jul - java.util.logging
– Java 1.4 之後的內建 logger API
23
logger 分類 - Implementation
● Logback - 今日主角● jul - java.util.logging
– Java 1.4 之後的內建 logger API● Log4j 1 - logging library for Java
– EOL @ 2015/8/5
24
logger 分類 - Implementation
● Logback - 今日主角● jul - java.util.logging
– Java 1.4 之後的內建 logger API● Log4j 1 - logging library for Java
– EOL @ 2015/8/5● Log4j 2 - Interface + Implementation
25
26
logback 內部架構
27
Architecture
28
Architecture
● Logger– effective level– basic selection rule
29
Architecture
● Logger– effective level– basic selection rule
● Appender– output destination– i.e. console, files, remote socket servers, DB, JMS, and
remote UNIX Syslog daemons
30
Architecture
● Logger– effective level– basic selection rule
● Appender– output destination– i.e. console, files, remote socket servers, DB, JMS, and remote
UNIX Syslog daemons● Layout
– formatting the event to string
31
Architecture - flow
● Get the filter chain decision● Apply the basic selection rule● Create a LoggingEvent object● Invoking appenders● Formatting the output● Sending out the LoggingEvent
32
Architecture - sequence diagram
33
Architecture - sequence diagram
34
如何用 logback 串接網路服務
35
logback.xml - 給使用者看的
36
logback.xml - 給使用者看的<appender name="REDMINE" class="example.MyOwnAppender"> <url>http://example.com</url> <apiKey>abcdef1234567890</apiKey> <projectId>5566</projectId> <title>Logback Redmine Appender</title> <onlyError>true</onlyError> <encoder class="c.q.l.c.e.PatternLayoutEncoder"> <pattern>${PATTERN}</pattern> <charset>${CHARSET}</charset> </encoder></appender>
37
logback.xml - 注意事項
38
logback.xml - 注意事項● camelCase
39
logback.xml - 注意事項● camelCase● follow the setter/getter JavaBeans conventions
40
logback.xml - 注意事項● camelCase● follow the setter/getter JavaBeans conventions● only tag, no attribute
41
logback.xml - 注意事項● camelCase● follow the setter/getter JavaBeans conventions● only tag, no attribute● Joran
42
Template - 給開發者看的MyOwnAppender extends AppenderBase<ILoggingEvent> {
@Override protected void append(ILoggingEvent event) { }}
write log
43
Template - 給開發者看的MyOwnAppender extends AppenderBase<ILoggingEvent> { @Override public void start() { super.start(); }
@Override protected void append(ILoggingEvent event) { }}
initialization
44
start() - initialization
45
start() - initialization
super.start();
46
start() - initialization
if (!checkProperties()) { addError("Some message");
return;}
super.start();
47
start() - initialization
if (!checkProperties()) { addError("Some message");
return;}
encoder.init(System.out);
super.start();
48
start() - initialization
if (!checkProperties()) { addError("Some message");
return;}
encoder.init(System.out);
layout = encoder.getLayout();
super.start();
49
start() - initialization
if (!checkProperties()) { addError("Some message");
return;}
encoder.init(System.out);
layout = encoder.getLayout();
initialProperties();
super.start();
50
append(ILoggingEvent) - write log
51
append(ILoggingEvent) - write log
String log = decorateLog(event);
52
append(ILoggingEvent) - write log
String log = decorateLog(event);
writeLogToSomewhere(log);
53
StackTraces
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
at com.hello.AppTest.testSms(AppTest.java:13)
54
ILoggingEvent
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:
55
ILoggingEvent
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:
event.getTimestamp()
56
ILoggingEvent
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:
event.getTimestamp()
event.getThreadName()
57
ILoggingEvent
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:
event.getTimestamp()
event.getThreadName()
event.getLevel()
58
ILoggingEvent
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:
event.getTimestamp()
event.getThreadName()
event.getLevel()
event.getLoggerName()
59
ILoggingEvent
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:
event.getTimestamp()
event.getThreadName()
event.getLevel()
event.getLoggerName()
event.getMessage()
60
ILoggingEvent
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:
event.getTimestamp()
event.getThreadName()
event.getLevel()
event.getLoggerName()
event.getMessage()
%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg
61
IThrowableProxy
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
event.getThrowableProxy()
62
IThrowableProxy
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
proxy.getClassName()
event.getThrowableProxy()
63
IThrowableProxy
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
proxy.getClassName()
proxy.getMessage()
event.getThrowableProxy()
64
StackTraceElement
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
at com.hello.AppTest.testSms(AppTest.java:13)
proxy .getStackTraceElementProxyArray()[n] .getStackTraceElement()
65
StackTraceElement
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
at com.hello.AppTest.testSms(AppTest.java:13)
elem.getClassName()
proxy .getStackTraceElementProxyArray()[n] .getStackTraceElement()
66
StackTraceElement
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
at com.hello.AppTest.testSms(AppTest.java:13)
elem.getClassName()
elem.getMethodName()
proxy .getStackTraceElementProxyArray()[n] .getStackTraceElement()
67
StackTraceElement
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
at com.hello.AppTest.testSms(AppTest.java:13)
elem.getClassName()
elem.getMethodName()
elem.getFileName()
proxy .getStackTraceElementProxyArray()[n] .getStackTraceElement()
68
StackTraceElement
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
at com.hello.AppTest.testSms(AppTest.java:13)
elem.getClassName()
elem.getMethodName()
elem.getFileName()
elem.getLineNumber()
proxy .getStackTraceElementProxyArray()[n] .getStackTraceElement()
69
StackTraceElement
2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops
at com.hello.AppTest.testSms(AppTest.java:13)
elem.toString()
proxy .getStackTraceElementProxyArray()[n] .getStackTraceElement()
70
Appenders @ kewang
71
Appenders @ kewang
● logback-redmine-appender
● logback-sms-appender
72
Appenders @ kewang
● logback-redmine-appender– merge the same StackTraces @ one issue
● logback-sms-appender
73
Appenders @ kewang
● logback-redmine-appender– merge the same StackTraces @ one issue– link StackTraces with git repo
● logback-sms-appender
74
Appenders @ kewang
● logback-redmine-appender– merge the same StackTraces @ one issue– link StackTraces with git repo– nested tag
● logback-sms-appender
75
Appenders @ kewang
● logback-redmine-appender– merge the same StackTraces @ one issue– link StackTraces with git repo– nested tag
● logback-sms-appender– custom output
76
Appenders @ kewang
● logback-redmine-appender– merge the same StackTraces @ one issue– link StackTraces with git repo– nested tag
● logback-sms-appender– custom output– only append ERROR level
77
Live DEMO
78
One more thing
80
Live DEMO
81
Features
82
Features
● one AP, multiple environments
83
Features
● one AP, multiple environments● one environment, multiple subscriptions
84
Features
● one AP, multiple environments● one environment, multiple subscriptions● RESTful API
85
Features
● one AP, multiple environments● one environment, multiple subscriptions● RESTful API● ...... etc.
86
logback-logpush-appender
<dependency> <groupId>io.logpush</groupId> <artifactId>logback-logpush-appender</artifactId> <version>0.1.0</version></dependency>
pom.xml
87
logback-logpush-appender
<appender name="LOGPUSH" class="i.l.l.a.LogpushAppender"> <token>THIS_IS_LOGPUSH_TOKEN</token> <onlyError>true</onlyError> <encoder class="c.q.l.c.e.PatternLayoutEncoder"> <pattern>${PATTERN}</pattern> <charset>${CHARSET}</charset> </encoder></appender>
logback.xml
88
Definition
● AP● Logpush Server● Firebase Cloud Messaging● Mobile device
89
Log flow
Firebase Cloud MessagingLogpush Server
Mobile deviceAP
90
Log flow - send log to Logpush
Firebase Cloud MessagingLogpush Server
Mobile deviceAP
send log via appender
91
Log flow - send msg to FCM
Firebase Cloud MessagingLogpush Server
Mobile deviceAP
send
send log via appender
92
Log flow - send log to subscription
Firebase Cloud MessagingLogpush Server
Mobile deviceAP
send
receivesend log via appender
93
Log flow - other features
Firebase Cloud MessagingLogpush Server
Mobile deviceAP
send
receivesend log via appender RESTful
API
94
95
早期版本，請小力鞭
96
早期版本，請小力鞭
97
References
● http://logpush.io● Logback● jdk-logging log4j logback、 、 日志介绍及原理● slf4j jcl jul log4j1 log4j2 logback、 、 、 、 、 大总结● Log4j Log4j 2 Logback SFL4J JUL JCL、 、 、 、 、 的比较
98
99
top related