手把手教你如何串接 log 到各種網路服務

手把手教你如何串接 log 到各種網路服務Kewang

Who I am

● 王慕羣● Java / Node.js / AngularJS● HBase / SQL-like● Git / DevOps

kewang @ GitHubkewang.information @ Facebookkewangtw @ Linkedinkewang @ SlideSharecpckewang @ Gmail

HadoopCon '14, '15MOPCON '14

https://github.com/kewang

https://fb.me/kewang.information

https://www.linkedin.com/in/kewangtw

http://www.slideshare.net/kewang

mailto:[email protected]?subject=%E6%89%8B%E6%8A%8A%E6%89%8B%E6%95%99%E4%BD%A0%E5%A6%82%E4%BD%95%E4%B8%B2%E6%8E%A5%20log%20%E5%88%B0%E5%90%84%E7%A8%AE%E7%B6%B2%E8%B7%AF%E6%9C%8D%E5%8B%99

http://www.slideshare.net/kewang/how-to-build-a-scalable-sns-using-hbase

http://www.slideshare.net/kewang/hedis-get-hbase-via-redis

http://www.slideshare.net/kewang/how-to-build-a-scalable-sns-via-polling-push

What Mitake is

三竹資訊● Qmi：企業內部即時通訊軟體● 全國最大的企業簡訊平台● 市佔近 100% 的行動下單● 市佔近 70% 的行動銀行● 企業內部應用、產壽險、金融相關政府機關● 其他：兌彩券、台灣匯率、三竹小股王、行動股市 ... 等

https://fb.me/mitake.tw

https://www.qmi.emome.net/

4

Agenda

5

Agenda

● 各 logger framework 簡單介紹

6

Agenda

● 各 logger framework 簡單介紹● logback 內部架構

7

Agenda

● 各 logger framework 簡單介紹● logback 內部架構● 如何用 logback 串接網路服務

8

各 logger framework 簡單介紹interface & implementation

9

logger framework

10

logger framework

● slf4j

11

logger framework

● slf4j● jcl

12

logger framework

● slf4j● jcl● jul

13

logger framework

● slf4j● jcl● jul● Log4j 1

14

logger framework

● slf4j● jcl● jul● Log4j 1● Log4j 2

15

logger framework

● slf4j● jcl● jul● Log4j 1● Log4j 2● Logback

17

logger 分類 - Interface

18


● slf4j - Simple Logging Facade for Java– 利用 Facade Pattern 提供一些 logging API ，可以轉換各種不同的 logger implementation

19


● slf4j - Simple Logging Facade for Java– 利用 Facade Pattern 提供一些 logging API ，可以轉換各種不同的 logger implementation

● jcl - Jakarta Commons Logging– Apache 開發出與 slf4j 類似功用的 library

20

logger 分類 - Implementation

21


● Logback - 今日主角

22


● Logback - 今日主角● jul - java.util.logging

– Java 1.4 之後的內建 logger API

23



– Java 1.4 之後的內建 logger API● Log4j 1 - logging library for Java

– EOL @ 2015/8/5

24



– Java 1.4 之後的內建 logger API● Log4j 1 - logging library for Java

– EOL @ 2015/8/5● Log4j 2 - Interface + Implementation

26

logback 內部架構

27

Architecture

28

Architecture

● Logger– effective level– basic selection rule

29

Architecture


● Appender– output destination– i.e. console, files, remote socket servers, DB, JMS, and

remote UNIX Syslog daemons

30

Architecture


● Appender– output destination– i.e. console, files, remote socket servers, DB, JMS, and remote

UNIX Syslog daemons● Layout

– formatting the event to string

31

Architecture - flow

● Get the filter chain decision● Apply the basic selection rule● Create a LoggingEvent object● Invoking appenders● Formatting the output● Sending out the LoggingEvent

32

Architecture - sequence diagram

33

Architecture - sequence diagram

34

如何用 logback 串接網路服務

35

logback.xml - 給使用者看的

36

logback.xml - 給使用者看的<appender name="REDMINE" class="example.MyOwnAppender"> <url>http://example.com</url> <apiKey>abcdef1234567890</apiKey> <projectId>5566</projectId> <title>Logback Redmine Appender</title> <onlyError>true</onlyError> <encoder class="c.q.l.c.e.PatternLayoutEncoder"> <pattern>${PATTERN}</pattern> <charset>${CHARSET}</charset> </encoder></appender>

37

logback.xml - 注意事項

38

logback.xml - 注意事項● camelCase

39

logback.xml - 注意事項● camelCase● follow the setter/getter JavaBeans conventions

40

logback.xml - 注意事項● camelCase● follow the setter/getter JavaBeans conventions● only tag, no attribute

41

logback.xml - 注意事項● camelCase● follow the setter/getter JavaBeans conventions● only tag, no attribute● Joran

42

Template - 給開發者看的MyOwnAppender extends AppenderBase<ILoggingEvent> {

@Override protected void append(ILoggingEvent event) { }}

write log

43

Template - 給開發者看的MyOwnAppender extends AppenderBase<ILoggingEvent> { @Override public void start() { super.start(); }

@Override protected void append(ILoggingEvent event) { }}

initialization

44

start() - initialization

45


super.start();

46


if (!checkProperties()) { addError("Some message");

return;}

super.start();

47



return;}

encoder.init(System.out);

super.start();

48



return;}


layout = encoder.getLayout();

super.start();

49



return;}


layout = encoder.getLayout();

initialProperties();

super.start();

50

append(ILoggingEvent) - write log

51


String log = decorateLog(event);

52


String log = decorateLog(event);

writeLogToSomewhere(log);

53

StackTraces

2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught: java.lang.RuntimeException: Oops

at com.hello.AppTest.testSms(AppTest.java:13)

54

ILoggingEvent

2016-10-03 22:00:09.174 [main] ERROR com.hello.Test - Caught:

55

ILoggingEvent


event.getTimestamp()

56

ILoggingEvent



event.getThreadName()

57

ILoggingEvent




event.getLevel()

58

ILoggingEvent




event.getLevel()

event.getLoggerName()

59

ILoggingEvent




event.getLevel()


event.getMessage()

60

ILoggingEvent




event.getLevel()


event.getMessage()

%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg

61

IThrowableProxy


event.getThrowableProxy()

62

IThrowableProxy


proxy.getClassName()


63

IThrowableProxy


proxy.getClassName()

proxy.getMessage()


64

StackTraceElement



proxy .getStackTraceElementProxyArray()[n] .getStackTraceElement()

65

StackTraceElement



elem.getClassName()


66

StackTraceElement



elem.getClassName()

elem.getMethodName()


67

StackTraceElement



elem.getClassName()


elem.getFileName()


68

StackTraceElement



elem.getClassName()


elem.getFileName()

elem.getLineNumber()


69

StackTraceElement



elem.toString()


70

Appenders @ kewang

71

Appenders @ kewang

● logback-redmine-appender

● logback-sms-appender

https://github.com/kewang/logback-redmine-appender

https://github.com/mitaketw/logback-sms-appender

72

Appenders @ kewang

● logback-redmine-appender– merge the same StackTraces @ one issue




73

Appenders @ kewang

● logback-redmine-appender– merge the same StackTraces @ one issue– link StackTraces with git repo




74

Appenders @ kewang

● logback-redmine-appender– merge the same StackTraces @ one issue– link StackTraces with git repo– nested tag




75

Appenders @ kewang


● logback-sms-appender– custom output



76

Appenders @ kewang


● logback-sms-appender– custom output– only append ERROR level



77

Live DEMO



78

One more thing

79

Logpushhttp://logpush.io

http://logpush.io/

80

Live DEMO

81

Features

82

Features

● one AP, multiple environments

83

Features

● one AP, multiple environments● one environment, multiple subscriptions

84

Features

● one AP, multiple environments● one environment, multiple subscriptions● RESTful API

85

Features

● one AP, multiple environments● one environment, multiple subscriptions● RESTful API● ...... etc.

86

logback-logpush-appender

<dependency> <groupId>io.logpush</groupId> <artifactId>logback-logpush-appender</artifactId> <version>0.1.0</version></dependency>

pom.xml

87

logback-logpush-appender

<appender name="LOGPUSH" class="i.l.l.a.LogpushAppender"> <token>THIS_IS_LOGPUSH_TOKEN</token> <onlyError>true</onlyError> <encoder class="c.q.l.c.e.PatternLayoutEncoder"> <pattern>${PATTERN}</pattern> <charset>${CHARSET}</charset> </encoder></appender>

logback.xml

88

Definition

● AP● Logpush Server● Firebase Cloud Messaging● Mobile device

89

Log flow

Firebase Cloud MessagingLogpush Server

Mobile deviceAP

90

Log flow - send log to Logpush


Mobile deviceAP

send log via appender

91

Log flow - send msg to FCM


Mobile deviceAP

send

send log via appender

92

Log flow - send log to subscription


Mobile deviceAP

send

receivesend log via appender

93

Log flow - other features


Mobile deviceAP

send

receivesend log via appender RESTful

API

95

早期版本，請小力鞭

96

早期版本，請小力鞭

97

References

● http://logpush.io● Logback● jdk-logging log4j logback、、日志介绍及原理● slf4j jcl jul log4j1 log4j2 logback、、、、、大总结● Log4j Log4j 2 Logback SFL4J JUL JCL、、、、、的比较

http://logpush.io/

http://logback.qos.ch/

http://blog.jobbole.com/102976/

https://my.oschina.net/pingpangkuangmo/blog/410224

http://www.cnblogs.com/penghongwei/p/3417179.html

http://www.cnblogs.com/penghongwei/p/3417179.html

手把手教你如何串接 log 到各種網路服務

Technology