machine learning from optimization …...• a few useful things to know about machine learning....
TRANSCRIPT
MACHINE LEARNING FROM OPTIMIZATION PERSPECTIVEZheng Han
June 15th, 2017
6/14/17
1
谢绝转载，转载请联系WSFC
Wall Street Fintech Club (WSFC), © All Rights Reserved ,www.wallstreetfintechclub.com, 公共号：华尔街 Fintech俱乐部，禁止未授权转载
• Machine Learning (ML)• Case Study• Optimization in ML• - Learning Theory• - Regularization• - Learning Algorithm
2
CONTENT
Wall Street Fintech Club (WSFC), © All Rights Reserved ,www.wallstreetfintechclub.com, 公共号：华尔街 Fintech俱乐部，禁止未授权转载
ML – A FAVORABLE PERSPECTIVE
ML – ANALOGY
ML – THREE COMPONENTS
Machine Learning = Representation + Evaluation + Optimization
CASE STUDY – DECISION TREE
CASE STUDY – DECISION TREE
● Representation: x = (Sunny Hot, High, Strong), tree structure to represent boolean function
● Evaluation: false positive rate, false negative rate, etc..
● Optimization: efficiently construct a tree that gives relatively low predictive error
Optimization:Stochastic Gradient Descent(SGD)s:Momentum / Nesterov accelerated gradientAdagrad / Adadelta / RMSprop / Adam
Backpropagation
CASE STUDY – NEURAL NETWORK
Representation: images -> pixels -> matrices
ML – AUTOMATE AUTOMATION
OPTIMIZATION
OPTIMIZATION IN ML
Representation
Evaluation
Optimization
Parameterized Optimization
OPTIMIZATION IN ML
Expected Risk
Empirical Risk
Learning Theory
OPTIMIZATION IN ML
Expected Risk
Empirical Risk
Structural Risk Minimization
Model Complexity
STRUCTURAL RISK MINIMIZATION BY REGULARIZATION
OPTIMIZATION ALGORITHM IN ML
Gradient Descent Method:
Second-order Method:
OPTIMIZATION ALG FOR DEEP LEARNING
• Reinforcement Learning: dynamic programming• Online Learning: online convex optimization• Evolutionary Algorithms: generic algorithms • Big Data: distributed optimization, sparse optimization
17
OTHER RELEVANT TOPICS
Wall Street Fintech Club (WSFC), © All Rights Reserved ,www.wallstreetfintechclub.com, 公共号：华尔街 Fintech俱乐部，禁止未授权转载
ML IN REAL-WORLD
• A Few Useful Things to Know about Machine Learning. Pedro Domingos• The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake
Our World. Pedro Domingos• Scikit-Learn: http://scikit-learn.org• An Extended Version Of The Scikit-Learn Cheat Sheet. Christophe Bourguignat• Deep Learning. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton• Optimization Methods for Large-Scale Machine Learning. Leon Bottou, Frank E. Curtis,
and Jorge Nocedal• The LION Way: Machine Learning plus Intelligent Optimization. Roberto Battiti, Mauro
Brunato
19
REFERENCES
Wall Street Fintech Club (WSFC), © All Rights Reserved ,www.wallstreetfintechclub.com, 公共号：华尔街 Fintech俱乐部，禁止未授权转载