introduction to programming - jun jijun.hansung.ac.kr/ml/docs-slides-lecture8-kr.pdf ·...
TRANSCRIPT
![Page 1: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/1.jpg)
신경망(Neural Networks): 표현
비선형모델
Machine Learning
![Page 2: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/2.jpg)
Andrew Ng
비선형분류
x1
x2
size# bedrooms# floorsage
![Page 3: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/3.jpg)
Andrew Ng
당신은이것을봅니다:
그러나카메라는이것을봅니다:
이것은무엇인가?
![Page 4: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/4.jpg)
Andrew Ng
Computer Vision: 자동차감지
Testing:
What is this?
Not a carCars
![Page 5: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/5.jpg)
Andrew Ng
Learning Algorithm
pixel 1
pixel 2
pixel 1
pixel 2
Raw image
Cars“Non”-Cars
![Page 6: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/6.jpg)
Andrew Ng
pixel 1
pixel 2
Raw image
Cars“Non”-Cars
Learning Algorithm
pixel 1
pixel 2
![Page 7: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/7.jpg)
Andrew Ng
pixel 1
pixel 2
Raw image
Cars“Non”-Cars
50 x 50 pixel images→ 2500 pixels(7500 if RGB)
pixel 1 intensity
pixel 2 intensity
pixel 2500 intensity
Quadratic features ( ): ≈3 millionfeatures
Learning Algorithm
pixel 1
pixel 2
![Page 8: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/8.jpg)
Andrew Ng
![Page 9: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/9.jpg)
신경망(Neural Networks): 표현
신경과뇌
Machine Learning
![Page 10: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/10.jpg)
Andrew Ng
신경망(Neural Networks)
기원: 뇌를모방하기위한알고리즘.
80년대와 90년대초에절리사용되었음; 90년대후반에감소.
최근제기: 많은응용분야에서의첨단기술
![Page 11: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/11.jpg)
Andrew Ng
청각피질이시각을학습한다.
청각피질(Auditory Cortex)
“학습알고리즘” 가설(hypothesis)
[Roe et al., 1992]
![Page 12: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/12.jpg)
Andrew Ng
체성감각피질이시각을학습한다.
체성감각피질(Somatosensory Cortex)
The “one learning algorithm” hypothesis
[Metin & Frost, 1989]
![Page 13: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/13.jpg)
Andrew Ng
혀로보기 사람반향위치측정 (음파탐지기:sonar)
촉각띠(Haptic belt): 방향감지 3번째눈심기
뇌에서의감각표현
[BrainPort; Welsh & Blasch, 1997; Nagel et al., 2005; Constantine-Paton & Law, 2009]
![Page 14: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/14.jpg)
Neural Networks: Representation
Machine Learning
모델표현 I
![Page 15: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/15.jpg)
Andrew Ng
뇌속의신경들
수상돌기
축색돌기
세포체
![Page 16: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/16.jpg)
Andrew Ng
뇌속의신경들
[Credit: US National Institutes of Health, National Institute on Aging]
![Page 17: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/17.jpg)
Andrew Ng
신경모델: 로지스틱(Logistic)단위
시그모이드(로지스틱) 활성함수
Sigmoid (logistic) activation function.
![Page 18: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/18.jpg)
Andrew Ng
Neural Network
Layer 3Layer 1 Layer 2
![Page 19: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/19.jpg)
Andrew Ng
Neural Network“activation” of unit in layer
matrix of weights controlling function mapping from layer to layer
If network has units in layer , units in layer , thenwill be of dimension .
![Page 20: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/20.jpg)
Andrew Ng
![Page 21: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/21.jpg)
Neural Networks: Representation
모델표현 II
Machine Learning
![Page 22: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/22.jpg)
Andrew Ng
Add .
전방향전파: 백터화구현
![Page 23: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/23.jpg)
Andrew Ng
Layer 3Layer 1 Layer 2
Neural Network learning its own features
![Page 24: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/24.jpg)
Andrew Ng
Layer 3Layer 1 Layer 2
Other network architectures
Layer 4
![Page 25: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/25.jpg)
Neural Networks: Representation
Examples and intuitions I
Machine Learning
![Page 26: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/26.jpg)
Andrew Ng
비선형분류예 : XOR/XNOR
, 는이진수(binary) (0 or 1).
x1
x2
x1
x2
![Page 27: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/27.jpg)
Andrew Ng
Simple example: AND
0 00 11 01 1
1.0
![Page 28: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/28.jpg)
Andrew Ng
Example: OR function
0 00 11 01 1
-10
20
20
![Page 29: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/29.jpg)
Neural Networks: Representation
Examples and intuitions II
Machine Learning
![Page 30: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/30.jpg)
Andrew Ng
부정(Negation):
01
![Page 31: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/31.jpg)
Andrew Ng
![Page 32: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/32.jpg)
Andrew Ng
Putting it together:
0 00 11 01 1
-30
20
20
10
-20
-20
-10
20
20
![Page 33: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/33.jpg)
Andrew Ng
Neural Network intuition
Layer 3Layer 1 Layer 2 Layer 4
![Page 34: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/34.jpg)
Andrew Ng
Handwritten digit classification
[Courtesy of Yann LeCun]
![Page 35: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/35.jpg)
Andrew Ng
Handwritten digit classification
[Courtesy of Yann LeCun]
![Page 36: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/36.jpg)
Andrew Ng
Neural Networks: Representation
Multi-class classification
Machine Learning
![Page 37: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/37.jpg)
Andrew Ng
Multiple output units: One-vs-all.
Pedestrian Car Motorcycle Truck
Want , , , etc.
when pedestrian when car when motorcycle
![Page 38: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/38.jpg)
Andrew Ng
Multiple output units: One-vs-all.
Want , , , etc.
when pedestrian when car when motorcycle
Training set:
one of , , ,
pedestrian car motorcycle truck
![Page 39: Introduction to Programming - Jun Jijun.hansung.ac.kr/ML/docs-slides-Lecture8-kr.pdf · 2016-10-05 · Introduction to Programming Author: OpenClassroom Created Date: 10/5/2016 9:36:29](https://reader035.vdocuments.pub/reader035/viewer/2022070717/5edbe0aaad6a402d66664cf9/html5/thumbnails/39.jpg)
Andrew Ng