cvim saisentan-6-4-tomoaki

Chapter 4. Deep Learning tomoaki_teshima 64 1.1. 2.1.

Deep Learning ILSVRC Deep Learning Neural Network NN Neural Network(NN)

Super Vision (Deep Learning) ImageNet Large Scale Visual Recognition Competition 2012 (ILSVRC2012) http://image-net.org/challenges/LSVRC/2012/results.html 0 0.05 0.1 0.15 0.2 0.25 0.3 Classification Error (5 guesses) 0 0.1 0.2 0.3 0.4 0.5 0.6 Localization Error (5 guesses)

Deep Learning Debut deep learning http://togetter.com/li/387872

ILSVRCDebut Why does deep learning require the construction of generative data models? http://www.quora.com/Why-does-deep-learning-require-the-construction- of-generative-data-models 20105 2009deliciousDeep Learning http://d.hatena.ne.jp/repose/20110112/1294837332 2011112 1940 1980 1990 2010 Deep learning NN Neocognitron(1982) CNN(1989) DBN(2006)

Deep Learning Deep Learning

(@sla) https://twitter.com/sla/status/166079710764335105 (@tomerun) https://twitter.com/tomerun/status/192659930208141312 (@shuyo) Active Learning http://www.slideshare.net/shuyo/introduction-to-active- learning-25787487

@ambee_whisper, Twitter / ambee_whisper: @shock_pan_man ... https://twitter.com/ambee_whisper/status/256039859528019968

Deep Learning (@ beam2d), Deep Learning http://www.slideshare.net/beam2d/deep-learning20140130 20133 20134 20138 201310 201312 20141 20144 :GoogleDNN research (G.Hinton ) :Baidu Institute of Deep Learning :Yahoo IQ Engines :Yahoo LookFlow :Facebook AI Lab (Y.LeCun) :Google Deep Mind :Google Street View CAPTCHA

Deep Learning DNN Research http://media.utoronto.ca/media-releases/u-of-t-neural-networks-start-up-acquired-by-google/ March 12, 2013

Deep Learning Institute of Deep Learning http://www.techinasia.com/baidu-opens-research-lab-california-deep-learning/ April 15, 2013

Deep Learning Yahoo IQ Engine http://techcrunch.com/2013/08/23/yahoo-acquires-image-recognition-startup-iq-engines/ August 23, 2013

Deep Learning Yahoo LookFlow http://techcrunch.com/2013/10/23/yahoo-acquires-startup-lookflow-to-work-on-flickr-and-deep-learning/ Oct 23, 2013

Deep Learning Facebook AI Lab https://www.facebook.com/yann.lecun/posts/10151728212367143 December 10, 2013

Deep Learning Google DeepMind http://techcrunch.com/2014/01/26/google-deepmind/ January 26, 2014

Deep Learning Street View http://jp.techcrunch.com/2014/04/17/20140416googles-new-street-view-image-recognition- algorithm-can-beat-most-captchas/ 2014417

Deep Learning TUEEEEEEEEE

NN In 2000, it was even pointed out by the organizers of the Neural Information Processing System (NIPS) conference that the term neural networks in the submission title was negatively correlated with acceptance. In contrast, positive correlations were made with support vector machines (SVMs), Bayesian networks, and variational methods. [11] Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.10.5032

() NN NN

NN A feedforward neural network is an artificial neural network where connections between the units do not form a directed cycle. This is different from recurrent neural networks. The feedforward neural network was the first and simplest type of artificial neural network devised. In this network, the information moves in only one direction, forward, from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network. http://en.wikipedia.org/wiki/Feedforward_neural_network

Backpropagation

d() p() 1 0 0.24 0.76 = (9)

d p 1 0 0.24 0.76 = (13)

= 1 3 2 3 d p 1 0 0.24 0.76 (14) =

(1) (1) (2) (2)

// backward pass, update weights for( i = l_count-1; i > 0; i-- ) { int n1 = layer_sizes->data.i[i-1], n2 = layer_sizes->data.i[i]; cvInitMatHeader( &_df, 1, n2, CV_64F, df[i] ); cvMul( grad1, &_df, grad1 ); cvInitMatHeader( &_w, n1+1, n2, CV_64F, weights[i] ); cvInitMatHeader( &_dw, n1+1, n2, CV_64F, dw->data.db + (weights[i] - weights[0]) ); cvInitMatHeader( x1, n1+1, 1, CV_64F, x[i-1] ); x[i-1][n1] = 1.; cvGEMM( x1, grad1, params.bp_dw_scale, &_dw, params.bp_moment_scale, &_dw ); cvAdd( &_w, &_dw, &_w ); if( i > 1 ) { grad2->cols = n1; _w.rows = n1; cvGEMM( grad1, &_w, 1, 0, 0, grad2, CV_GEMM_B_T ); } CV_SWAP( grad1, grad2, temp ); } OpenCV2.4.8 opencvsourcesmodulesmlsrcann_mlp.cpp calc_activ_func_deriv( x2, &_df, _w.data.db + _w.rows*_w.cols ); cvInitMatHeader( grad1, 1, ovcount, CV_64F, buf_ptr ); (15)(15)

Neural Network

NN() Back Propagation

Deep Learning Dropout Neural Network Deep learning, CNN

http://togetter.com/li/387872 https://twitter.com/dandelion1124/status/255700185089470464

ImageNet Large Scale Visual Recognition Competition 2012 (ILSVRC2012) http://www.image-net.org/challenges/LSVRC/2012/results.html deep learning http://togetter.com/li/387872 Why does deep learning require the construction of generative data models? Quora http://www.quora.com/Why-does-deep-learning-require-the-construction-of-generative-data-models deep learning & An Analysis of Single-Layer Networks in Unsupervised Feature Learning (NIPS2010) - http://d.hatena.ne.jp/repose/20110112/1294837332 @sla, Twitter / sla: ... https://twitter.com/sla/status/166079710764335105 @tomerun, Twitter / tomerun: ... https://twitter.com/tomerun/status/192659930208141312 , Active Learning http://www.slideshare.net/shuyo/introduction-to-active-learning-25787487 @ambee_whisper, Twitter / ambee_whisper: @shock_pan_man ... https://twitter.com/ambee_whisper/status/256039859528019968 (@ beam2d), Deep Learning http://www.slideshare.net/beam2d/deep-learning20140130 U of T neural networks start-up acquired by Google | University of Toronto Media Room http://media.utoronto.ca/media-releases/u-of-t-neural-networks-start-up-acquired-by-google/ Steven Millward , Baidu Opens Lab in Silicon Valley Devoted to Research into Deep Learning http://www.techinasia.com/baidu-opens-research-lab-california-deep-learning/ Chris Velazco, Yahoo Acquires Image-Recognition Startup IQ Engines To Improve Flickr Photo Organization & Search | TechCrunch http://techcrunch.com/2013/08/23/yahoo-acquires-image-recognition-startup-iq-engines/ Yahoo Acquires Startup LookFlow To Work On Flickr And `Deep Learning | TechCrunch http://techcrunch.com/2013/10/23/yahoo-acquires-startup-lookflow-to-work-on-flickr-and-deep- learning/ Yann LeCun - Big news today! Facebook has created a new research... https://www.facebook.com/yann.lecun/posts/10151728212367143 Catherine Shu, Google Acquires Artificial Intelligence Startup DeepMind For More Than $500M | TechCrunch http://techcrunch.com/2014/01/26/google-deepmind/ Frederic Lardinois, Google MapsStreet ViewCAPTCHA ? TechCrunch http://jp.techcrunch.com/2014/04/17/20140416googles-new-street-view-image-recognition-algorithm- can-beat-most-captchas/ Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.10.5032