cvim saisentan-6-4-tomoaki

of 39 /39
Chapter 4. Deep Learning tomoaki_teshima ココココココココココココココココ 6 ココ 4 コ §1.1. … §2.1.

Upload: tomoaki0705

Post on 19-Aug-2014

2.294 views

Category:

Engineering


5 download

Embed Size (px)

DESCRIPTION

関東CV勉強会第23回の発表資料

TRANSCRIPT

  • Chapter 4. Deep Learning tomoaki_teshima 64 1.1. 2.1.
  • Deep Learning ILSVRC Deep Learning Neural Network NN Neural Network(NN)
  • Super Vision (Deep Learning) ImageNet Large Scale Visual Recognition Competition 2012 (ILSVRC2012) http://image-net.org/challenges/LSVRC/2012/results.html 0 0.05 0.1 0.15 0.2 0.25 0.3 Classification Error (5 guesses) 0 0.1 0.2 0.3 0.4 0.5 0.6 Localization Error (5 guesses)
  • Deep Learning Debut deep learning http://togetter.com/li/387872
  • ILSVRCDebut Why does deep learning require the construction of generative data models? http://www.quora.com/Why-does-deep-learning-require-the-construction- of-generative-data-models 20105 2009deliciousDeep Learning http://d.hatena.ne.jp/repose/20110112/1294837332 2011112 1940 1980 1990 2010 Deep learning NN Neocognitron(1982) CNN(1989) DBN(2006)
  • Deep Learning Deep Learning
  • (@sla) https://twitter.com/sla/status/166079710764335105 (@tomerun) https://twitter.com/tomerun/status/192659930208141312 (@shuyo) Active Learning http://www.slideshare.net/shuyo/introduction-to-active- learning-25787487
  • @ambee_whisper, Twitter / ambee_whisper: @shock_pan_man ... https://twitter.com/ambee_whisper/status/256039859528019968
  • Deep Learning (@ beam2d), Deep Learning http://www.slideshare.net/beam2d/deep-learning20140130 20133 20134 20138 201310 201312 20141 20144 :GoogleDNN research (G.Hinton ) :Baidu Institute of Deep Learning :Yahoo IQ Engines :Yahoo LookFlow :Facebook AI Lab (Y.LeCun) :Google Deep Mind :Google Street View CAPTCHA
  • Deep Learning DNN Research http://media.utoronto.ca/media-releases/u-of-t-neural-networks-start-up-acquired-by-google/ March 12, 2013
  • Deep Learning Institute of Deep Learning http://www.techinasia.com/baidu-opens-research-lab-california-deep-learning/ April 15, 2013
  • Deep Learning Yahoo IQ Engine http://techcrunch.com/2013/08/23/yahoo-acquires-image-recognition-startup-iq-engines/ August 23, 2013
  • Deep Learning Yahoo LookFlow http://techcrunch.com/2013/10/23/yahoo-acquires-startup-lookflow-to-work-on-flickr-and-deep-learning/ Oct 23, 2013
  • Deep Learning Facebook AI Lab https://www.facebook.com/yann.lecun/posts/10151728212367143 December 10, 2013
  • Deep Learning Google DeepMind http://techcrunch.com/2014/01/26/google-deepmind/ January 26, 2014
  • Deep Learning Street View http://jp.techcrunch.com/2014/04/17/20140416googles-new-street-view-image-recognition- algorithm-can-beat-most-captchas/ 2014417
  • Deep Learning TUEEEEEEEEE
  • Deep Learning ILSVRC Deep Learning Neural Network NN Neural Network(NN)
  • NN In 2000, it was even pointed out by the organizers of the Neural Information Processing System (NIPS) conference that the term neural networks in the submission title was negatively correlated with acceptance. In contrast, positive correlations were made with support vector machines (SVMs), Bayesian networks, and variational methods. [11] Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.10.5032
  • NN
  • NN NN
  • () NN NN
  • Deep Learning ILSVRC Deep Learning Neural Network NN Neural Network(NN)
  • NN A feedforward neural network is an artificial neural network where connections between the units do not form a directed cycle. This is different from recurrent neural networks. The feedforward neural network was the first and simplest type of artificial neural network devised. In this network, the information moves in only one direction, forward, from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network. http://en.wikipedia.org/wiki/Feedforward_neural_network
  • NN
  • (w,b)
  • Backpropagation
  • d() p() 1 0 0.24 0.76 = (9)
  • d p 1 0 0.24 0.76 = (13)
  • = 1 3 2 3 d p 1 0 0.24 0.76 (14) =
  • (1) (1) (2) (2)
  • // backward pass, update weights for( i = l_count-1; i > 0; i-- ) { int n1 = layer_sizes->data.i[i-1], n2 = layer_sizes->data.i[i]; cvInitMatHeader( &_df, 1, n2, CV_64F, df[i] ); cvMul( grad1, &_df, grad1 ); cvInitMatHeader( &_w, n1+1, n2, CV_64F, weights[i] ); cvInitMatHeader( &_dw, n1+1, n2, CV_64F, dw->data.db + (weights[i] - weights[0]) ); cvInitMatHeader( x1, n1+1, 1, CV_64F, x[i-1] ); x[i-1][n1] = 1.; cvGEMM( x1, grad1, params.bp_dw_scale, &_dw, params.bp_moment_scale, &_dw ); cvAdd( &_w, &_dw, &_w ); if( i > 1 ) { grad2->cols = n1; _w.rows = n1; cvGEMM( grad1, &_w, 1, 0, 0, grad2, CV_GEMM_B_T ); } CV_SWAP( grad1, grad2, temp ); } OpenCV2.4.8 opencvsourcesmodulesmlsrcann_mlp.cpp calc_activ_func_deriv( x2, &_df, _w.data.db + _w.rows*_w.cols ); cvInitMatHeader( grad1, 1, ovcount, CV_64F, buf_ptr ); (15)(15)
  • Neural Network
  • NN() Back Propagation
  • Deep Learning Dropout Neural Network Deep learning, CNN
  • http://togetter.com/li/387872 https://twitter.com/dandelion1124/status/255700185089470464
  • ImageNet Large Scale Visual Recognition Competition 2012 (ILSVRC2012) http://www.image-net.org/challenges/LSVRC/2012/results.html deep learning http://togetter.com/li/387872 Why does deep learning require the construction of generative data models? Quora http://www.quora.com/Why-does-deep-learning-require-the-construction-of-generative-data-models deep learning & An Analysis of Single-Layer Networks in Unsupervised Feature Learning (NIPS2010) - http://d.hatena.ne.jp/repose/20110112/1294837332 @sla, Twitter / sla: ... https://twitter.com/sla/status/166079710764335105 @tomerun, Twitter / tomerun: ... https://twitter.com/tomerun/status/192659930208141312 , Active Learning http://www.slideshare.net/shuyo/introduction-to-active-learning-25787487 @ambee_whisper, Twitter / ambee_whisper: @shock_pan_man ... https://twitter.com/ambee_whisper/status/256039859528019968 (@ beam2d), Deep Learning http://www.slideshare.net/beam2d/deep-learning20140130 U of T neural networks start-up acquired by Google | University of Toronto Media Room http://media.utoronto.ca/media-releases/u-of-t-neural-networks-start-up-acquired-by-google/ Steven Millward , Baidu Opens Lab in Silicon Valley Devoted to Research into Deep Learning http://www.techinasia.com/baidu-opens-research-lab-california-deep-learning/ Chris Velazco, Yahoo Acquires Image-Recognition Startup IQ Engines To Improve Flickr Photo Organization & Search | TechCrunch http://techcrunch.com/2013/08/23/yahoo-acquires-image-recognition-startup-iq-engines/ Yahoo Acquires Startup LookFlow To Work On Flickr And `Deep Learning | TechCrunch http://techcrunch.com/2013/10/23/yahoo-acquires-startup-lookflow-to-work-on-flickr-and-deep- learning/ Yann LeCun - Big news today! Facebook has created a new research... https://www.facebook.com/yann.lecun/posts/10151728212367143 Catherine Shu, Google Acquires Artificial Intelligence Startup DeepMind For More Than $500M | TechCrunch http://techcrunch.com/2014/01/26/google-deepmind/ Frederic Lardinois, Google MapsStreet ViewCAPTCHA ? TechCrunch http://jp.techcrunch.com/2014/04/17/20140416googles-new-street-view-image-recognition-algorithm- can-beat-most-captchas/ Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.10.5032