R&D, Azoftrnd.azoft.com

1. 2. 3. 4. Caffe5. 6. deep learning7. 8.

learning rate

momentum

Nesterov momentum Adam

-

drop out

C drop out

Weight decay. .

. , /.

. .

. , .

PCA whitening

ZCA whitening

/ /

,

CNN Caffe

. .

Conv + ReLU

lstm. natural language processing speech recognition. . . , . . natural language processing .

deep learningCaffetheanolasagnetorchTensorFlow Google ()

Fractional max poolingRecurrent convolutional layers

CNN

feature maps

. . . Random Forest

Recurrent convolutional layers

cifar-10, mnist . , , , . computer vision

!R&D, Azoftrnd.azoft.com