python ai note - k-techlabo · pdf filemecab の出力の形式は次の通りである....

Download python ai note - K-TechLabo · PDF fileMeCab の出力の形式は次の通りである. 表層形(タブ)品詞, 品詞細分類1, 品詞細分類2, 品詞細分類3, 活用型

If you can't read please download the document

Upload: trandang

Post on 06-Feb-2018

246 views

Category:

Documents


1 download

TRANSCRIPT

  • Python

    Copyright c 2017, Katsunori Nakamura

    2017 9 9

  • 1 11.1 MeCab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Open JTalk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

    1.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Julius . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    1.3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3.2 WAV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

    1.4 OpenSSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

    1.4.1.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.4.1.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.4.1.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

    1.4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

    2 132.1 GMP/MPFR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    2.1.1 GMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    2.1.1.1 GMP . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.1.1.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.1.1.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

    2.1.2 MPFR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

    2.1.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.1.2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    2.1.3 MPFR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    3 Python 223.1 MeCab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

    3.1.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2 Open JTalk PyAudio . . . . . . . . . . . . . . . . . . . . 25

    4 27

    I

  • 1

    1.1 MeCab

    MeCab1

    MeCabMeCab

    test11.txt MeCab

    test11.txt

    1 23 4 5 C67 8 (document) 9

    10 P y t h o n O S ( ) 11 P y t h o n12 13 1415 P y t h o n W e b16 (script) 17 18 19 20 G o o g l e

    MeCab mecab test11.txt

    ,,*,*,*,*,,,,,*,*,*,*,,,,,*,*,*,*,,,,,*,*,*,*,,,,,*,*,*,*,,,,,,*,*,*,*,,*,*,*,*,,,,,*,*,*,*,,, EOS ,,*,*,*,*,,, ,,,*,*,*,,,

    1 EOS MeCab

    1 MeCab http://taku910.github.io/mecab/

    1

  • MeCab

    , 1, 2, 3, , , , ,

    MeCab

    -O wakati .-o .

    mecab -O wakati test11.txt

    2

  • 1.2 Open JTalk

    Open JTalk 2 hts engine API WAVOS open jtalk

    Open JTalk

    http://open-jtalk.sourceforge.net/

    open jtalk

    open jtalk -x -m -s -r -ow

    Hz

    WAV .wav

    1.2.1

    Windows jtest1.txt WAV jtest1.wav Open JTalk

    C:\OpJTalk|+- dic utf 8| || +- char.bin| || +- left-id.def| || +- matrix.bin| || +- pos-id.def| || +- rewrite.def| || +- right-id.def| || +- sys.dic| || +- unk.dic|+- mei angry.htsvoice|+- mei bashful.htsvoice|+- mei happy.htsvoice|+- mei normal.htsvoice |+- mei sad.htsvoice|+- open jtalk.exe Open JTalk

    mei normal.htsvoice OpenJTalk

    2http://software.web.nitech.ac.jp/

    3

  • C:\OpJTalk\open jtalk -x C:\OpJTalk\dic utf 8 -m C:\OpJTalk\mei normal.htsvoice -s 44100 -r 1.0 -ow jtest1.wav jtest1.txt

    4

  • 1.3 Julius

    Julius Julius 3 IPA 4 OSDN Julius http://julius.osdn.jp/

    Julius API Julius OS TCP/IPWAV

    5 1

    1: Ver.4.4

    Julius 1dictation-kit-v4.4

    1bin JuliusOS OS

    Ver.4.4

    Julius Julius Julius

    Julius HMMHMM GMM-HMMGaussian Mixture Model DNN-HMM 2Julius 2

    3 Julius 45Ver.4.4 ZIP 407MB 830MB

    5

  • 1) GMM

    CPU

    2) DNN

    CPU

    Julius

    1.3.1

    Julius 6

    WindowsWindows

    1) run-win-dnn.bat : DNN2) run-win-dnncli.bat : DNN3) run-win-gmm.bat : GMM

    1)

    run-win-dnn Enter

    Julius

    2: run-win-dnn

    > 10

    6

    6

  • pass1 best: input rejected by short inputpass1 best: sentence1: >

    run-win-dnn.bat

    .binwindowsjulius.exe -C main.jconf -C am-dnn.jconf -demo-charconv utf-8 sjis -dnnconf julius.dnnconf

    Julius

    main.jconf

    am-dnn.jconf

    julius.dnnconf

    1.3.2 WAV

    WAV

    snd01.wav : snd02.wav :

    OS Windows

    to-file.bat

    .binwindowsjulius.exe -C myconf.jconf -C am-dnn 2.jconf -dnnconf julius.dnnconf -charconv utf-8 sjis

    to-file.bat myconf.jconf main.jconf

    7

  • myconf.jconf

    1 ## 2 -logfile log.txt3 -quiet45 ## 6 -input rawfile7 -filelist flist.txt8 -spsegment9

    10 ## 11 -outfile1213 ## 2 - gram ,3- g r a m14 -d model/lang_m/bccwj .60k.bingram1516 ## 17 -v model/lang_m/bccwj .60k.htkdic1819 #### ####20 -b 150021 -b2 10022 -s 50023 -m 1000024 -n 3025 -output 1 # 2 26 -zmeanframe # D C ( H T K )2728 #### ####29 -rejectshort 800 #

    6, 7, 11 -input rawfile WAV -filelistWAV Julius

    flist.txt 7

    snd01.wavsnd02.wav

    11 -outfile .out

    snd01.out snd02.out

    2

    to-file.bat

    snd01.outsnd02.out

    8

  • 1snd01.out

    1 sentence1: 2 wseq1: + + + + + + + +

    + + 3 phseq1: sp_S | o_B N_I s_I e:_E | n_B i_I N_I sh_I i_I k_I i_E | n_B o_E | sp_S |

    t_B e_I s_I u_I t_I o_E | o_S | sh_B i_E | t_B e_E | i_S | m_B a_I s_I u_E |

    sp_S

    4 cmscore1: 0.902 0.844 0.894 0.312 0.790 0.992 0.215 0.988 0.995 0.543 0.769 1.0005 score1: 421.860229 (AM: 716.412537 LM: -294.552307)

    2snd02.out

    1 sentence1: 2 wseq1: + + + + + + 3 phseq1: sp_S | k_B e_I q_I k_I a_E | w_B a_E | sp_S | d_B o:_E | d_B e_I sh_I o:_E

    | k_B a_E | sp_S

    4 cmscore1: 0.591 0.972 0.925 0.425 0.848 0.980 0.733 1.0005 score1: 509.048126 (AM: 665.216736 LM: -156.168594)

    sentence1: wseq1:

    Julius Julius

    9

  • 1.4 OpenSSL

    OpenSSL openssl OpenSSL

    https://www.openssl.org/

    OpenSSL

    SSL 2.0, 3.0 TLS 1.0, 1.1, 1.2 DTLS 1.0, 1.2

    Blowfish, Camellia, DES, RC2, RC4, RC5, SEED, IDEA, AES

    MD5, MD2, SHA-1, SHA-2, MDC-2

    OpenSSL

    RSA, DSA, Diffie-Hellman

    1.4.1

    OpenSSLopensslRSA

    1.4.1.1 openssl genrsa

    openssl genrsa > secret.key

    secret.key

    openssl genrsa 2048 > secret.key

    2048 secret.key

    secret.key public.key openssl rsa -pubout < secret.key > public.key

    10

  • openssl rsa -pubout

    1.4.1.2 public.key original.dat

    encrypted.dat

    openssl rsautl -encrypt -pubin -inkey public.key < original.dat > encrypted.dat

    openssl rsautl -encrypt, -pubin, -inkey -inkey encrypted.dat

    1.4.1.3 secret.key encrypted.dat decrypted.dat

    openssl rsautl -decrypt -inkey secret.key < encrypted.dat > decrypted.dat

    openssl rsautl -decrypt, -inkey -inkey decrypted.dat

    1.4.2

    1. original.dat sign.dat openssl

    openssl dgst -sha1 -sign secret.key < original.dat > sign.dat

    original.dat sign.dat secret.key

    2.

    3. openssl

    openssl dgst -sha1 -verify public.key -signature sign.dat < original.dat

    public.key

    Verified OK

    11

  • Verification Failure

    openssl dgst -sha1 -md5-sha256-sha512

    12

  • 2

    2.1 GMP/MPFR

    C Java 7 GMPGNU Multi-PrecisionLibrary MPFR C C++

    GMP

    https://gmplib.org/

    GMPGMP MPFR MPFR http://www.mpfr.org/

    GMP MPFR C/C++

    2.1.1 GMP

    GMP mpz tmpq tmpf t 8

    GMP 3

    1. mpz tmpq tmpf t GMP

    2. Clong, double, char* GMP

    3.

    4.

    12

    1:

    mpz t mpz init( x ) mpz t xmpz inits( x1, x2, ,(mpz t *)0 ) mpz t x1,x2,

    0 mpq t mpq init( x ) mpq t x

    mpq inits( x1, x2, ,(mpq t *)0 ) mpq t x1,x2, 0/1

    mpf t mpf init2( x, p ) mpf t x p 0 p mp bitcnt t

    7

    8mpz t Z mpq t Q

    13

  • 2: mpz t x mpq t x mpf t x

    long n mpz set si( x, n ) mpq set si( x, n, d ) mpf set si( x, n )x n / (unsigned long)d

    double n mpz set d( x, n ) mpq set d( x, n ) mpf set d( x, n )

    mpz t n mpz set( x, n ) mpq set z( x, n ) mpf set z( x, n )

    mpq t n mpz set q( x, n ) mpq set( x, n ) mpf set q( x, n )

    mpf t n mpz set f( x, n ) mpq set f( x, n ) mpf set( x, n )

    (char *)n mpz set str( x, n, Base ) mpq set str( x, n, Base ) mpf set str( x, n, Base )

    Base int

    3: GMP (char *)s

    mpz t n mpz get str( s, Base, n )

    mpq t n mpq get str( s, Base, n )

    mpf t n mpf get str( s, &E, Base, Digit, n )

    2,3 Base int E mp exp t E Digit size t Digit 0

    mpf t mpf t mpf set default prec

    mpf set default prec()

    mp bitcnt t mpf t a, b, c, mpf inits mpf inits( a, b, c, , (mpf t *)0 )

    2.1.1.1 GMP

    mpz t mpz add ui( y, x, n) : (mpz t)y (mpz t)x + (unsigned long)nmpz add( y, x