asakusa framework スモールジョブ実行エンジン & windows対応

24
NAUTILUS 1 Asakusa Framework スモールジョブ実行エンジン & Windows対応 Akira, KAWAGUCHI Nautilus Technologies, Inc. 2015/1/8 Asakusa Framework 勉強会 2015冬

Upload: apirakun

Post on 03-Aug-2015

447 views

Category:

Documents


2 download

TRANSCRIPT

1. NAUTILUS 1 Asakusa Framework & Windows Akira, KAWAGUCHI Nautilus Technologies, Inc. 2015/1/8 Asakusa Framework 2015 2. NAUTILUS 2 n Twitter: @apirakun n Asakusa Framework n , n , , 3. NAUTILUS 3 Asakusa n 2014/11/20: Version 0.7.1 n 2015/1/5: Version 0.7.2 Windows () n Hortonworks Data Platform n Hive 0.14 4. NAUTILUS 4 5. NAUTILUS 5 n Hadoop 2 n Hadoop n 100MB Hadoop n 40 6. NAUTILUS 6 n 5 n Asakusa n 7. NAUTILUS 7 n n n 8. NAUTILUS 8 1: n MB 10MBHadoop5~10 GBHadoop n Hadoop 9. NAUTILUS 9 n n n BOM Hadoop Hadoop& SJEE 184 180 50 15 20 17 8 3 BOM 60 10 65 52 SJEE: 10. NAUTILUS 10 2: n Hadoop stage1 stage2 stage3 stage4 Hadoop Asakusa (Runtime) stage1 stage3 stage2 stage4 10GB 1MB 8GB 3MB 11. NAUTILUS 11 3: n com.asakusafw.inprocess.limit n asakusa-resouces.xml : Hadoop n (byte) 10MB com.asakusafw.inprocess.limit5242880 asakusa-resources.xml 12. NAUTILUS 12 n n n 13. NAUTILUS 13 CategorySummaryJobTest (4) Hadoop () 46.2 26.0 () 5.5 n 14. NAUTILUS 14 1) n build.gradle asakusa-test-inprocess-ext n : (TestDriver) - n Eclipsebuild.gradle dependencies { ... testRuntime group: 'com.asakusafw', name: 'asakusa-test-inprocess-ext', version: asakusafw.asakusafwVersion build.gradle 15. NAUTILUS 15 2) n Shafu Eclipse build.gradle 16. NAUTILUS 16 Windows n Windows n 17. NAUTILUS 17 Windows 18. NAUTILUS 18 Windows n Windows Windows n Windows 7 Windows 8.1 19. NAUTILUS 19 for Windows n Asakusa Framework for Windows Windows n Jinrikisha n YAESSHiveJinrikisha 20. NAUTILUS 20 n YAESS n Windows n n n Direct I/O 21. NAUTILUS 21 Shafu n AsakusaEclipse Eclipse n 22. NAUTILUS 22 Shafu (version 0.4.0) n Asakusa Framework n (new!) n Asakusa Framework (new!) ASAKUSA_HOME n DMDLAsakusa DSL n Asakusa (new!) (Hadoop) n Windows (new!) 23. NAUTILUS 23 n API (CSV) n WindowsYAESS n APIID n com.asakusafw.testdriver.tools.runner.BatchTestRunner n : (TestDriver) - int result = new BatchTestRunner("example.summarizeSales") .withArgument("date", "2011-04-01") .execute(); 24. NAUTILUS 24 n n Windows Windows ShafuEclipse