1. Hive 2015/05/26 LIVESENSE TechNight 1 2. / G Hive/PrestoHadoop / Job, , , 2015/05/26 LIVESENSE TechNight 2 3. () 2015/05/26 LIVESENSE TechNight 3 Game SNS 4. () 2015/05/26 LIVESENSE TechNight 4 Game SNS 5. Hive HadoopSQL-like v0.12, v0.13, v0.14(!?) 2015/05/26 LIVESENSE TechNight 5 select page, count(distinct(uid)) uu, count(1) pv from access where day = 2015-05-26 group by page UUPV 6. Hive v0.12 v0.13.1 v0.12 UDF v0.14 2015/05/26 LIVESENSE TechNight 6 7. 2015/05/26 LIVESENSE TechNight 7 hard 8. 2015/05/26 LIVESENSE TechNight 8 hard 9. /tmp 2015/05/26 LIVESENSE TechNight 9 $ ls -lt /tmp total 3721552 drwxr-xr-x 2 hive hive 4096 May 25 10:57 1826d22e-cbbf-4e7b-8e9a- e5d1a300994e_resources drwxr-xr-x 2 hive hive 4096 May 25 10:08 08df2afb-779e-425c-9cf3- cedca2cc051a_resources drwxr-xr-x 2 hive hive 4096 May 25 10:04 4d1edc92- ddd4-4aa2-904b-4b19182adf4f_resources Disk Usage 10. Hive Server RUNNING 2015/05/26 LIVESENSE TechNight 10 API Server Clients Web UI 11. 2015/05/26 LIVESENSE TechNight 11 hard 12. NameNode 2015/05/26 LIVESENSE TechNight 12 RCFile, ORCFileJSON.gz INSERT hourly, daily 13. 2015/05/26 LIVESENSE TechNight 13 hard 14. Hiveschema-on-read MySQLschema-on-write External table Fluentd HDFSJSON JSON PHP 2015/05/26 LIVESENSE TechNight 14 15. 2015/05/26 LIVESENSE TechNight 15 CREATE EXTERNAL TABLE `item_consume`( `time` bigint, `uid` int, `device` string, `request_status` int ) {time: 1432134000, uid: 22275, device: android, request_status: 1} {time: 1432167000, uid: 22275, device: iphone, request_status: false} (JSON) 16. 2015/05/26 LIVESENSE TechNight 16 CREATE EXTERNAL TABLE `item_consume`( `time` bigint, `uid` int, `device` string, `request_status` int ) {time: 1432134000, uid: 22275, device: android, request_status: 1} {time: 1432167000, uid: 22275, device: iphone, request_status: false} (JSON) 17. 2015/05/26 LIVESENSE TechNight 17 ClassCastException Y^Y^Y^Y^Y^Y^Y^Y^ NumberFormatException 18. NULL SerDe 2227522275, false 0, NULL Schema intfloat NULL 2015/05/26 LIVESENSE TechNight 18 19. 2015/05/26 LIVESENSE TechNight 19 hard 20. (1/2) [HIVE-9753] Wrong results when using multiple levels of Joins. When table alias of one of the table is null with left outer joins. JOIN aliasalias 2015/05/26 LIVESENSE TechNight 20 21. (2/2) [HIVE-9743] Incorrect result set for vectorized left outer join JOIN EXPLAIN v1.2.0 ( 2015/05/26 LIVESENSE TechNight 21 22. Hive v1.2.0 Redshift, BigQuery, TreasureData 2015/05/26 LIVESENSE TechNight 22 23. 2015/05/26 LIVESENSE TechNight 23 HAPPY HACKING