The 4th Workshop for NExT++

CHUA Tat-Seng/ Sun Maosong/ Wendy Hall

SINGAPORE

1 Nov 2018

NEXT SEARCH CENTRE下一代搜索技术联合研究中心a NUS-Tsinghua-Southampton joint centre on extreme search

• NExT++: NUS-Tsinghua Centre on Extreme Search

o Research on Big Unstructured data Analytics with Applications in Wellness, Fintech and Smart Nation

o We are among the first to look into this topic in 2010

o Phase I: May 2010 to Sep 2016 with a Grant of S$11 million

Emphasis: Technology for unstructured data analytics

o Phase II: Oct 2016 to Sep 2021 with a Grant of S$12 million

Emphasis : Deep unstructured data analytics

o Additional Collaborator: Southampton University

o With active participation of over 15 professors, over 30 PhD students, and 10 full-time researchers

• We focus on unstructured data analyticso Two key challenges: big data and paradigm change

• Big Data Challenges:1) Multi-source Unstructured Data Analytics

2) Rich Media Analytics

3) Recommendation

4) Multimodal KG & Chabot

5) Fintech

6) Big Data Wellness Analytics

• Paradigm Change Challenges:1) From Video to 3D and VR

2) From Recommendation to Influence

Better Augmenting User Decisions

Human Experts

Internal Structured Data External Unstructured Data Knowledge Graph

AI Platform

Decisions on:

• Futures, Commodities, Assets forecasting

• Fraud Detection

• Leveraging temporal and relational data for stock prediction

– A neural network-based solution in a learning-to-rank fashion.

– A Temporal Graph Convolution to capture domain knowledge of stock relation.

LSTM: Learn stock-wise sequential embedding from historical data.TGC: Revised stock embeddings by modeling stock relations in a time-sensitive way.

➢ Long-Term aim is on price prediction & forecasting for Futures & Commodities

• Multi-frequency data fusion for long-term commodity prediction. – Fuse data that are updated in different frequencies.

– Capture long-term temporal dependencies for long-term prediction

– Working with a industrial partners for base meta prediciton.

Flood in Thailand Price of rice

• Entity extraction is well researched, we focus on relation inference and knowledge extraction from image & video– Bring level of video semantics to that of text/

language

– Work on advanced video applications: videoQA, multimodal Chabot, and video description

• Completing a new dataset with 100K videos– Offer as grand challenge topic for ACM MM

– Use as basis for transfer learning to other video domains

• Continue research on video to 3D

3) Multi-modal Knowledge Graph (MMKG)

▪ Building MMKGs for fashion, food/wellness and travel domains

▪ Developing multimodal Conversation system for fashion and travel domains

• Strong needs for conversational recommender systems.– Human-computer conversation has large commercial potentials, such as

Amazon Alexa, Google Assistant, Apple Siri

– Deep learning and reinforcement learning make the building of a dialogue system (DiaSys) require a minimum amount of hand-crafting

– DiaSys is strong at interacting with users.

– RecSys is strong at learning user preference.

Language Understanding

Dialogue State Tracking

Policy Learning

Response Generation

RecSys

Combine them to build a more intelligent assistant to better satisfy user information need!

▪ Key components are: lifestyle data gathering, analytics, intervention and action planning

▪ Support for both: personal self-management and primary cares practitioners

▪ Target: to reduce incident

of chronic diseases in Singapore by 25% in 5 years

▪ Target: a practical visual food recognition system • Accurate, robust and scalable

▪ 2017 Beijing Science and Technology Progress Award First Prize: Jie Tang, Juanzi Li, Bin Xu: Scientific Big Data Mining and Service platform

▪ 2017 VLDB Early Research Contribution Award: Guoliang Li

▪ CIKM2017 Best Full Paper Award: Guoliang Li et al.: Hike: A Hybrid Human-Machine Method for Entity Alignment in Large-Scale Knowledge Bases

▪ Nicolas D. Georganas Best Paper Award 2018 (ACM

Trans of MM): Hanwang Zhang, Xindi Shang, Huanbo

Luan, Meng Wang, Tat-Seng Chua: Learning from

Collective Intelligence: Feature Learning Using Social

Images and Tags (TOMM 13(1), 2016)

▪ IEEE Multimedia Best Department Paper Award

2018: Peng Cui, Wenwu Zhu, Tat-Seng Chua, Ramesh

Jain: Social-Sensed Multimedia Computing. IEEE

Multimedia 2016.

▪ ACM MM 2018 Best Demo Award 2018: Taoran

Tang, Hanyang Mao, Jia Jia: AniDance: Real-Time

Dance Motion Synthesize to the Song

The Programme

We need to advanced research on:

▪ Basic mm technologies for entity & relation recognition on text & video

▪ Deep analytics to predict onset of chronic diseases and others• must be explainable

▪ Support for recommendation, nudging & influence• must be fair, robust and personalized

▪ Data ownership, privacy and incentive• Users are owners of own data and must benefit from sharing

▪ Similar Issues in Fintech and other domains

Take-home at the end of this session:

▪ Key components of wellness research

▪ Techniques for interaction with, nudging and educating users

▪ What key research you can do and participate?

Take-home at the end of this session:

▪ Issues in use of blockchain in Fintech and Wellness?

▪ Use of blockchain as base for privacy preserving & marketplace?

▪ Key issues in trust and accountability

Take-home at the end of this session:

▪ How to achieve explainability, fairness and robustness in AI?

▪ Key research that we can do these emerging these topics

▪ Others

THANKS

Visit our Web Observatory:http:////WWW.NEXTCENTER.ORG/