智慧城市生態圈的ai應用技術 - cinsa.org.cn · deputy director, state key lab of iot...
TRANSCRIPT
![Page 1: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/1.jpg)
引領創新科技 建設智慧澳門
智慧城市生態圈的AI應用技術賈維嘉,講座教授、實驗室副主任
Weijia Jia, Chair Professor
Deputy Director, State Key Lab of IoT for Smart City
University of Macau 1
![Page 2: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/2.jpg)
引領創新科技 建設智慧澳門
➢ 智能搜索➢ 語義理解➢ 知識圖譜
➢ 對象關聯➢ 實體識別➢ 霧式管理
➢ 全效感知➢ 時空覆蓋➢ 三元協同
Feed
back
Co
ntro
l
Ap
p D
ata
2
Info/Data & Knowledge Flows
IoT/sensor networksApplications
候選子圖 Top-k結果圖知識圖譜用戶查詢圖
應用層
中間層
感知層
引領創新科技 建設智慧澳門
我們要做什麼?智慧城市物聯網生態圈!
![Page 3: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/3.jpg)
引領創新科技 建設智慧澳門
1. Background
2. AI at Bottom: Sensors/IoT
3. AI at Middle: Fog/Edge Networks
4. AI at Upper: Knowledge Graph
5. Discussions: AI S&P
Contents
3
![Page 4: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/4.jpg)
引領創新科技 建設智慧澳門
o 2018年7月獲國家科技
部批准
o 2018年10月8日揭牌
背景
4引領創新科技 建設智慧澳門
![Page 5: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/5.jpg)
引領創新科技 建設智慧澳門 5
智慧城市物聯網國家重點實驗室
![Page 6: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/6.jpg)
引領創新科技 建設智慧澳門
1. 全球已啟動或在建的智慧城市達1000多個
• 正在以20%以上的速度增加
• 歐盟在2009年提出《歐洲2020戰略》
• 日本提出 “智慧日本”
• 新加坡推動政府服務的電子化
• 荷蘭提出 Amsmarterdam 城市能源地圖
2. 國內截止2015年底
• 85%以上的城市都在進行智慧城市建設
• 智慧城市試點已接近300個
• 規劃對智慧城市的投資總規模將逾5000億元
智慧城市國內外研究現狀
6引領創新科技 建設智慧澳門
![Page 7: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/7.jpg)
引領創新科技 建設智慧澳門
澳門的挑戰與機遇
7引領創新科技 建設智慧澳門
安全防災
能源配置 生態影響
交通擁塞
![Page 8: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/8.jpg)
引領創新科技 建設智慧澳門
State Key Lab@University of Macau
Fundamental Research & Key
Technology 基础研究/关键技术
为支撑
Talent Training人才培养为平台
City Development 城市发展为主线
IOTSC 智慧城市物联网国家重点实验室
Applications应用支持为特征
8
![Page 9: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/9.jpg)
引領創新科技 建設智慧澳門
Research focuses
Smart Sensing 感
知技术
Data Analysis and Control
数据分析与控制技术
Advance Networking &
Fog Computing网络存储,传输与雾
计算
Platformof AI/Big
Data
Smart Grid
能源物联网
Smart
Transportation
基于物联网的智能交通
Public Safety Monitoring
and Disaster Prevention
城市公共安全监控和灾害防治
Big Data &
Intelligent Tech
大数据与智能技术
Intelligent Sensing
and Net Comm.
智能傳感與網路通信
9
![Page 10: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/10.jpg)
引領創新科技 建設智慧澳門
1. Background
2. Bottom: Sensors/IoT
3. Middle: Fog/Edge Networks
4. Upper: Knowledge Graph
5. Discussions: AI S&P
Contents
10
![Page 11: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/11.jpg)
引領創新科技 建設智慧澳門
➢ 智能搜索➢ 語義理解➢ 知識圖譜
➢ 對象關聯➢ 實體識別➢ 霧式管理
➢ 全效感知➢ 時空覆蓋➢ 三元協同
Feed
back
Co
ntro
l
Ap
p D
ata
11
Info/Data & Knowledge Flows
IoT/sensor networksApplications
候選子圖 Top-k結果圖知識圖譜用戶查詢圖
應用層
中間層
感知層
引領創新科技 建設智慧澳門
智慧城市物聯網生態圈!
![Page 12: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/12.jpg)
Our IoT devices
1. High frequency
radio, low power
consumption, wall
penetration
2. Real-time video
push and alarm to
user's mobile
phone
3. WeChat alarm and
personal service
inquiry and report
12
![Page 13: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/13.jpg)
Optimal patterns achieving 2D-coverage and k-connectivity: k = [1,…, 6]
Rs – Sensing RadiusRc – Connection Radius
2D Optimal Connectivity/Coverage
13
![Page 14: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/14.jpg)
引領創新科技 建設智慧澳門
1. Background
2. Bottom: Sensors/IoT
3. Middle: Fog/Edge Networks
4. Upper: Knowledge Graph
5. Discussions: AI S&P
Contents
14
![Page 15: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/15.jpg)
引領創新科技 建設智慧澳門
Key Tech: Fog/Edge Computing
PublicCloud A
Cloud/Servers
Fog/edge networks
App Environments
IoT & SensorNetworks
Feed
back
Co
ntro
l
EnterpriseCloud B
PrivateCloud C
Ap
p D
ata
IoT/sensors
Info/Data Flows
Bottom layer
Middle layer
Upper layer
How the data/tasks are handled by the
fog nodes?15
![Page 16: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/16.jpg)
引領創新科技 建設智慧澳門
Issues with Fog/Edge
• Resource allocations
• Power/energy awareness
• Real-time QoS
• Mobility service support
• Offloading
• Edge-to-edge data transfer
• Edge-to-edge AI services …
![Page 17: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/17.jpg)
Container Migrations
Migrating data/tasks➔Allocating resources from fog node 𝐹1to 𝐹3 through containers ➔ QoS ↑ costs↓ .
Migration Modeling and Learning Algorithms for Containers in Fog Computing, IEEE Transactions on Service
Computing (TSC), 2018 17
![Page 18: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/18.jpg)
• Containers are much lighter than VMs
• Compared with VM, application data and runtime libs are migrated without migrating the guest OS.
Containers: Why not VM?
18
![Page 19: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/19.jpg)
引領創新科技 建設智慧澳門
Delay
• Network delay:
• Computation delay:
• Total delay:
![Page 20: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/20.jpg)
引領創新科技 建設智慧澳門
Parameter Attentions
• Resource utilization:
• Total power consumption:
• Container migration cost:
![Page 21: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/21.jpg)
引領創新科技 建設智慧澳門
Nightmare: Dimensional Curse
• Delay of Ci at Fj
• Matrix of delays:
• Overall state space:
![Page 22: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/22.jpg)
• To minimize• Delay 𝑑𝑡𝑜𝑡𝑎𝑙• Power 𝑝𝑡𝑜𝑡𝑎𝑙• Migration cost 𝑚𝑡𝑜𝑡𝑎𝑙 subject to resource
requirements:min𝐶 = 𝜔1𝑑𝑡𝑜𝑡𝑎𝑙 +𝜔2𝑝𝑡𝑜𝑡𝑎𝑙 +𝑚𝑡𝑜𝑡𝑎𝑙
𝑠. 𝑡. 0 ≤ σ1 𝐶𝑗.𝑙 𝑡 =𝐹𝑖
𝐶𝑗.𝑎 𝑡
𝐹𝑖.𝑐≤ 1 𝑖 = 1,2, … ,𝑚
• Difficulties: • Bin-pack (NP-hard) • High dimensional state
Objectives
22
![Page 23: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/23.jpg)
• RL Loop:
• State 𝑆𝜏, Action 𝐴𝜏, and Reward 𝑅𝜏:• 𝑆𝜏 = 𝑋 𝜏 , 𝐶. 𝑙 𝑡0 + 𝜏 𝑇 , 𝐶. 𝑎 𝑡0 + 𝜏 𝑇 ∈ 𝑺.• 𝐴𝜏 = 𝐶. 𝑙 𝑡0 + 𝜏 𝑇 ∈ 𝑨.
• 𝑅𝜏 = − 𝜔1𝑑𝑡𝑜𝑡𝑎𝑙 𝜏 + 𝜔2𝑝𝑡𝑜𝑡𝑎𝑙 𝜏 + 𝑚𝑡𝑜𝑡𝑎𝑙 𝜏
• 𝑄 𝑣𝑎𝑙𝑢𝑒: 𝑄(𝑆𝜏−1, 𝐴𝜏−1)
Reinforcement Learning (RL)
23
![Page 24: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/24.jpg)
引領創新科技 建設智慧澳門
1. Background
2. Bottom: Sensors/IoT
3. Middle: Fog/Edge Networks
4. Upper: Knowledge Graph – The key
5. Discussions: AI S&P
Contents
24
![Page 25: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/25.jpg)
引領創新科技 建設智慧澳門
AI framework
PublicCloud A
Cloud/Servers
Fog/edge networks
App Environments
IoT & SensorNetworks
Feed
back
Co
ntro
l
EnterpriseCloud B
PrivateCloud C
Ap
p D
ata
IoT/sensors
Info/Data Flows
Bottom layer
Middle layer
Upper layerAI Framework
25
![Page 26: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/26.jpg)
引領創新科技 建設智慧澳門
• 重點研究知識圖譜與數據相結合的學習與表示技術算法
• 探討城市大數據的多源獲取,提出多源數據的融合技術,利用遷移學習和聯邦學習對城市大數據進行挖掘和分析
• 提出將結構化信息和數據結合的聯合模型,以刻畫知識圖譜內外實體的有效聯繫
26
城市大數據與智能技術
引領創新科技 建設智慧澳門
![Page 27: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/27.jpg)
27
研發具國際領先的智慧城市物聯網共性基礎科學及平臺方案;人-物&物-物之間的多關係提取與智能鏈接。
城市大數據與物聯網智能
代表性成果:
• 物聯網邊緣動態傳算:IEEE TPDS 2019,TSC2018
- 長尾任務動態調度
- 容器智能遷移
- 強化學習以降低解空間維度
• 語意實體多關係提取:AAAI 2019, ACL2019, EMNLP 2018, NAACL2019
- 實體多關係並行提取
- 優化膠囊網絡及设计新型對抗網路進行關係精煉提取
• 多反饋網絡實體信息抽象集成, IJCAI 2019,ACM Trans Data Mining/Discovery 2019
- 多反饋網絡建模- 關係網絡對演模型
![Page 28: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/28.jpg)
引領創新科技 建設智慧澳門
1. Extracting Entities (E)
2. Extracting Relations (R)
3. Predicating E-R in KG
Knowledge Graph (KG) --Fundamental Challenges
28
![Page 29: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/29.jpg)
引領創新科技 建設智慧澳門
What is Knowledge Graph?• Freebase, Concept Graph, NELL, WordNet…
• Triplet: (head entity--A, relation--r, tail entity--B)
• ”The height of Liu is 174cm” ➔ (Liu, height, 174cm)
![Page 30: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/30.jpg)
引領創新科技 建設智慧澳門
Entity Triple Extractions
• Entity linking forming KG• entity = node• relation = edge• fact = triple (A, r, B)
• Typical knowledge graphs• WordNet, Freebase, Nell• DBpedia, Wikidata• …
![Page 31: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/31.jpg)
App Example: Time-Sync CommentsVideo-Tagging
• Time-sync comments are crowdsourced & interactive
• Content involving current video, feelings of users or replies to other comments
• Obtain more accurate and specific tags
(1) Semantic relevance
(2) Real-time
(3) Inter-dependence
(4) Noise
31
![Page 32: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/32.jpg)
Mining Tags as Relations
• 1. Graph Construction
• 2. Topic Partitioning & Radius
• 3. Weight Distribution and Tag Extraction
ICME, 2017, ICME 2019
Outcomes: {User A, “Messi”, User B} { Video V, “Messi”, [0:18, 0:24]}
32
![Page 33: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/33.jpg)
引領創新科技 建設智慧澳門
1. Extracting Entities (E)
2. Extracting Relations (R)
3. Predicating E-R in KG
Fundamental challenges
33
![Page 34: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/34.jpg)
引領創新科技 建設智慧澳門
Two recent works on extracting relations from
Natural Language:
• Single Relation Extraction in Sentence
• – EMNLP-18
• Multiple Overlapping Relation Extractions
• – AAAI-19
2. Extracting Relations
34
![Page 35: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/35.jpg)
引領創新科技 建設智慧澳門
Problem and Challenges
• Multi-labeled Relation Extraction
ID Instances Relations
S1[Arthur Lee], the leader of Love, died on
Thursday in [Memphis].person/place_death
person/place_lived
S2[Arthur Lee] was born in [Memphis],
Tennessee, and lived there until 1952.person/place_birth
person/place_lived
S3
[Abraham Lincoln] was an American
statesman and lawyer who served as the
16th President of the [United States].
person/president_of
person/nationality
Overlapping
Relations
Discrete
Features
![Page 36: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/36.jpg)
引領創新科技 建設智慧澳門
Extracting Overlapping Features
• Capsule Network
Capsule Network inImage Recognition
Capsule Network inRelation Extraction (RE)
Examples for Capsule Network in image recognition come from Aurelien Geron’s lecture.https://youtu.be/pPN8d0E3900
[Arthur Lee] was born in [Memphis], Tennessee, and lived there until 1952.
place_birth
place_lived
![Page 37: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/37.jpg)
引領創新科技 建設智慧澳門
Attentive Capsule Network
• Attention-based Routing Algorithm
• Sliding-margin loss
![Page 38: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/38.jpg)
引領創新科技 建設智慧澳門
Experiments• Experiments on two previous benchmarks (NYT-10 and
SemEval-2010 Task 8)
![Page 39: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/39.jpg)
引領創新科技 建設智慧澳門
1. Extracting Entities (E)
2. Extracting Relations (R)
3. Predicating E-R in KG
Fundamental challenges
39
![Page 40: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/40.jpg)
引領創新科技 建設智慧澳門
KGC– Predict Entity/Relations
• Triples in KGs.• (Isaac Newton, _birth_in, Lincolnshire)
• How to predict the missing part: (A,r, ?), (A, ?, B), (?,r, B)
John was bornin ?
John was born in Miami
KnowledgeGraph
VectorSpace
![Page 41: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/41.jpg)
引領創新科技 建設智慧澳門
Translation-based Models
• Representation learning: Similar entities arerepresented as similar vectors
• Typical: TransE (Translation Embedding):
𝐴, 𝑟, 𝐵 ➔ Ԧ𝐴 + Ԧ𝑟 = 𝐵
• Optimization: minimizing Ԧ𝐴 + Ԧ𝑟 − 𝐵• E.g.: TransH, TransR, TransG, DKRL, TKRL,
SSP, …
TransT: Type-based Multiple Embedding Representations for Knowledge Graph Completion, Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2017.
![Page 42: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/42.jpg)
引領創新科技 建設智慧澳門
Contributions: Multiple Embeddings
• Assign one vector for each semantics
𝒑𝒕𝒓𝒖𝒆 𝑨, 𝒓, 𝑩 = σ𝑖=1𝑛𝐴 σ𝑗=1
𝑛𝐵 𝑤𝐴,𝑖𝑤𝐵,𝑗𝑝𝑡𝑟𝑢𝑒 𝑣𝐴,𝑖 , 𝑣𝑟 , 𝑣𝐵,𝑗
𝑝𝑡𝑟𝑢𝑒is defined as
𝑝𝑡𝑟𝑢𝑒 𝑣𝐴,𝑖 , 𝑣𝑟 , 𝑣𝐵,𝑗 = 𝑒− 𝑣𝐴,𝑖+𝑣𝑟−𝑣𝐵,𝑗
![Page 43: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/43.jpg)
引領創新科技 建設智慧澳門
Relation Prediction Classifications
• Predicting the missing relation when given two entities. FB15K is the benchmark dataset.
• Triple classifications predict if a given triple is correct. FB15K is the benchmark dataset of this task.
![Page 44: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/44.jpg)
引領創新科技 建設智慧澳門
1. Introduction
2. Bottom: Sensors/IoT
3. Middle : Fog computing
4. Upper: Knowledge Graph
5. Discussions: AI S&P
Contents
44
![Page 45: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/45.jpg)
From attacker’s view: What can attackers do in AI?
From AI’s view: How can we prevent attacks from AI?
ATTACKER
AI
privacysecurity
DATAMODEL
Security & Privacy
45
![Page 46: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/46.jpg)
DeepFakes
46
![Page 47: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/47.jpg)
SpamTracer: Manual Fake Review Detection for O2O Commercial Platforms by Using Geolocation Features, Inscrypt 2018.
Machine learning for detectingfake reviewers
47
![Page 48: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/48.jpg)
Bae H, Jang J, Jung D, et al. Security and Privacy Issues in Deep Learning, ACM Computing Surveys 2018
Potential Threats in AI
48
![Page 49: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/49.jpg)
引領創新科技 建設智慧澳門
➢ 智能搜索➢ 語義理解➢ 知識圖譜
➢ 對象關聯➢ 實體識別➢ 霧式管理
➢ 全效感知➢ 時空覆蓋➢ 三元協同
S&
P F
eed
back
Co
ntro
l
S&
P A
pp
Data
49
S&P Info/Data & KnowledgeFlows
S&P 5G & IoT/sensor networksApplications
候選子圖 Top-k結果圖知識圖譜用戶查詢圖
應用層
中間層
感知層
引領創新科技 建設智慧澳門
智慧城市物聯網生態圈
![Page 50: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/50.jpg)
引領創新科技 建設智慧澳門
MOST China; FDCT Macau, SAR; NSFC
University of Macau
Shanghai Jiaotong University
Acknowledgements
50
![Page 51: 智慧城市生態圈的AI應用技術 - cinsa.org.cn · Deputy Director, State Key Lab of IoT for Smart City ... •Typical knowledge graphs •WordNet, Freebase, Nell ... SpamTracer:](https://reader033.vdocuments.pub/reader033/viewer/2022053107/6074b2693e43fa4d1643b375/html5/thumbnails/51.jpg)
引領創新科技 建設智慧澳門
謝謝!Thanks!
51