mole: motion leaks through smartwatch sensors
TRANSCRIPT
MoLe: Motion Leaks through Smartwatch
SensorsB00902069 傅蕎
2
Demo video• https://www.youtube.com/watch?v=scZEHExzems
&feature=youtu.be
3
MoLe• Motion sensor leaks input words• Accelerometer & gyroscope on Samsung watch
4
Related work• Inferring keystroke on keyboard• Acoustic• Input timing• Electromagnetic
• Information from smart sensors• Typing on virtual keyboard• Keystroke cause vibration
5
Specialty• No need training user• No brand-new signal processing or learning algorithm• No right hand data, with watch on left hand• Must dictionary word• Must not typing error• Must “correct” typing way• Output top-K possible words• Not support whole sentence, difficulty in “space bar”• Can support different keyboard
6
System overview
7
Watch data• Watch & phone camera & keyboard log• CPC(character point cloud)
8
Keystroke detector - timing• Z axis• Error occurs• Moving from one key to another• Subtle motion, e.g. “asdf”
9
Keystroke detector - location• Origin as “F” key• Gravity removal• Watch’s axis → absolute• Kalman smoothing
10
Cloud fitting• UPC(unlabeled point cloud)• Reference template
11
Bayesian inference• Output ranked list of possible word• Conditional probability• # of keystrokes• Consecutive character, e.g. “er”, “sa”• Watch displacement• Character transition, e.g. “va” VS “ra”• Keystroke interval
12
Evaluation• 8 subjects familiar with English-typing• Variable factor• Word length• Bayesian model• Sampling rate• Keyboard
13
Question?
14
Thanks!:D