Developing for Google Glass

February 4th 2014

Mark Billinghurst, Gun Lee HIT Lab NZ

University of Canterbury

The Glass Class – Feb 17th – 21st

  Intensive Glass Course   See http://arforglass.org/

Overview   Introduction to Glass   Concept Design Non-programmers   Rapid Prototyping Simple programming   Mirror API - Server Development   Native Applications - Java/Android Developers   More Resources

Introduction

Google Glass

View Through Google Glass

Always available peripheral information display Combining computing, communications and content capture

User Interface

  dfasdf

  Hardware  CPU TI OMAP 4430 – 1 Ghz   16 GB SanDisk Flash,1 GB Ram   570mAh Battery

  Input   5 mp camera, 720p recording, microphone  GPS, InvenSense MPU-9150 inertial sensor

  Output   Bone conducting speaker   640x360 micro-projector display

User Experience   Truly Wearable Computing

  Less than 46 ounces

  Hands-free Information Access   Voice interaction, Ego-vision camera

  Intuitive User Interface   Touch, Gesture, Speech, Head Motion

  Access to all Google Services   Map, Search, Location, Messaging, Email, etc

  https://glass.google.com/glassware Glassware Applications

Concept Design Tools

Glass Sim – http://glasssim.com/

  Simulate the view through Google Glass   Multiple card templates

Glass UI Templates

  Google Glass PSD Templates   http://glass-ui.com/   http://dsky9.com/glassfaq/the-google-glass-psd-template/

  UXPin Wireframing Tool   http://uxpin.com/

Glass Application Storyboard

  http://dsky9.com/glassfaq/google-glass-storyboard-template-download/

Glass Application Flow

Rapid Prototyping

Processing   Programming tool for Artists/Designers

  http://processing.org   Easy to code, Free, Open source, Java based   2D, 3D, audio/video support

  Processing For Android   http://wiki.processing.org/w/Android  Generates Glass Ready .apk file

Development Environment

Hello World //called initially at the start of the Processing sketch!void setup() {! size(640, 360);! background(0);!} !!//called every frame to draw output!void draw() {! background(0);! //draw a white text string showing Hello World! fill(255);! text("Hello World", 50, 50);!}!

Demo

Hello World Image PImage img; // Create an image variable!!void setup() {! size(640, 360);! //load the ok glass home screen image! img = loadImage("okGlass.jpg"); // Load the image into the program !}!!void draw() {! // Displays the image at its actual size at point (0,0)! image(img, 0, 0);!}!

Demo

Touch Pad Input   Tap recognized as DPAD input

!void keyPressed() {!!if (key == CODED){!! !if (keyCode == DPAD) {!

!// Do something ..!

  Java code to capture rich motion events   import android.view.MotionEvent;!

Motion Event //Glass Touch Events - reads from touch pad!public boolean dispatchGenericMotionEvent(MotionEvent event) {! float x = event.getX(); // get x/y coords ! float y = event.getY();! int action = event.getActionMasked(); // get code for action! ! switch (action) { // let us know which action code shows up! !case MotionEvent.ACTION_DOWN:! ! !touchEvent = "DOWN";! ! !fingerTouch = 1;! !break; ! !case MotionEvent.ACTION_MOVE:! ! !touchEvent = "MOVE";! ! !xpos = myScreenWidth-x*touchPadScaleX;! ! !ypos = y*touchPadScaleY;! !break;!

Demo

Sensors   Ketai Library for Processing

  https://code.google.com/p/ketai/

  Support all phone sensors  GPS, Compass, Light, Camera, etc

  Include Ketai Library   import ketai.sensors.*;!  KetaiSensor sensor;!

Using Sensors   Setup in Setup( ) function

  sensor = new KetaiSensor(this);!  sensor.start();!  sensor.list();

  Event based sensor reading void onAccelerometerEvent(…)!{! accelerometer.set(x, y, z);!}!

Sensor Demo

Other Tools   App Inventor – visual programming for Android

  http://appinventor.mit.edu/

  Phone Gap – Android Apps using Javascript   http://phonegap.com/

  Phone Gap for Glass   https://github.com/brendajin/googleglass-phonegap-example

  WearScript   Using Javascript/HTML to develop Glass Applications   http://www.wearscript.com/

Mirror API + Glass GDK

Timeline Metaphor

Glassware and Timeline

Glassware and Timeline   Static Cards

  Static content with text, HTML, images, and (link to) video.   e.g. notification messages, news clip

  Live Cards   Dynamic content updated frequently.   e.g. compass, timer

  Immersions   Takes over the whole control, out from timeline.   e.g. interactive game

Glassware Development   Mirror API

  Server programming, online/web application   Static cards / timeline management

  GDK   Android programming, Java (+ C/C++)   Live cards & Immersions

  https://developers.google.com/glass/

  REST API   Java servlet, PHP, Go,

Python, Ruby, .NET   Timeline based apps

  Static cards -  Text, HTML, media attachment (image & video) -  Standard and custom menu items

  Manage timeline -  Subscribe to timeline notifications -  Sharing with contacts -  Location based services

Mirror API

Mirror API based Web App

3. Insert a static card

User sees the card

Glassware Web app

Develop with Mirror API   Create a Google APIs Console project   Prepare a web server

  https callback for OAuth 2.0  Minimum storage for credentials

  Create a web application   Java servlet, PHP, Go, Python, Ruby, .NET   Implement Oauth 2.0 authentication  Use Mirror API to make REST API calls

-  Wrapper classes/methods provided

Example

Example

Example

Example

Example On your Glass

GDK   Glass Development Kit   Android 4.0.3 ICS + Glass specific APIs   Use standard Android Development Tools

GDK   GDK add-on features

  Timeline and cards  Menu and UI   Touch pad and gesture  Media (sound, camera and voice input)

Live Cards vs. Immersions   Live cards display frequently updated

information to the left of the Glass clock.   Integrate rich content into the timeline   Simple text/images to full-blown 3D graphics

  Immersions let you build a user experience outside of the timeline.   Build interactive experiences   Extra control, less user input constraints

Live Cards vs. Immersions

Live Cards

Immersions

Develop with GDK   Android 4.0.3 (API 15) SDK and GDK Sneak

Peek from the Android SDK Manager.   Project settings:

  Minimum and Target SDK Versions: 15   Compile with: GDK Sneak Peek   Theme: None (allows the Glass theme to be applied.)

  GDK samples   File > New Project > Android Sample Project

  On Glass, turn on USB debugging   Settings > Device Info > Turn on debug

Example   Android App with a full-screen Activity   Add Voice Trigger to launch   Handle touch input   Manipulating timeline

Example   Add Voice Trigger to launch

  Add intent filter to AndroidManifest.xml

Example   Add Voice Trigger to launch (continued)

  Add xml/voice_trigger.xml to res folder

  Add voice_trigger_title string resource

Example   Handle touch input

Example   Adding static cards to timeline

Summary   Use Mirror API if you need ...

  Use GDK if you need ...

  Or use both

Resources   Main Developer Website

  https://developers.google.com/glass/

  Glass Apps Developer Site   http://glass-apps.org/glass-developer

  Google Glass Emulator   http://glass-apps.org/google-glass-emulator

  AR for Glass Website   http://www.arforglass.org/

Books   Programming Google Glass

  Eric Redmond

  Rapid Android Development: Build Rich, Sensor-Based Applications with Processing  Daniel Sauter

More Information

  Mark Billinghurst  mark.billinghurst@hitlabnz.org

  Gun Lee   gun.lee@hitlabnz.org