device apis at takeoff conference

W3C Device APIsWhat are they, how to use them and where are they going

Diana Cheng

TakeOff ConferenceLille, January 2013

Diana Cheng – Vodafone Group, R&D

[email protected]

daianacheng

W3C DAP and SysApps member, formerly also Geolocation

January 2013 Take Off Conf – Device APIs 2

mailto:[email protected]

Contents

• Introduction: DAP Working Group and scope

• DAP specs and APIs (with code and demos)• HTML Media Capture

• getUserMedia

• Battery Status

• Vibration

• Network Service Discovery

• Pick Media Intent• Web Intents• Other Intents-based APIs

• Network Information

• Conclusions: What now and future work

• Questions?

3Take Off Conf – Device APIsJanuary 2013

W3C DAP Working Group

• Device APIs working group

• Scope of the group and its specifications has changed over time: recharted in 2011

• Edits a number of specs with different degrees of maturity

• Focuses on APIs that run in the browser security context and are therefore sandboxed

• Works also on „Task Forces“ together with the Web Intents and WebRTC WGs

4January 2013 Take Off Conf – Device APIs

So what are Device APIs?

• Browser-based Javascript APIs

• Provide access to the device hardware and services

• Allows us to build rich Web applications Making the Web THE platform!

• A lot of people seem to consider them part of HTML5

Such functionality was previously only available to native apps

PhoneGap, now Apache Cordova, was started to fill this void in mobile devices

5Take Off Conf – Device APIsJanuary 2013

Device APIs

Take Off Conf – Device APIsJanuary 2013 6

Specification Status Desktop Implementations

Mobile Implementations

HTML Media Capture LC Draft ✔✔✔ ✔✔✔

getUserMedia (WebRTC) PW Draft ✔✔✔

Battery API CR ✔ ✔

Vibration API CR ✔

Network Service Discovery PW Draft ✔

Contacts: Pick Contacts Intent PW Draft ?

“Gallery”: Pick Media Intent PW Draft ?

Proximity API LC Draft ✔

Ambient Lights API LC Draft ✔

Network Information Pretty bad? PW Draft

?

HTML Media Capture (1/2)

• extends the HTML Input Element with a capture attribute

• capture is a hint for the source of the media.

• Possible values: camera, camcorder, microphone, filesystem

• Media type could be: image, video, audio

• multiple won’t work on mobile devices due to the camera interface


<input type="file" accept="image/*;capture=camera” multiple>

HTML Media Capture (2/2)

Support:

My own experience:

• Desktop versions tested on Mac

• Opera Mobile seems to provide no real access to camera, only to device Gallery

• Demo won’t work in Opera Desktop (FileReader problem)*

* Opera for Mac 12.1 has a bug in the FileReader’s getDataAsURL and the demo for this API won’t work there.


Desktop Safari, Firefox, Chrome, Opera latest versions

Mobile Chrome for Android, Firefox for Android, Android 4.0 stock browser, Blackberry browser

getUserMedia (WebRTC) (1/3)

• Access to the device’s camera and microphone on the Web today!

• Farewell to plugins!

• Call will prompt the user for permission and invoke the errorCallback if denied

• Can request a LocalMediaStream with audio, video or both

constraints = {video: true, audio: true}

• Still prefixed in several implementations


navigator.getUserMedia (constraints, successCallback, errorCallback);


Support:

http://caniuse.com/stream





My own experience:

• Support in Opera Mobile 12.10?

• Making it work cross-browser requires considering the prefixes and careful testing

• spec keeps on changing: e.g. MediaStreamRecorder out? currently no standard way to record a device’s live stream:

(see recorder demo for a workaround)


recorder = localMediaStream.recorder();

localMediaStream.stop(); recorder.getRecordedData(function(blob) { //do smth…});

Battery Status API (1/2)

• Programmatic access to the device’s battery level and events!

• Avoid tasks in our Web apps that are not suitable for a low battery level: Power-efficient Web apps!

• Example use cases: • avoid network operations like syncing with the server, or do them with a lower

frequency

• Avoid power-consuming, processor-intensive animations


navigator.battery

Battery Status API (2/2)

• Attributes: level (]0 - 1.0]), charging, chargingtime*, dischargingtime*

chargingtime = Infinity (battery is discharging)chargingtime = 0 (battery is full)dischargingtime = Infinity (battery is charging)

• Events: levelchange, chargingchange, chargingtimechange, dischargingtimechange

• How often the update takes place (events fire) is left to the implementation

• Support: Firefox and Firefox for Android

• My own experience:

• Couldn’t be simpler to use! But it’s not too easy to test

* in seconds


Vibration API (1/1)

• Programmatically trigger feedback in the form of vibration

• Particularly nice for mobile games; not aimed for generic notification

• time is in milliseconds

• pattern is an array of consecutive vibration durations, e.g. [1000, 500, 2000]

• navigator.vibrate(0) or navigator.vibrate([ ]) stop the vibration


navigator.vibrate(time);navigator.vibrate(pattern);

Network Service Discovery API (1/2)

• Asynchronous discovery of local HTTP-based services in Web Apps via a single API call!

• Common discovery protocols: SSDP (UPnP), DNS-SD (ZeroConf)

• Use cases:• Remotely controlling local network appliances from Web Apps, e.g. Web-based

TV remote control

• Browse content in a local network and select the appropriate renderer among those found

• type: • ‘upnp:urn:schemas-upnp-org:service:ContentDirectory:1’,

urn:schemas-upnp-org:service:RenderingControl:1• zeroconf:_boxee-jsonrpc._tcp


navigator.getNetworkServices(type, success, error);

Network Service Discovery API (2/2)

• After discovery, the URL of a requested service is whitelisted by the browser for the given context; this allows to do cross-site requests via AJAX, Web Sockets, etc.

• Events for changes in service reachability: onserviceonline, onserviceoffline

• Only implementation at the moment is an experimental build by Opera supporting only UPnP and with no UI for user consent yet

http://dev.opera.com/articles/view/network-service-discovery-api-support-in-opera/(two very cool demos there too)

• My own experience:

• The Opera build works as expected

• Familiarity with the networking protocols (and XML/SOAP for UPnP) required Take Off Conf – Device APIsJanuary 2013 16

http://dev.opera.com/articles/view/network-service-discovery-api-support-in-opera/




Pick Media Intent

“The Pick Media Intent defines a Web Intent …”

Web Intents?


Android Intents

18Take Off Conf – Device APIsJanuary 2013 18

Web Intents

Sharing

19

1001 APIs to deal with…


The Problem in the Web today

Web Intents fixes that

• Applications register their ability to handle an Intent (action). They express a certain capability

• A given app can delegate functionality it doesn’t support to any of those apps that have registered it

20

<intent action="http://webintents.org/share" type="image/*" />

var intent = new Intent();intent.action = 'http://webintents.org/share';intent.type = 'image/*';intent.data = [imgData];

window.navigator.startActivity(intent);


Web Intents fixes that

• Users select which app to use

• The original site will then send the payload over to the handling app, and will get the result back in a callback

21

window.intent.postResult([ /*... some data ...*/ ]);


Pick Media Intent (1/2)

• Defines a new Web Intent in order to access the user’s media on the device or in the cloud

• Intent action: pick• Intent type: media• Extras: hints for the Intents Service

• Why: users store media on a variety of cloud services and Web apps should not require prior knowledge of them. Local or remote, the sources (services) just need to be registered with the browser


var intent = new Intent({ action: "http://webintents.org/pick", type: "http://w3.org/type/media", extras: { search: "olympic", filters: ["title", "description", "author”], limit: 100 }});

Pick Media Intent (2/2)

• The spec also defines a Media dictionary with the relevant Metadata

• Other DAP specs which are based on Web Intents:

• Pick Contacts Intent

• Web Intents Addendum for Local Services: Local Services (UPnP and mDNS) register as Service Intents, i.e. their ability to perform e.g. media playback


Sensor APIs

• Proximity Events API

• Ambient Lights API

• More coming soon: Temperature? Atmospheric Pressure?


And few issues with ….


Network Information API (1/4)




Premise: bandwidth on mobile is still a constraint

• Provide an API that gives information about the speed of the connection the device is currently on

• Use cases: • help us serve the right images to the device according to the available bandwidth

(as it is done according to screen size and resolution)

• avoid initiating network-hungry requests if they are unlikely to be completed

• Interesting research by PPK: http://www.quirksmode.org/blog/archives/2012/11/principles_of_m.html

http://www.quirksmode.org/blog/archives/2012/11/principles_of_m.html

http://www.quirksmode.org/blog/archives/2012/11/principles_of_m.html



What information to provide:

1. Connection type vs. bandwidth (actual speed)

Connection type Bandwidth

• can be obtained from the underlying OS

• can be a poor indicator. e.g : a crowded conference Wifi might offer slower connectivity than 3G

• tricky to measure • power and data consuming• highly dependent on both the

client and the server• can change quickly as the user

moves in/out/inside buildings, on a fast train, etc.

• a single “sample” might not be representative so as to make a decision



2. roaming vs. metered

• Should developers be making this kind of decisions?

• Should we rely on the browser to do batched requests (if indicated by the developer) in order to preserve a low-power radio state?

…..SUGGESTIONS?

roaming metered

• user agent could obtain this from the underlying OS

• definite indicator that networking should be limited? the user might not care about costs: e.g. I have a European flat rate from my employer, other people might have a lot of money

• in other words: is the user on a limited data plan? e.g: 1GB/month

• useful but this information is hard to obtain

• the user could be prompted but they can provide wrong information

And whatever happened to…

• Messaging

• Calendar

• System Information

• Gallery

• Telephony …?


SysApps Working Group

http://www.w3.org/2012/sysapps/

• A new scope: a different runtime environment and security model and APIs to build system-level Web Apps

• Standardize and converge specifications inspired by projects such as Mozilla’s Firefox OS and Tizen





What can you do?

Provide your feedback!

• DAP Mailing list: http://lists.w3.org/Archives/Public/public-device-apis/

• SysApps Mailing list: http://lists.w3.org/Archives/Public/public-sysapps/

or join?


http://lists.w3.org/Archives/Public/public-device-apis/

http://lists.w3.org/Archives/Public/public-device-apis/

http://lists.w3.org/Archives/Public/public-sysapps/

http://lists.w3.org/Archives/Public/public-sysapps/

Questions?


device apis at takeoff conference

Technology