building a web api platform with open source oauth 2.0, rest, and nosql (javaone 2012)
DESCRIPTION
More and more companies provide Web APIs for their core services as an effective way to foster an ecosystem, but you need to have an API platform to host and manage the Web APIs. Building one from scratch can be challenging. This session teaches you how to create your API platform based on oAuth 2.0, REST, and NoSQL technologies by using open source stacks, including Apache projects such as Tomcat, Tuscany, Wink, Amber, and HTTP clients together with NoSQL solutions such as MongoDB and Redis. From real-world experience, you’ll learn the key components and techniques for creating a robust and scalable Web API server.TRANSCRIPT
BUILDING A WEB API PLATFORM WITH OPEN SOURCE OAUTH 2.0, REST, AND NOSQL
SESSION 6946JAVAONE 2012
Raymond Feng
Luciano Resende
About the speakers
Raymond Feng Staff Software Engineer – Shutterfly, Inc. Member – Apache Software Foundation Committer – Apache Tuscany, Wink, Amber Co-author – Tuscany SCA In Action
Agenda
Why a Web API platform The key components of Web API platform
stack OAuth 2.0 Http Reverse Proxy Metrics API discovering, exploring and playing
Q&A
Overview of a Web API Platform
Why a Web APIs platform?
Why Web API? A great way to build the ecosystem For some companies, APIs = products Proliferation of mobile clients Universal access for internal systems/web or
mobile fronts/third party apps This talk is about the
platform/infrastructure behind the curtain to support Web API calls. It’s NOT about Web API design/development/security.
What’s behind the scene?
What’s behind an oAuth 2.0 protected REST API call? GET
https://api.<mycompany>.com/me/albums?access_token=<oAuth 2.0 access token>
POST https://api.<mycompany>.com/me/albums?access_token=<oAuth 2.0 access token> Content-Type: application/json Accept: application/json Request body: {“name”: “Summer 2012”}
A Web API Platform
HTTP R
everse
Prox
y
Client Application
Auth
Metrics
Throttling
…
Mediation
ProtectedResource
Infrastructure
Distributed data grid(service registry, metrics, quota/usage)
ProtectedResource
Security(Identity management, client application registration, user authentication, token
management, resource ACLs)
User Sign-in Page
Persistence
QoS plugins
Client Application
Key components
Authentication/Authorization (oAuth 2.0) Reverse proxy (API dispatching) Mediation Monitoring Analytics Backend infrastructure
Client/Token management Service registry Metrics aggregation Metrics visualization
Developer portal Application registration Dashboard API documentation API playground
API invocation flow
An API request (REST/JSON) comes in using HTTPS Threat detection Authentication (oAuth 2.0 access_token) Authorization (token – scopes – resources) Rate limiting Caching Mediation Routing Analytics API Logic
oAuth 2.0
oAuth 2.0
Based on Apache Amber which implements the latest oAuth 2.0 spec http://tools.ietf.org/html/draft-ietf-oauth-v2-
31 http://incubator.apache.org/amber/
Additional SPIs are identified for oAuth 2.0 related metadata management and integration with the security infrastructure
oAuth 2.0 concepts
OAuth defines four roles: resource owner: An entity capable of granting
access to a protected resource (e.g. end-user). resource server: The server hosting the protected
resources, capable of accepting and responding to protected resource requests using access tokens.
client: An application making protected resource requests on behalf of the resource owner and with its authorization.
authorization server: The server issuing access tokens to the client after successfully authenticating the resource owner and obtaining authorization.
oAuth 2.0 flows/grant types
An authorization grant is a credential representing the resource owner's authorization (to access its protected resources) used by the client to obtain an access token.
oAuth 2.0 specification defines four grant types: authorization code implicit resource owner password credentials client credentials
User cases: Trusted client applications
Trusted clients include: Internal applications (running at the server
side or client side) Mobile clients
Mapping to oAuth 2.0 Client credentials for access token (super
user) Resource owner password credentials for
access token (run as the “resource owner”)
Use cases: Third party applications
We plan to roll out web APIs to 3rd party applications
oAuth 2.0 will be used as the authentication/authorization mechanism
Mapping to oAuth 2.0 Authorization code flow Implicit grant flow
oAuth 2.0 SPI – Client registration
Client applications will be registered Contact e-mail Application name/description Callback URIs Authentication scheme
oAuth 2.0 SPI – resource ACL
Determine if a resource is protected Public Client-id specific Resource-owner specific
Who are the resource owners? Scopes of the access
{"id": "media_resource","operations": [ "ALL”],"path": "/media","scopes": [ "media”]}
{"scope": "media","description": "Media","expiresIn": 604800,"requiredAccessLevel": 1,"resourceOwnerAuthorizationRequired": false}
oAuth 2.0 SPI – Authentication & Authorization
Make sure a client is registered Make sure a token/code is valid Authenticate a client using
client_id/client_secret Authenticate a resource owner using
user name/password Check the token against the protected
resources (ACL) Establish the principal/subject
oAuth 2.0 SPI – Token Management
Generate access/refresh tokens for a given client, resource owner, and scopes
Generate authorization codes for a given client, resource owner, and scopes
Look up the token metadata based on the token string
Look up the authorization code metadata based on the code string
Expiration Refresh tokens MongoDB as the backend store for tokens
Internet
oAuth 2.0 infrastructure
Authorization
Endpoint
TokenEndpoint
ClientRegistration
Endpoint
Protected Resources(such as
user media or address
book)
Client Registration
s
Tokens and Authorizatio
n Codes
Users (Resource Owners)
Authenticator
Token Manager
Client Registration
Manager
Resource Access
Manager Resource Permissions and Scope Definitions
oAuth 2.0 Resource
Filter
HTTP proxy
Resource
Owner
Client
User Agent
(browser)
Authorization Server
ResourceServer
Mapping oAuth 2.0 scopes
oAuth 2.0
token
ScopeScope
Scope
Client
Resource Owner
Protected
ResourceProtecte
d Resource
Protected
Resource
GET /services/addressbook/…
POST/services/addressbook/…
GET /services/media/…
…
…
See an example at:http://developers.facebook.com/docs/reference/api/permissions/
oAuth 2.0 metadata model
Client/Resource management (administrative)
We need to have UI to manage the resource endpoints (URI
patterns and HTTP operations) define oAuth 2.0 scopes to map to a list of
resource access permissions Manage client applications (enable/disable,
setting quota, …) Manage access tokens/authorization codes
Dashboard
Monitor the API usages (administrative) By client id By user id By resource URIs By timeline
Let developers see their client applications (developer) Registrations Usages Granted permissions
REST APIs for API management Get the list of defined scopes Display a scope Get the list of defined resources Display a token Display an authorization code Display a client registration Get the list of registered clients for a given user List tokens by client id/user id List authorization codes by client id/user id List granted permissions
Performance
oAuth 2.0
HTTP reverse proxy
DO NOT deploy the services on the api server directly
API server dispatches API calls to the back-end services
Open source tools: Apache http components: http
://hc.apache.org/
HTTP reverse proxy – connection management
HTTP connection pool Persistent connections (keep-alive)
Check the keep alive settings at the backend web servers
Chunked transfer encoding support Some Nginx servers don’t support chunked
mode Make sure the http entity is “consumed” so
that the http connection will be released back to the pool
http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html
Routing/Proxying
URL mapping service registry URL templating
<uriMapping><source>/addressbook/{uid}</source>
<target>http://backend.xyz.com/services/addressbook/{uid}</target>
</uriMapping>
api.xyz.com/addressbook/me/contatcs The “me” or “self” will be replaced with the
resource owner from the oAuth 2.0 access token
Sync vs. Async
Sync: Servlet 2.5 and Apache HTTP client 4.x
Async: Servlet 3.0 async filter (Tomcat 7.0.x or Jetty 8.x) Apache HttpAsyncClient 4.0 beta 2
final AsyncContext asyncContext = request.startAsync();
asyncContext.start(new Runnable() {public void run() {
asyncDispatch(asyncContext, target); } });
Java vs. Node.js
Which one is better? Overhead Scalability
Servlet 2.5 sync mode + Apache http client
Servlet 3.0 async mode + Apache http async client
Node.js event-driven mode + http
Mediations
Some examples: Protocol/data translation
XML JSON API tracking
https://github.com/codahale/metrics http://graphite.wikidot.com/
Analytics CORS enablement
API discovery and documentation
API discovery and documentation Some apis/tools that help developers to discover
and explore the apis A spec to describe the apis (urls, methods,
input/output/exception data model) A UI to discover and explore the apis Some integration with the api implementation stacks
such as JAX-RS to introspect the api signatures Client code generation tool
Open source tools http://swagger.wordnik.com/ https://developers.google.com/discovery/
Swagger UI
Q&A
Thank you!