building a web api platform with open source oauth 2.0, rest, and nosql (javaone 2012)

BUILDING A WEB API PLATFORM WITH OPEN SOURCE OAUTH 2.0, REST, AND NOSQL

SESSION 6946JAVAONE 2012

Raymond Feng

Luciano Resende

About the speakers

Raymond Feng Staff Software Engineer – Shutterfly, Inc. Member – Apache Software Foundation Committer – Apache Tuscany, Wink, Amber Co-author – Tuscany SCA In Action

Agenda

Why a Web API platform The key components of Web API platform

stack OAuth 2.0 Http Reverse Proxy Metrics API discovering, exploring and playing

Q&A

Overview of a Web API Platform

Why a Web APIs platform?

Why Web API? A great way to build the ecosystem For some companies, APIs = products Proliferation of mobile clients Universal access for internal systems/web or

mobile fronts/third party apps This talk is about the

platform/infrastructure behind the curtain to support Web API calls. It’s NOT about Web API design/development/security.

What’s behind the scene?

What’s behind an oAuth 2.0 protected REST API call? GET

https://api.<mycompany>.com/me/albums?access_token=<oAuth 2.0 access token>

POST https://api.<mycompany>.com/me/albums?access_token=<oAuth 2.0 access token> Content-Type: application/json Accept: application/json Request body: {“name”: “Summer 2012”}

A Web API Platform

HTTP R

everse

Prox

y

Client Application

Auth

Metrics

Throttling

…

Mediation

ProtectedResource

Infrastructure

Distributed data grid(service registry, metrics, quota/usage)

ProtectedResource

Security(Identity management, client application registration, user authentication, token

management, resource ACLs)

User Sign-in Page

Persistence

QoS plugins

Client Application

Key components

Authentication/Authorization (oAuth 2.0) Reverse proxy (API dispatching) Mediation Monitoring Analytics Backend infrastructure

Client/Token management Service registry Metrics aggregation Metrics visualization

Developer portal Application registration Dashboard API documentation API playground

API invocation flow

An API request (REST/JSON) comes in using HTTPS Threat detection Authentication (oAuth 2.0 access_token) Authorization (token – scopes – resources) Rate limiting Caching Mediation Routing Analytics API Logic

oAuth 2.0

oAuth 2.0

Based on Apache Amber which implements the latest oAuth 2.0 spec http://tools.ietf.org/html/draft-ietf-oauth-v2-

31 http://incubator.apache.org/amber/

Additional SPIs are identified for oAuth 2.0 related metadata management and integration with the security infrastructure

oAuth 2.0 concepts

OAuth defines four roles: resource owner: An entity capable of granting

access to a protected resource (e.g. end-user). resource server: The server hosting the protected

resources, capable of accepting and responding to protected resource requests using access tokens.

client: An application making protected resource requests on behalf of the resource owner and with its authorization.

authorization server: The server issuing access tokens to the client after successfully authenticating the resource owner and obtaining authorization.

oAuth 2.0 flows/grant types

An authorization grant is a credential representing the resource owner's authorization (to access its protected resources) used by the client to obtain an access token.

oAuth 2.0 specification defines four grant types: authorization code implicit resource owner password credentials client credentials

User cases: Trusted client applications

Trusted clients include: Internal applications (running at the server

side or client side) Mobile clients

Mapping to oAuth 2.0 Client credentials for access token (super

user) Resource owner password credentials for

access token (run as the “resource owner”)

Use cases: Third party applications

We plan to roll out web APIs to 3rd party applications

oAuth 2.0 will be used as the authentication/authorization mechanism

Mapping to oAuth 2.0 Authorization code flow Implicit grant flow

oAuth 2.0 SPI – Client registration

Client applications will be registered Contact e-mail Application name/description Callback URIs Authentication scheme

oAuth 2.0 SPI – resource ACL

Determine if a resource is protected Public Client-id specific Resource-owner specific

Who are the resource owners? Scopes of the access

{"id": "media_resource","operations": [ "ALL”],"path": "/media","scopes": [ "media”]}

{"scope": "media","description": "Media","expiresIn": 604800,"requiredAccessLevel": 1,"resourceOwnerAuthorizationRequired": false}

oAuth 2.0 SPI – Authentication & Authorization

Make sure a client is registered Make sure a token/code is valid Authenticate a client using

client_id/client_secret Authenticate a resource owner using

user name/password Check the token against the protected

resources (ACL) Establish the principal/subject

oAuth 2.0 SPI – Token Management

Generate access/refresh tokens for a given client, resource owner, and scopes

Generate authorization codes for a given client, resource owner, and scopes

Look up the token metadata based on the token string

Look up the authorization code metadata based on the code string

Expiration Refresh tokens MongoDB as the backend store for tokens

Internet

oAuth 2.0 infrastructure

Authorization

Endpoint

TokenEndpoint

ClientRegistration

Endpoint

Protected Resources(such as

user media or address

book)

Client Registration

s

Tokens and Authorizatio

n Codes

Users (Resource Owners)

Authenticator

Token Manager

Client Registration

Manager

Resource Access

Manager Resource Permissions and Scope Definitions

oAuth 2.0 Resource

Filter

HTTP proxy

Resource

Owner

Client

User Agent

(browser)

Authorization Server

ResourceServer

Mapping oAuth 2.0 scopes

oAuth 2.0

token

ScopeScope

Scope

Client

Resource Owner

Protected

ResourceProtecte

d Resource

Protected

Resource

GET /services/addressbook/…

POST/services/addressbook/…

GET /services/media/…

…

…

See an example at:http://developers.facebook.com/docs/reference/api/permissions/

oAuth 2.0 metadata model

Client/Resource management (administrative)

We need to have UI to manage the resource endpoints (URI

patterns and HTTP operations) define oAuth 2.0 scopes to map to a list of

resource access permissions Manage client applications (enable/disable,

setting quota, …) Manage access tokens/authorization codes

Dashboard

Monitor the API usages (administrative) By client id By user id By resource URIs By timeline

Let developers see their client applications (developer) Registrations Usages Granted permissions

REST APIs for API management Get the list of defined scopes Display a scope Get the list of defined resources Display a token Display an authorization code Display a client registration Get the list of registered clients for a given user List tokens by client id/user id List authorization codes by client id/user id List granted permissions

Performance

oAuth 2.0

HTTP reverse proxy

DO NOT deploy the services on the api server directly

API server dispatches API calls to the back-end services

Open source tools: Apache http components: http

://hc.apache.org/

HTTP reverse proxy – connection management

HTTP connection pool Persistent connections (keep-alive)

Check the keep alive settings at the backend web servers

Chunked transfer encoding support Some Nginx servers don’t support chunked

mode Make sure the http entity is “consumed” so

that the http connection will be released back to the pool

http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html

Routing/Proxying

URL mapping service registry URL templating

<uriMapping><source>/addressbook/{uid}</source>

<target>http://backend.xyz.com/services/addressbook/{uid}</target>

</uriMapping>

api.xyz.com/addressbook/me/contatcs The “me” or “self” will be replaced with the

resource owner from the oAuth 2.0 access token

Sync vs. Async

Sync: Servlet 2.5 and Apache HTTP client 4.x

Async: Servlet 3.0 async filter (Tomcat 7.0.x or Jetty 8.x) Apache HttpAsyncClient 4.0 beta 2

final AsyncContext asyncContext = request.startAsync();

asyncContext.start(new Runnable() {public void run() {

asyncDispatch(asyncContext, target); } });

Java vs. Node.js

Which one is better? Overhead Scalability

Servlet 2.5 sync mode + Apache http client

Servlet 3.0 async mode + Apache http async client

Node.js event-driven mode + http

Mediations

Some examples: Protocol/data translation

XML JSON API tracking

https://github.com/codahale/metrics http://graphite.wikidot.com/

Analytics CORS enablement

API discovery and documentation

API discovery and documentation Some apis/tools that help developers to discover

and explore the apis A spec to describe the apis (urls, methods,

input/output/exception data model) A UI to discover and explore the apis Some integration with the api implementation stacks

such as JAX-RS to introspect the api signatures Client code generation tool

Open source tools http://swagger.wordnik.com/ https://developers.google.com/discovery/

Swagger UI

Q&A

Thank you!