new world of openstack restful api•openstack •openstack restful api ... •openstack provides...
TRANSCRIPT
Agenda • Who am I
• OpenStack
• OpenStack RESTful API
• Nova RESTful API
• Issues of Nova RESTful API
• Going Solutions
• Current Situation
• Summary
2
• Software developer
• Join OpenStack community for its quality assurance since 2012
• Working Areas – OpenStack QA (Quality Assurance)
– OpenStack Compute (Nova)
4
• OpenStack components
6
Neutron Network
Nova Compute
Glance Image
Cinder Block storage
・・・ ・・・
Horizon Dashboard
Keystone Identity
OpenStack
• OpenStack community – Rapid Growth
7
0
200
400
600
800
1000
1200
1400
A B C D E F G H I
Companies
Developers
OpenStack Versions
2013 2012 2011 2010 2014
• Community development – Importance of code review
• Can know the internal implementation
• Can know the trend
• Can get folks
– Importance of communication
• Daily : IRC or e-mail
• Weekly: IRC meeting
• Version: F2F at Design Summit
8
• RESTful API – Scalable and stateless
– Simple and consistent interfaces
• OpenStack provides many features through RESTful API
10
RESTful API
OpenStack Server
POST http://<api-server>/servers
RESTful Client
Create a virtual
machine
Hypervisor (kvm, etc)
VM 01
VM 02
• Many RESTful client libraries are available for OpenStack – Python - .NET
– Java - …
• Dashboard and commands request operations via RESTful APIs
OpenStack Server
RESTful API RESTful Client library
Commands $ nova .. $ cinder ..
Horizon Dashboard
• Each OpenStack component also is connected via RESTful APIs
Neutron Network
Nova Compute Glance
Image
Horizon Dashboard
Keystone Identity
Nova RESTful API
Commands $ nova .. $ cinder ..
• OpenStack core functions – Create a VM (virtual machine)
– Delete a VM
– Attach a volume to a VM
– …
• Scalable
• Stable
• Current API version is v2
• Nova v2 API contains 250+ APIs (Icehouse)
• API usage – Create a VM (virtual machine)
Nova v2 API
Nova server
RESTful Client
URL
{ “server”: { “name”: “VM01”, “flavorRef”: “42”, “imageRef”: “d57886..” } }
Body
http://<api-server>/servers
POST Hypervisor (kvm, etc)
VM01
• Inconsistent interfaces – Inconsistent naming
• “instance” or “server”, “tenant” or “project”
• CamelCase, or snake_case
17
{ “instanceAction”: [ { “action”: “reboot”, “events”: [], “instance_uuid”: “b48316c5-[..]”, “project_id”: “147”, [..] }
“Show insance actions” API
{ “server”: { “accessIPv4”: “”, “accessIPv6”: “”, “id”: “893c7791-[..]”, “name”: “test-server”, “tenant_id”: “a54313c7-[..]”, [..] } }
“Show server” API
CamelCase CamelCase
snake_case snake_case
• Inconsistent interfaces – Return incorrect success codes
• Should return 202(ACCEPTED) but some APIs return 201(CREATED) even if not complete to create a resource yet.
18
• Lacks of input validation – Cause internal errors, output a lot of error logs
in server, and return useless error messages
• Ex) Pass non-dict as metadata to “create a VM”
19
Nova v2 API
Nova API server
RESTful Client
{ “server”: { “name”: “VM01”, “flavorRef”: “42”, “imageRef”: “d57886..”, “metadata”: “non-dict” } }
Body
POST
Internal Error
The server has either erred or is incapable of performing the requested operation
HTTP500
# this bug has been already fixed.
• Lacks of input validation – Ignore unexpected data which a client sends,
and the request operates without some details which a client expects
20
Nova v2 API
Nova API server
RESTful Client
{ “server”: { “name”: “VM01”, “flavorRef”: “42”, “imageRef”: “d57886..” }, “os:scheduler_fints”: { “group”: “group-01” } }
Body
POST
typo
Create a VM without scheduler
hints HTTP202
• Nova v3 API (Havana -> Juno) – Re-define consistent naming
– Re-define consistent success codes
– Comprehensive input validation with JSONSchema
22
v3 API (Experimen
tal)
JSONSch
ema
input v
alid
atio
n
v2.0 API
(Current)
Nova Internal
implementation
Nova API Server Weak Validation
Inconsistent interfaces
Strong Validation
V2 API Client
V3 API Client
Consistent interfaces
Internal Error
• JSONSchema – Common data format definition library
– Clear, human- and machine-readable data format definition
23 23
{ “get_console_output”: { “length”: 100 } }
Correct request data
{ “type”: “object”, “properties”: { “get_console_output”: { “type”: “object”, “properties”: { “length”: { “type”: “integer”, “minimum”: -1 } } } }, “additionalProperties”: False, “required”: [“get_console_output”] } Data format definition with JSONSchema
Pass
• JSONSchema – Contains basic validation features in nature
– Ex) minimum
24 24
{ “get_console_output”: { “length”: -100 } }
Invalid request data
{ “type”: “object”, “properties”: { “get_console_output”: { “type”: “object”, “properties”: { “length”: { “type”: “integer”, “minimum”: -1 } } } }, “additionalProperties”: False, “required”: [“get_console_output”] } Data format definition with JSONSchema
Fail
• JSONSchema – Can deny undefined parameters
– Ex) additionalProperties: False
25 25
{ “get_console_output”: { “length”: 100 }, “undefined”: 100 }
Invalid request data
{ “type”: “object”, “properties”: { “get_console_output”: { “type”: “object”, “properties”: { “length”: { “type”: “integer”, “minimum”: -1 } } } }, “additionalProperties”: False, “required”: [“get_console_output”] } Data format definition with JSONSchema
Fail
• Use JSONSchema for input validation – Strong validation, then avoid internal errors
– Return useful error message if a client sends invalid format data
26 26
before JSONSchema validation
The server has either erred or is incapable of
performing the requested operation
Invalid input for field/attribute length.
Value: foo. foo is not of type ’integer'
after JSONSchema validation
• Use JSONSchema for input validation – Clean up logic code by separating validation
code
– Validation code also can be reduced
27 27
try: min_count = int(str(min_count)) except ValueError: msg = _('min_count must be an integer value') raise exc.HTTPBadRequest(explanation=msg) if min_count < 1: msg = _('min_count must be > 0') raise exc.HTTPBadRequest(explanation=msg)
'min_count': { 'type': 'integer', 'minimum': 1 }
Validation code of v2 API Validation code of v3 API
• Input validation development history – 2013.03 The prototype proposal
– 2013.04 The first proposal in Portland summit
– 2013.07 The framework patch was approved, but merger failed
– 2013.08 Do not merge * 2
• Input validation development history – 2013.08 Investigated input validations of all API
parameters and proposed necessary features.
– 2013.11 The second proposal in Hong Kong summit. We have gotten a consensus.
– 2013.11 The framework patch has been merged.
Necessary to merge API definition patches still.
• Input validation development status – Merged patches: 14
– Necessary to be reviewed/merged patches: 20
• Stopped v3 API development – Backward incompatibility against v2 API
– Large maintenance cost due to 250+ APIs for each API version
– Huge mail thread to keep/drop v3 API
32
v3 API (Experimen
tal)
JSONSch
ema
input v
alid
atio
n
v2.0 API
(Current)
Nova Internal
implementation
Nova API Server
Backward incompatibility
Strong Validation
V3 API Client
V2 API Client
Internal Error
• Proposed v2.1 API – Translate v2 request/response and use v3
implementation
– Strong API validation
– Keep maintenance cost low
33
v3 API (Experimen
tal)
JSONSch
ema
input v
alid
atio
n
v2.0 API
(Current)
Nova Internal
implementation
Nova API Server
v2.1 API (v2 – v3 Translator)
Keep compatibility with strong validation
V3 API Client
V2 API Client
Internal Error
Strong Validation Backward incompatibility
• Consensuses after Atlanta Summit – Implement v2.1 API in Juno cycle
– Implement JSONSchema validation for v2.1 API
– The future of v3 API is unclear now
34
• Nova v2.1 API will be available after Juno – Strong input validation
• The future of Nova v3 API is unclear – Consistent interfaces
– Strong input validation
• Join OpenStack community
– Code review is important
– Communication is important
36