announcing aws step functions - december 2016 monthly webinar series

48
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Announcing AWS Step Functions Andy Katz, Senior Product Manager September 2016 Tim Bray, Senior Principal Engineer

Upload: amazon-web-services

Post on 08-Jan-2017

435 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Announcing AWS Step Functions

Andy Katz, Senior Product Manager

September 2016Tim Bray, Senior Principal Engineer

Page 2: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Enterprise Resource Planning

Finance

Inventory

Manu-facturing

��

��

��

��

Sales

From Monoliths to Microservices to Functions

Page 3: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

λλλ

DBMS

λ λλ λλ

λ λλ

λ

Queue

Modern Serverless app

Page 4: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Modern Serverless app

Page 5: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I want to sequence functions”

“I want to select functions based on data”“I want to retry functions”“I want try/catch/finally”

“I have code that runs for hours”

“I want to run functions in parallel”

Functions into apps

Page 6: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Scales out Doesn’t lose state Deals with errors/timeouts Easy to build & operate Auditable

Coordination must-haves

Page 7: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

AWS Step Functions…

…makes it easy to coordinate the components of distributed applications

using visual workflows.

Page 8: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Benefits of AWS Step Functions

Diagnose and debug problems faster

Adapt to change

Easy to connect and coordinate distributed components and

microservices to quickly create apps

Manages the operations and infrastructure of service coordination to ensure

availability at scale, and under failure

Productivity Agility Resilience

Page 9: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

How Does AWS Step Functions Work?

Page 10: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Application Lifecycle in AWS Step Functions

Visualize in the Console

Define in JSON Monitor Executions

Amazon Confidential

Page 11: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Define in JSON and Then Visualize in the Console

{ ”Comment”: “Hello World Example", "StartAt” : "HelloWorld”, "States” : { "HelloWorld” : { "Type” : "Task", "Resource” :

"arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME”,

"End” : true } }

}

Page 12: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Execute One or One Million

Start

End

HelloWorld

Start

End

HelloWorld

Start

End

HelloWorld

Start

End

HelloWorld

Start

End

HelloWorld

Start

End

HelloWorld

Start

End

HelloWorld

Start

End

HelloWorld

Page 13: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Monitor Executions from the Console

Page 14: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Monitor Executions from Amazon CloudWatch

Page 15: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I want to sequence functions”

AWS Step Functions, we can easily change and iterate on the application workflow of our food delivery service in order to optimize operations and continually improve delivery times. AWS Step Functions lets us dynamically scale the steps in our food delivery algorithm so we can manage spikes in customer orders and meet demand. Mathias Nitzsche, CTO, foodpanda

Page 16: Announcing AWS Step Functions - December 2016 Monthly Webinar Series
Page 17: Announcing AWS Step Functions - December 2016 Monthly Webinar Series
Page 18: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

var Twit = require('twit'); var T = new Twit(require('botfiles/config.js')); exports.handler = function myBot(event, context, callback) { var list = event.inputList; var textToTweet = list.shift(); var output = { inputList: list } T.post('statuses/update', { status: textToTweet }, function(err, reply) { if (err) { console.log('error:', err); context.fail(); } else { console.log('tweet:', reply); callback(null, output); } }); };

Page 19: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Demo

Page 20: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I want to select functions based on data”

“With AWS Step Functions, it was easy to build a multi-step product updating system to ensure our database and website always have the latest price and availability information.

AWS Step Functions let us replace a manual updating process with an automated series of steps, including built-in retry conditions and error handling, so we can reliably scale before a big show, and keep pace with rapidly changing fashions.”

Jared Browarnik, CTO, TheTake

Page 21: Announcing AWS Step Functions - December 2016 Monthly Webinar Series
Page 22: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

"ChoiceState": { "Type" : "Choice", "Choices": [ { "Variable": "$.productSource", "StringEquals": "screen-scrape", "Next": "ScreenScrapeState" },{ "Variable": "$.productSource", "StringEquals": "vendor-a", "Next": "VendorA" },{ "Variable": "$.productSource", "StringEquals": "vendor-b", "Next": "VendorB" },

{ "Variable": "$.productSource", "StringEquals": "vendor-c", "Next":"VendorC" },{ "Variable": "$.productSource", "StringEquals": "updateProduct", "Next":"UpdateProduct" } ], "Default": "ScreenScrapeState” }

Page 23: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I want to retry functions”

We get transient errors from a RESTful service we depend on, once every four or five times we call it. But if we keep retrying, it eventually works.

Page 24: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

{ "Comment": "Call out to a RESTful service", "StartAt": "Call out", "States": { "Call out": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:RestCallout", "Retry": [ { "ErrorEquals": [ "HandledError" ], "MaxAttempts": 10 } ], "End": true } } }

Page 25: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I want try/catch/finally”

AWS Step Functions makes it simple to coordinate information from many different infrastructure systems using easy to design workflows and create a more intelligent monitoring system for our Platform as a Service (PaaS).

With AWS Step Functions, we can reliably automate monitoring decisions and actions in order to reduce human intervention by over 60%, which improves infrastructure operation productivity and customer application availability on our platform.

Pedro Pimenta, VP R&D, OutSystems

Page 26: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

13 AWS Lambda Task States

6 Choice States

1 Fail State

“I want try/catch/finally”

Page 27: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Try/Catch/Finally for Media Management

Page 28: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

"Access Media": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:FindMedia", "TimeoutSeconds": 2, "Next": "Graceful Exit", "Retry": [ { "ErrorEquals": [ "States.Timeout" ], "IntervalSeconds": 2, "MaxAttempts": 2, "BackoffRate": 1.5 } ], "Catch": [ { "ErrorEquals": [ "States.ALL" ], "Next": "Clean Up" } ] },

Page 29: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Demo

Page 30: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I have code that runs for hours”

We need to gather data from our production line, in units of 8-hour shifts.“

Page 31: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Task States Poll or Push

AWS Lambda…On-premises

AWS container, EC2 instance, or…

Long poll

Push

Page 32: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Task State to Dispatch Work with Push

{ "StartAt": "HelloWorld", "States": { "HelloWorld": { "Type": "Task", "Resource":"arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME", "End": true } }

}

Page 33: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Task State to Dispatch Work by Polling

{ "StartAt": "HelloWorld", "States": { "HelloWorld": { "Type": "Task", "Resource":"arn:aws:states:REGION:ACCOUNT_ID:activity:FUNCTION_NAME", "End": true } }

}

Page 34: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

"NextShift": { "Type": "Wait", "TimestampPath": "$.ShiftStart", "Next": "Gather Plant Data" }, "Gather Plant Data": { "Type": "Task", "Resource": "arn:aws:states:ap-northeast-1:123456789012:activity:PlWatch", "TimeoutSeconds": 30000, "HeartBeatSeconds": 120, "Next": "Clean up" }

Page 35: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I want to run functions in parallel”

We want to send the captured image to three OCR providers and take the result with the highest confidence value.“

Page 36: Announcing AWS Step Functions - December 2016 Monthly Webinar Series
Page 37: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

"Send for OCR": { "Type": "Parallel", "Next": "Pick result", "Branches": [ { "StartAt": "Prep1", "States": { "Prep1": { "Type": "Pass", "Result": { "inputList": [ "OCR Provider 1" ] }, "Next": "Go1" }, "Go1": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:StatesBot", "End": true } }

Page 38: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Where does transient application state live?

In the machine, in JSON texts passing from state to state.A:

Q:

Page 39: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Input processing{ "title": "Numbers to add", "numbers": [ 3, 4 ] }

{ "Type": "Task", "InputPath": "$.numbers", "Resource": "arn:aws:lambda…" …

[ 3, 4 ]

Raw input:

State spec:

Task input:

Page 40: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Input processing

Q: InputPath not provided?A: State gets raw input as-is.

Q: InputPath is null?A: State gets an empty JSON object: {}

Q: InputPath produces plural output?A: State gets it wrapped in a JSON array.

Page 41: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Result placement{ "title": "Numbers to add", "numbers": [ 3, 4 ] }

{ "Type": "Task", "InputPath": "$.numbers", "ResultPath": "$.sum”, …

{ "title": "Numbers to add", "numbers": [ 3, 4 ], ”sum": 7}

Raw input:

State spec:

Task input:

Page 42: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Result placement

Q: ResultPath is null?A: State input is state output.

Q: ResultPath produces plural output?A: Not allowed, validator won’t accept.

Q: ResultPath not provided?A: Input discarded, raw output used.

Page 43: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

“I want to sequence functions”

“I want to select functions based on data”“I want to retry functions”“I want try/catch/finally”

Is this you?

“I have code that runs for hours”

“I want to run functions in parallel”

Page 44: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Where is AWS Step Functions Available?

Region Region Code

US East (N. Virginia) us-east-1

US East (Ohio) us-east-2

US West (Oregon) us-west-2

EU (Dublin) eu-west-1

Asia Pacific (Tokyo) ap-northeast-1

Page 45: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

aws.amazon.com/step-functions

Page 46: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Getting Started

Page 47: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

aws.amazon.com/step-functions

Page 48: Announcing AWS Step Functions - December 2016 Monthly Webinar Series

Thank you!