Optimizing Lambda performance for your serverless applications · 2020. 8. 1. · © 2020, Amazon...

Post on 24-Aug-2020

0 views 0 download

transcript

© 2020, Amazon Web Services, Inc. or its Affiliates.

James Beswick

Senior Developer Advocate, AWS Serverless

@jbesw

Optimizing Lambda performance for

your serverless applications

© 2020, Amazon Web Services, Inc. or its Affiliates.

About me

• James Beswick

• Email: jbeswick@amazon.com

• Twitter: @jbesw

• Senior Developer Advocate – AWS Serverless

• Self-confessed serverless geek

• Software Developer

• Product Manager

• Previously:

• Multiple start-up tech guy

• Rackspace, USAA, Morgan Stanley, J P Morgan

© 2020, Amazon Web Services, Inc. or its Affiliates.

Agenda

Memory and profiling

© 2020, Amazon Web Services, Inc. or its Affiliates.

How does Lambda work?

© 2020, Amazon Web Services, Inc. or its Affiliates.

Anatomy of an AWS Lambda function

Your function

Language runtime

Execution environment

Lambda service

Compute substrate

© 2020, Amazon Web Services, Inc. or its Affiliates.

Where you can impact performance…

Your function

Language runtime

Execution environment

Lambda service

Compute substrate

© 2020, Amazon Web Services, Inc. or its Affiliates.

Anatomy of an AWS Lambda function

Handler () function

Function to be executed

upon invocation

Event object

Data sent during Lambda

function Invocation

Context object

Methods available to

interact with runtime

information (request ID,

log group, more)

// Python

import jsonimport mylib

def lambda_handler(event, context):# TODO implementreturn {

'statusCode': 200,'body': json.dumps('Hello World!')

}

// Node.js

const MyLib = require(‘my-package’)const myLib = new MyLib()

exports.handler = async (event, context) => { # TODO implementreturn {

statusCode: 200,body: JSON.stringify('Hello from Lambda!')

}}

© 2020, Amazon Web Services, Inc. or its Affiliates.

Function lifecycle – worker host

Execute

INIT code

Execute

handler code

Full

cold start

Partial

cold start

Warm

start

Download

your code

Start new

Execution

environment

AWS optimization Your optimization

© 2020, Amazon Web Services, Inc. or its Affiliates.

Measuring with AWS X-Ray

Profile and troubleshoot

serverless applications:

• Lambda instruments

incoming requests and can

capture calls made in code

• API Gateway inserts tracing

header into HTTP calls and

reports data back to X-Ray

const AWSXRay = require(‘aws-xray-sdk-core’)

const AWS = AWSXRay.captureAWS(require(‘aws-sdk’))

AWSXRay.captureFunc('annotations', subsegment => {

subsegment.addAnnotation('Name', name)

subsegment.addAnnotation('UserID', event.userid)

})

© 2020, Amazon Web Services, Inc. or its Affiliates.

X-Ray Trace Example

© 2020, Amazon Web Services, Inc. or its Affiliates.

Three areas of performance

Throughput CostLatency

© 2020, Amazon Web Services, Inc. or its Affiliates.

Cold starts

© 2020, Amazon Web Services, Inc. or its Affiliates.

Function lifecycle – a warm start

Request made to Lambda’s API

Service identifies if

warm execution environments is

available

Invoke handlerComplete invocation

Yes

© 2020, Amazon Web Services, Inc. or its Affiliates.

Function lifecycle – a full cold start

Request made to Lambda’s API

Service identifies if

warm execution environments is

available

Invoke handler

Find available compute resource

Download customer code

Start execution environment

Execute INITComplete invocation

No

© 2020, Amazon Web Services, Inc. or its Affiliates.

Cold starts - Execution environment

The facts:

• <1% of production workloads

• Varies from <100ms to >1s

Be aware…

• You cannot target warm

environments

• Pinging functions to keep them

warm is limited

© 2020, Amazon Web Services, Inc. or its Affiliates.

Cold starts - Execution environment

The facts:

• <1% of production workloads

• Varies from <100ms to >1s

Be aware…

• You cannot target warm

environments

• Pinging functions to keep them

warm is limited

Cold starts occur when…

• Environment is reaped

• Failure in underlying resources

• Rebalancing across Azs

• Updating code/config flushes

• Scaling up

© 2020, Amazon Web Services, Inc. or its Affiliates.

Cold starts - Execution environment

Influenced by:

• Memory allocation

• Size of function package

• How often a function is called

• Internal algorithms

AWS optimizes for this part of a

cold start.

© 2020, Amazon Web Services, Inc. or its Affiliates.

Lambda + VPC – Major performance improvement

Before: 14.8 sec duration

After: 933ms duration

Read more at

http://bit.ly/vpc-lambda

© 2020, Amazon Web Services, Inc. or its Affiliates.

Cold starts - Static initialization

The facts:

• Code run before handler

• Used to initialize objects,

establish connections, etc.

• Biggest impact on cold-starts

Also occurs when…

• A new execution environment

is run for the first time

• Scaling up

Dependencies,

configuration

information

© 2020, Amazon Web Services, Inc. or its Affiliates.

Cold starts - Static initialization

Influenced by:

• Size of function package

• Amount of code

• Amount of initialization work

The developer is responsible for

this part of a cold start.

What can help…

• Code optimization• Trim SDKs

• Reuse connections

• Don’t load if not used

• Lazily load variables

• Provisioned Concurrency

© 2020, Amazon Web Services, Inc. or its Affiliates.

Provisioned Concurrency on AWS Lambda

Pre-creates execution environments,

running INIT code.• Mostly for latency-sensitive, interactive

workloads

• Improved consistency across the long tail of

performance

• Minimal changes to code or Lambda usage

• Integrated with AWS Auto Scaling

• Adds a cost factor for per concurrency

provisioned but a lower duration cost for

execution

• This could save you money when heavily

utilized

© 2020, Amazon Web Services, Inc. or its Affiliates.

Function lifecycle – a Provisioned Concurrency start

Function configured with

Provisioned Concurrency

Find available compute resource

Download customer code

Start execution environment

Execute INIT

© 2020, Amazon Web Services, Inc. or its Affiliates.

Function lifecycle – a Provisioned Concurrency

invocation

Request made to Lambda’s API

Service identifies if

warm execution environment is

available

Invoke handlerComplete invocation

Yes

This becomes the default for all

provisioned concurrency

execution environments

© 2020, Amazon Web Services, Inc. or its Affiliates.

Provisioned Concurrency – things to know

• Reduces the start time to <100ms

• Can’t configure for $LATEST

• Use versions/aliases

• Provisioning rampup of 500 per minute

• No changes to function handler code

performance

• Requests above provisioned concurrency follow

on-demand Lambda limits and behaviors for cold-

starts, bursting, pricing

• Still overall account concurrency per limit region

• Wide support (CloudFormation, Terraform,

Serverless Framework, Datadog, Epsagon,

Lumigo, Thundra, etc.)

© 2020, Amazon Web Services, Inc. or its Affiliates.

Provisioned Concurrency

Things to know:

• We provision more than you request.

• We still reap these environments.

• There is less CPU burst than On-Demand during init

© 2020, Amazon Web Services, Inc. or its Affiliates.

Provisioned Concurrency - configuration

© 2020, Amazon Web Services, Inc. or its Affiliates.

Provisioned Concurrency - configuration

© 2020, Amazon Web Services, Inc. or its Affiliates.

Provisioned Concurrency – Application Auto Scaling

© 2020, Amazon Web Services, Inc. or its Affiliates.

Memory and profiling

© 2020, Amazon Web Services, Inc. or its Affiliates.

Memory 👉 Power

Resources allocation

© 2020, Amazon Web Services, Inc. or its Affiliates.

CPU-bound example

“Compute 1,000 times all prime numbers <= 1M”

128 MB 11.722 sec $0.024628

256 MB 6.678 sec $0.028035

512 MB 3.194 sec $0.026830

1024 MB 1.465 sec $0.024638

© 2020, Amazon Web Services, Inc. or its Affiliates.

Meet AWS Lambda Power Tuning

Features:

• Data-driven cost and

performance optimization for

AWS Lambda

• Available as an AWS Serverless

Application Repository app

• Easy to integrate with CI/CD

https://github.com/alexcasalboni/aws-lambda-power-tuning

© 2020, Amazon Web Services, Inc. or its Affiliates.

Meet AWS Lambda Power Tuning

https://github.com/alexcasalboni/aws-lambda-power-tuning

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (input)

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (input)

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (input)

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (input)

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (input)

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (input)

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (output)

© 2020, Amazon Web Services, Inc. or its Affiliates.

AWS Lambda Power Tuning (visualization)

https://github.com/alexcasalboni/aws-lambda-power-tuning

Fastest execution timeFastest execution time

Lowest costs

© 2020, Amazon Web Services, Inc. or its Affiliates.

Real-world examples

© 2020, Amazon Web Services, Inc. or its Affiliates.

No-Op (trivial data manipulation <100ms)

© 2020, Amazon Web Services, Inc. or its Affiliates.

CPU-bound (numpy: inverting 1500x1500 matrix)

© 2020, Amazon Web Services, Inc. or its Affiliates.

CPU-bound (prime numbers)

© 2020, Amazon Web Services, Inc. or its Affiliates.

CPU-bound (prime numbers – more granularity)

© 2020, Amazon Web Services, Inc. or its Affiliates.

Network-bound (third-party API call)

© 2020, Amazon Web Services, Inc. or its Affiliates.

Network-bound (3x DDB queries)

© 2020, Amazon Web Services, Inc. or its Affiliates.

Network-bound (S3 download – 150MB)

© 2020, Amazon Web Services, Inc. or its Affiliates.

Network-bound (S3 download multithread – 150MB)

© 2020, Amazon Web Services, Inc. or its Affiliates.

Cost/performance patterns

https://github.com/alexcasalboni/aws-lambda-power-tuning

© 2020, Amazon Web Services, Inc. or its Affiliates.

How billing rounding impacts cost optimization

Example 1

Time = 480msBilling = 500ms

Time = 408ms Billing = 500ms

A 15% performance optimization...

... leads to a 0% costoptimization

Time = 310msBilling = 400ms

Time = 294ms Billing = 300ms

... leads to a 25% costoptimization

Example 2

A 5% performance optimization...

© 2020, Amazon Web Services, Inc. or its Affiliates.

Architecture and best practices

© 2020, Amazon Web Services, Inc. or its Affiliates.

Optimization best practices

Avoid «monolithic» functions

Reduce deployment package size

Micro/Nano services

Optimize dependencies (and imports)

E.g. up to 120ms faster with Node.js SDK

Minify/uglify production code

Browserify/Webpack

Lazy initialization of shared libs/objs

Helps if multiple functions per file

© 2020, Amazon Web Services, Inc. or its Affiliates.

Optimized dependency usage (Node.js SDK & X-Ray)

// const AWS = require('aws-sdk’)const DynamoDB = require('aws-sdk/clients/dynamodb’) // 125ms faster

// const AWSXRay = require('aws-xray-sdk’)const AWSXRay = require('aws-xray-sdk-core’) // 5ms faster

// const AWS = AWSXRay.captureAWS(require('aws-sdk’))const dynamodb = new DynamoDB.DocumentClient()AWSXRay.captureAWSClient(dynamodb.service) // 140ms faster

© 2020, Amazon Web Services, Inc. or its Affiliates.

Lazy initialization example (Python & boto3)

import boto3

S3_client = Noneddb_client = None

def get_objects(event, context): if not s3_client:

s3_client = boto3.client("s3")# business logic

def get_items(event, context): if not ddb_client:

ddb_client = boto3.client(”dynamodb")# business logic

© 2020, Amazon Web Services, Inc. or its Affiliates.

Amazon RDS Proxy

Fully managed, highly available database proxy for Amazon RDS. Pools and shares connections to make applications more scalable, more resilient to database failures, and more secure.

© 2020, Amazon Web Services, Inc. or its Affiliates.

Externalize orchestrationAvoid idle/sleep – delegate to Step Functions

Transform, not TransportMinimize data transfer (S3 Select, advanced filtering, EventBridge input transformer, etc.)

Discard uninteresting events asapTrigger configuration (S3 prefix, SNS filter, EventBridge content filtering, etc.)

Optimization best practices (performance/cost)

© 2020, Amazon Web Services, Inc. or its Affiliates.

Reusing connections with Keep-Alive

• For functions using http(s) requests

• Use in SDK with environment variable

• Or set keep-alive property in your

function code

• Can reduce typical DynamoDB

operation from 30ms to 10ms

• Available in most runtime SDKs

For more details, visit

https://bit.ly/reuse-connection

© 2020, Amazon Web Services, Inc. or its Affiliates.

Asynchronous:

• The caller receives ack quickly

• Minimizes cost of waiting

• Queueing separates fast and

slow processes

• Process change is easy

• Uses manages services

Comparing sync and async

Synchronous:

• The caller is waiting

• Waiting incurs cost

• Downstream slowdown affects

entire process

• Process change is complex

• Dependent on custom code

SQS queueService A Service BService A Service B

© 2020, Amazon Web Services, Inc. or its Affiliates.

Lambda Performance - Summary

Cold starts

• Causes of a cold start

• VPC improvements

• Provisioned Concurrency

Memory and profiling

• Memory is power for Lambda

• AWS Lambda Power Tuning

• Trade-offs of cost and speed

Architecture and optimization

• Best practices

• RDS Proxy

• Async and service integration

© 2020, Amazon Web Services, Inc. or its Affiliates.

Thanks!James Beswick, AWS Serverless

jbeswick@amazon.com