© 2020, Amazon Web Services, Inc. or its Affiliates.
James Beswick
Senior Developer Advocate, AWS Serverless
@jbesw
Optimizing Lambda performance for
your serverless applications
© 2020, Amazon Web Services, Inc. or its Affiliates.
About me
• James Beswick
• Email: [email protected]
• Twitter: @jbesw
• Senior Developer Advocate – AWS Serverless
• Self-confessed serverless geek
• Software Developer
• Product Manager
• Previously:
• Multiple start-up tech guy
• Rackspace, USAA, Morgan Stanley, J P Morgan
© 2020, Amazon Web Services, Inc. or its Affiliates.
Agenda
Memory and profiling
© 2020, Amazon Web Services, Inc. or its Affiliates.
How does Lambda work?
© 2020, Amazon Web Services, Inc. or its Affiliates.
Anatomy of an AWS Lambda function
Your function
Language runtime
Execution environment
Lambda service
Compute substrate
© 2020, Amazon Web Services, Inc. or its Affiliates.
Where you can impact performance…
Your function
Language runtime
Execution environment
Lambda service
Compute substrate
© 2020, Amazon Web Services, Inc. or its Affiliates.
Anatomy of an AWS Lambda function
Handler () function
Function to be executed
upon invocation
Event object
Data sent during Lambda
function Invocation
Context object
Methods available to
interact with runtime
information (request ID,
log group, more)
// Python
import jsonimport mylib
def lambda_handler(event, context):# TODO implementreturn {
'statusCode': 200,'body': json.dumps('Hello World!')
}
// Node.js
const MyLib = require(‘my-package’)const myLib = new MyLib()
exports.handler = async (event, context) => { # TODO implementreturn {
statusCode: 200,body: JSON.stringify('Hello from Lambda!')
}}
© 2020, Amazon Web Services, Inc. or its Affiliates.
Function lifecycle – worker host
Execute
INIT code
Execute
handler code
Full
cold start
Partial
cold start
Warm
start
Download
your code
Start new
Execution
environment
AWS optimization Your optimization
© 2020, Amazon Web Services, Inc. or its Affiliates.
Measuring with AWS X-Ray
Profile and troubleshoot
serverless applications:
• Lambda instruments
incoming requests and can
capture calls made in code
• API Gateway inserts tracing
header into HTTP calls and
reports data back to X-Ray
const AWSXRay = require(‘aws-xray-sdk-core’)
const AWS = AWSXRay.captureAWS(require(‘aws-sdk’))
AWSXRay.captureFunc('annotations', subsegment => {
subsegment.addAnnotation('Name', name)
subsegment.addAnnotation('UserID', event.userid)
})
© 2020, Amazon Web Services, Inc. or its Affiliates.
X-Ray Trace Example
© 2020, Amazon Web Services, Inc. or its Affiliates.
Three areas of performance
Throughput CostLatency
© 2020, Amazon Web Services, Inc. or its Affiliates.
Cold starts
© 2020, Amazon Web Services, Inc. or its Affiliates.
Function lifecycle – a warm start
Request made to Lambda’s API
Service identifies if
warm execution environments is
available
Invoke handlerComplete invocation
Yes
© 2020, Amazon Web Services, Inc. or its Affiliates.
Function lifecycle – a full cold start
Request made to Lambda’s API
Service identifies if
warm execution environments is
available
Invoke handler
Find available compute resource
Download customer code
Start execution environment
Execute INITComplete invocation
No
© 2020, Amazon Web Services, Inc. or its Affiliates.
Cold starts - Execution environment
The facts:
• <1% of production workloads
• Varies from <100ms to >1s
Be aware…
• You cannot target warm
environments
• Pinging functions to keep them
warm is limited
© 2020, Amazon Web Services, Inc. or its Affiliates.
Cold starts - Execution environment
The facts:
• <1% of production workloads
• Varies from <100ms to >1s
Be aware…
• You cannot target warm
environments
• Pinging functions to keep them
warm is limited
Cold starts occur when…
• Environment is reaped
• Failure in underlying resources
• Rebalancing across Azs
• Updating code/config flushes
• Scaling up
© 2020, Amazon Web Services, Inc. or its Affiliates.
Cold starts - Execution environment
Influenced by:
• Memory allocation
• Size of function package
• How often a function is called
• Internal algorithms
AWS optimizes for this part of a
cold start.
© 2020, Amazon Web Services, Inc. or its Affiliates.
Lambda + VPC – Major performance improvement
Before: 14.8 sec duration
After: 933ms duration
Read more at
http://bit.ly/vpc-lambda
© 2020, Amazon Web Services, Inc. or its Affiliates.
Cold starts - Static initialization
The facts:
• Code run before handler
• Used to initialize objects,
establish connections, etc.
• Biggest impact on cold-starts
Also occurs when…
• A new execution environment
is run for the first time
• Scaling up
Dependencies,
configuration
information
© 2020, Amazon Web Services, Inc. or its Affiliates.
Cold starts - Static initialization
Influenced by:
• Size of function package
• Amount of code
• Amount of initialization work
The developer is responsible for
this part of a cold start.
What can help…
• Code optimization• Trim SDKs
• Reuse connections
• Don’t load if not used
• Lazily load variables
• Provisioned Concurrency
© 2020, Amazon Web Services, Inc. or its Affiliates.
Provisioned Concurrency on AWS Lambda
Pre-creates execution environments,
running INIT code.• Mostly for latency-sensitive, interactive
workloads
• Improved consistency across the long tail of
performance
• Minimal changes to code or Lambda usage
• Integrated with AWS Auto Scaling
• Adds a cost factor for per concurrency
provisioned but a lower duration cost for
execution
• This could save you money when heavily
utilized
© 2020, Amazon Web Services, Inc. or its Affiliates.
Function lifecycle – a Provisioned Concurrency start
Function configured with
Provisioned Concurrency
Find available compute resource
Download customer code
Start execution environment
Execute INIT
© 2020, Amazon Web Services, Inc. or its Affiliates.
Function lifecycle – a Provisioned Concurrency
invocation
Request made to Lambda’s API
Service identifies if
warm execution environment is
available
Invoke handlerComplete invocation
Yes
This becomes the default for all
provisioned concurrency
execution environments
© 2020, Amazon Web Services, Inc. or its Affiliates.
Provisioned Concurrency – things to know
• Reduces the start time to <100ms
• Can’t configure for $LATEST
• Use versions/aliases
• Provisioning rampup of 500 per minute
• No changes to function handler code
performance
• Requests above provisioned concurrency follow
on-demand Lambda limits and behaviors for cold-
starts, bursting, pricing
• Still overall account concurrency per limit region
• Wide support (CloudFormation, Terraform,
Serverless Framework, Datadog, Epsagon,
Lumigo, Thundra, etc.)
© 2020, Amazon Web Services, Inc. or its Affiliates.
Provisioned Concurrency
Things to know:
• We provision more than you request.
• We still reap these environments.
• There is less CPU burst than On-Demand during init
© 2020, Amazon Web Services, Inc. or its Affiliates.
Provisioned Concurrency - configuration
© 2020, Amazon Web Services, Inc. or its Affiliates.
Provisioned Concurrency - configuration
© 2020, Amazon Web Services, Inc. or its Affiliates.
Provisioned Concurrency – Application Auto Scaling
© 2020, Amazon Web Services, Inc. or its Affiliates.
Memory and profiling
© 2020, Amazon Web Services, Inc. or its Affiliates.
Memory 👉 Power
Resources allocation
© 2020, Amazon Web Services, Inc. or its Affiliates.
CPU-bound example
“Compute 1,000 times all prime numbers <= 1M”
128 MB 11.722 sec $0.024628
256 MB 6.678 sec $0.028035
512 MB 3.194 sec $0.026830
1024 MB 1.465 sec $0.024638
© 2020, Amazon Web Services, Inc. or its Affiliates.
Meet AWS Lambda Power Tuning
Features:
• Data-driven cost and
performance optimization for
AWS Lambda
• Available as an AWS Serverless
Application Repository app
• Easy to integrate with CI/CD
https://github.com/alexcasalboni/aws-lambda-power-tuning
© 2020, Amazon Web Services, Inc. or its Affiliates.
Meet AWS Lambda Power Tuning
https://github.com/alexcasalboni/aws-lambda-power-tuning
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (input)
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (input)
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (input)
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (input)
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (input)
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (input)
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (output)
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Lambda Power Tuning (visualization)
https://github.com/alexcasalboni/aws-lambda-power-tuning
Fastest execution timeFastest execution time
Lowest costs
© 2020, Amazon Web Services, Inc. or its Affiliates.
Real-world examples
© 2020, Amazon Web Services, Inc. or its Affiliates.
No-Op (trivial data manipulation <100ms)
© 2020, Amazon Web Services, Inc. or its Affiliates.
CPU-bound (numpy: inverting 1500x1500 matrix)
© 2020, Amazon Web Services, Inc. or its Affiliates.
CPU-bound (prime numbers)
© 2020, Amazon Web Services, Inc. or its Affiliates.
CPU-bound (prime numbers – more granularity)
© 2020, Amazon Web Services, Inc. or its Affiliates.
Network-bound (third-party API call)
© 2020, Amazon Web Services, Inc. or its Affiliates.
Network-bound (3x DDB queries)
© 2020, Amazon Web Services, Inc. or its Affiliates.
Network-bound (S3 download – 150MB)
© 2020, Amazon Web Services, Inc. or its Affiliates.
Network-bound (S3 download multithread – 150MB)
© 2020, Amazon Web Services, Inc. or its Affiliates.
Cost/performance patterns
https://github.com/alexcasalboni/aws-lambda-power-tuning
© 2020, Amazon Web Services, Inc. or its Affiliates.
How billing rounding impacts cost optimization
Example 1
Time = 480msBilling = 500ms
Time = 408ms Billing = 500ms
A 15% performance optimization...
... leads to a 0% costoptimization
Time = 310msBilling = 400ms
Time = 294ms Billing = 300ms
... leads to a 25% costoptimization
Example 2
A 5% performance optimization...
© 2020, Amazon Web Services, Inc. or its Affiliates.
Architecture and best practices
© 2020, Amazon Web Services, Inc. or its Affiliates.
Optimization best practices
Avoid «monolithic» functions
Reduce deployment package size
Micro/Nano services
Optimize dependencies (and imports)
E.g. up to 120ms faster with Node.js SDK
Minify/uglify production code
Browserify/Webpack
Lazy initialization of shared libs/objs
Helps if multiple functions per file
© 2020, Amazon Web Services, Inc. or its Affiliates.
Optimized dependency usage (Node.js SDK & X-Ray)
// const AWS = require('aws-sdk’)const DynamoDB = require('aws-sdk/clients/dynamodb’) // 125ms faster
// const AWSXRay = require('aws-xray-sdk’)const AWSXRay = require('aws-xray-sdk-core’) // 5ms faster
// const AWS = AWSXRay.captureAWS(require('aws-sdk’))const dynamodb = new DynamoDB.DocumentClient()AWSXRay.captureAWSClient(dynamodb.service) // 140ms faster
© 2020, Amazon Web Services, Inc. or its Affiliates.
Lazy initialization example (Python & boto3)
import boto3
S3_client = Noneddb_client = None
def get_objects(event, context): if not s3_client:
s3_client = boto3.client("s3")# business logic
def get_items(event, context): if not ddb_client:
ddb_client = boto3.client(”dynamodb")# business logic
© 2020, Amazon Web Services, Inc. or its Affiliates.
Amazon RDS Proxy
Fully managed, highly available database proxy for Amazon RDS. Pools and shares connections to make applications more scalable, more resilient to database failures, and more secure.
© 2020, Amazon Web Services, Inc. or its Affiliates.
Externalize orchestrationAvoid idle/sleep – delegate to Step Functions
Transform, not TransportMinimize data transfer (S3 Select, advanced filtering, EventBridge input transformer, etc.)
Discard uninteresting events asapTrigger configuration (S3 prefix, SNS filter, EventBridge content filtering, etc.)
Optimization best practices (performance/cost)
© 2020, Amazon Web Services, Inc. or its Affiliates.
Reusing connections with Keep-Alive
• For functions using http(s) requests
• Use in SDK with environment variable
• Or set keep-alive property in your
function code
• Can reduce typical DynamoDB
operation from 30ms to 10ms
• Available in most runtime SDKs
For more details, visit
https://bit.ly/reuse-connection
© 2020, Amazon Web Services, Inc. or its Affiliates.
Asynchronous:
• The caller receives ack quickly
• Minimizes cost of waiting
• Queueing separates fast and
slow processes
• Process change is easy
• Uses manages services
Comparing sync and async
Synchronous:
• The caller is waiting
• Waiting incurs cost
• Downstream slowdown affects
entire process
• Process change is complex
• Dependent on custom code
SQS queueService A Service BService A Service B
© 2020, Amazon Web Services, Inc. or its Affiliates.
Lambda Performance - Summary
Cold starts
• Causes of a cold start
• VPC improvements
• Provisioned Concurrency
Memory and profiling
• Memory is power for Lambda
• AWS Lambda Power Tuning
• Trade-offs of cost and speed
Architecture and optimization
• Best practices
• RDS Proxy
• Async and service integration
© 2020, Amazon Web Services, Inc. or its Affiliates.
Thanks!James Beswick, AWS Serverless