Full Stack Load Testing Load Testing On Alfresco 5.1

Derek Hulley Alfresco Founding Engineer

Introduction

•  Resource Limits

•  Load Test Accuracy

•  Load Test Software

Introduction

Resource Limitations

Limiting Factors •  Some Basic Server Limitations

–  Memory consumption –  CPU usage –  IOPS / Bandwidth / IO

•  Other Server Limitations –  Monitoring (getting at logs, charts, monitors, etc) –  Backups –  CRON jobs

•  Features –  Supported file types –  Auditing –  Jodconverter –  Clustering

Resources

Moving Parts

Resources

Caches  

Start with a guess and try it out in anger Alfresco  and  

Share  20%  

Transforma4ons  

10%  

SOLR  30%  

Database  30%  

Content  5%  

Load  Balancer  5%  

Resource  Guess  0  

Alfresco  and  Share   Transforma2ons   SOLR  

Database   Content   Load  Balancer  

Resources

Monitor all Run for days Resolve Issues Repeat Adjust resources

Focuse.g. SOLR

Resources

Alfresco  and  Share  20%  

Transforma4ons  

10%  

SOLR  30%  

Database  30%  

Content  5%  

Load  Balancer  5%  

Resource  Guess  0  

Alfresco  and  Share   Transforma2ons   SOLR  

Database   Content   Load  Balancer  

Resource Focus Example: SOLR •  CPU: underused

•  Memory: underused

•  IO: underused

•  Network: Keep and eye on this!

Resources

Reduce spend and iterate …

Resources

Alfresco  and  Share  23%  

Transforma4ons  

12%  SOLR  18%  

Database  35%  

Content  6%  

Load  Balancer  6%  

Resource  Guess  2  

Alfresco  and  Share   Transforma2ons   SOLR  

Database   Content   Load  Balancer  

Load Test Accuracy

Load Test Risk Factors: Scale

•  Increases Cost –  Your Time –  Getting the data in does not normally produce results

•  Bulk Loading –  Different hardware for this phase? –  Simulate bulk loading as accurately as necessary

•  Number of Documents –  Easier to account for with experience –  Data storage costs –  Backup/Restore procedures

•  Users and Groups –  LDAP sync –  Complex Group Hierarchy –  Site Group Permissions –  Mass-authentication

Load Test Accuracy

Load Test Risk Factors: Documents •  Document Types

–  Image thumbnail generation –  Text to PDF for preview –  Document library listings (many transformations a.s.a.p)

•  Document Size –  Network and IO costs and limits –  CPU used for streaming –  PDF to text memory consumption –  External converters

•  More CPU or more infrastructure

Load Test Accuracy

Load Test Risk Factors: API Usage Profile •  Load Profile

–  Burst load –  Trickle load

•  Data Structure –  Paging –  Sorting –  Permissions

•  The Internet –  CMIS across the Atlantic –  Southern hemisphere client apps

Load Test Accuracy

Load Test Risk Factors: UI Complexity •  Share APIs

–  /private •  Upload File

–  Stream binary –  Then all of these at once:

•  Register user activity •  Generate thumbnail •  Generate PDF preview and display •  Get and display document metadata •  Get and display document version history

•  Replay Doom (You Only Live Once)

Load Test Accuracy

Load Test Risk Factors: UI Complexity Load Test Accuracy

Load Test Risk Factors: UI Complexity •  Page-Object Pattern: Browsers do what they do

Login Event Processor Login Page Dashboard Page Browser

processEvent loginAs

Send keystrokes

Login button click DashboardPage

Tim

ed

render Wait For: Activities

Wait For: My Sites

Wait For: My Sites

Wait For: My Sites

Wait For: My Docs

Wait For: etc

Increasing Complexity

Server

x100

Load Test Accuracy

Load Test Software Alfresco Benchmark Server and Tests

References •  Martin Bergljung’s walkthrough •  Alfresco Wiki: Benchmark_Testing_with_Alfresco •  GitHub: AlfrescoBenchmark

Test Software

Alfresco Benchmark Suite

Test Software

Benchmark  Server  

Tomcat  7  

Rest  API  

MongoDB  Config  Data  

Services  

MongoDB  Test  Data  

UI  

Benchmark  Driver  (xN)  Benchmark  Driver  (xN)  Benchmark  Driver  

Tomcat  7   Extras  (Selenium)  

Servers  /  APIs   Servers  /  APIs  

Load  Balancer  

Servers  /  APIs  

Test  

Services  

Rest  API  

Load Tests: /tests/ent-signup •  Create users •  User base reused

–  Prerequisite for other tests

Test Software

Load Tests: /tests/dataload •  Reuse user base •  Create

–  Sites –  Site Members

•  Site data –  Folders –  Files uploaded using CMIS or

Test Software

Load Tests: /tests/dataload •  AlfrescoBenchmark/alfresco-text-gen

•  Text data generated in server •  Option to push data

into ContentStore

Test Software

Load Tests: GitHub benchmark-cmis •  Burst load •  Easy to extend

Test Software

Load Tests: FTP sample files vs spoofing •  FTP server

–  A few thousand samples –  Use for Share tests

•  Transformations •  Text extractions •  Renditions

•  Spoofing –  Stress SOLR –  document library size

•  Search: searchterms.txt

Test Software

Share Load Testing: /tests/share •  Concurrent Users:

–  A new user logs in every N seconds –  The user logs out after spending Y seconds –  Number of concurrent users C = Y/N

•  Selenium Driver Overload –  Use Windows –  5s to start a browser on EC2 m3.2xlarge –  Up to 40 browsers per load driver on EC2 m3.2xlarge –  Drivers required D = max( 5s/N, C/40) –  m3.2xlarge Windows à spot price of $0.3/hour –  500 concurrent user à20 drivers à $6/hour

Test Software

Summary

Summary •  Build up load testing software and skills •  Design tests around existing tools

–  Understand risks and where to extend or introduce new tests •  Iterate

Summ

ary

Questions

Thank you!

Condividi su

#AlfrescoDayRoma