Manage Pipeline of Performance Testwith Jenkins
Vol.01 Mar/07/2017Takeshi TakizawaEcosystem Service Department, Rakuten, Inc.
Self-introduction
2
DEMO
3
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)
Test Client
Test Client
Test Client
Performance Test Pipeline
4
There are some issues😖😖😖
5
Performance Test Pipeline
6
• It’s difficult to estimate concrete requests 📈• Need to clarify that requests come permanently
or instantly 📊
Performance Test Pipeline
7
• Need programming knowledge 📚
Need programming knowledge
• It’s not 10 QPS 🙅
8
• Expected: Send requests in parallel
Need programming knowledge
9
Test Client APP
Need programming knowledge
• It’s not 10 QPS 🙅
10
• Actual: Send requests sequentially
Need programming knowledge
11
Test Client APP
Performance Test Pipeline
12
• Need infrastructure knowledge 📚• How to install packages• How to configure number of file descriptor
• Sometimes Test ENG doesn’t have enough permission to build server so need to ask INFRA ENG 🐢
Performance Test Pipeline
13
• Managing distributed test clients is complicated 👯
When I want to send heavy requests
• 1 Test Client doesn’t make enough load
14
Client APP
I can send 1000 QPSI can respond 2000 QPS
When I want to send heavy requests
• Scale Up Spec of Client Server?
15
Client APP
I can send 2500 QPS I can respond 2000 QPS
When I want to send heavy requests
• Scale Out is better because more flexible
16
Client
APPClient
Client
We can send 3000 QPS
I can respond 2000 QPS
But
17
Managing distributed test clients is complicated
• Concurrency 👯– Kick test clients at the same time– Only one Client is running on one Server
18
Performance Test Pipeline
19
• It’s boring to generate Test Report from distributed logs manually 🙏
Performance Test Pipeline
20
• Sometimes Test ENG doesn’t have enough permission to delete server so need to ask INFRA ENG 🐢
• People make mistakes 😲
Automate / Improve Test Pipeline
21
Not Coveredby this slide
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)HashiCorp (https://www.hashicorp.com/)Chef Software, Inc. (https://www.chef.io/)
Need programming knowledge
• Gatling code is writable & readable 🙆
22
Test ENG doesn’t have enough permission
23
• Provide Jenkins jobs to create/delete new Test Client Servers (Jenkins Slaves)
Manage Test Clients with Jenkins Pipeline
24
Pipeline Code Sample
25
26
27
28
29
30
31
Collect Test Logs
• Use stash / unstash DSL
32
Generate Report from Test Logs
• gatling command w/ --reports-only option
33
Generate Report from Test Logs
34
Publish Test Report
• Use other Jenkins plugins through pipeline– HTML Publisher Plugin– Compress Artifacts Plugin
35
36
Automate / Improve Test Pipeline
37
Not Coveredby this slide
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)HashiCorp (https://www.hashicorp.com/)Chef Software, Inc. (https://www.chef.io/)
Manage Pipeline of Performance Test with Jenkins
38
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)
We’re Hiring Senior QA Engineer!!
Looking for a talented and experienced Senior QA engineer
with over 3 years of experience in software QA who can take on challenging tasks in running
full-scale testing on company’s worldwide services
https://jobs.rakuten.careers/careersection/rakuten_ext_cs/jobdetail.ftl?job=00004405
39