Post on 12-Apr-2017
transcript
Manage Pipeline of Performance Testwith Jenkins
Vol.01 Mar/07/2017Takeshi TakizawaEcosystem Service Department, Rakuten, Inc.
Self-introduction
2
DEMO
3
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)
Test Client
Test Client
Test Client
Performance Test Pipeline
4
There are some issuesššš
5
Performance Test Pipeline
6
ā¢ Itās difficult to estimate concrete requests šā¢ Need to clarify that requests come permanently
or instantly š
Performance Test Pipeline
7
ā¢ Need programming knowledge š
Need programming knowledge
ā¢ Itās not 10 QPS š
8
ā¢ Expected: Send requests in parallel
Need programming knowledge
9
Test Client APP
Need programming knowledge
ā¢ Itās not 10 QPS š
10
ā¢ Actual: Send requests sequentially
Need programming knowledge
11
Test Client APP
Performance Test Pipeline
12
ā¢ Need infrastructure knowledge šā¢ How to install packagesā¢ How to configure number of file descriptor
ā¢ Sometimes Test ENG doesnāt have enough permission to build server so need to ask INFRA ENG š¢
Performance Test Pipeline
13
ā¢ Managing distributed test clients is complicated šÆ
When I want to send heavy requests
ā¢ 1 Test Client doesnāt make enough load
14
Client APP
I can send 1000 QPSI can respond 2000 QPS
When I want to send heavy requests
ā¢ Scale Up Spec of Client Server?
15
Client APP
I can send 2500 QPS I can respond 2000 QPS
When I want to send heavy requests
ā¢ Scale Out is better because more flexible
16
Client
APPClient
Client
We can send 3000 QPS
I can respond 2000 QPS
But
17
Managing distributed test clients is complicated
ā¢ Concurrency šÆā Kick test clients at the same timeā Only one Client is running on one Server
18
Performance Test Pipeline
19
ā¢ Itās boring to generate Test Report from distributed logs manually š
Performance Test Pipeline
20
ā¢ Sometimes Test ENG doesnāt have enough permission to delete server so need to ask INFRA ENG š¢
ā¢ People make mistakes š²
Automate / Improve Test Pipeline
21
Not Coveredby this slide
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)HashiCorp (https://www.hashicorp.com/)Chef Software, Inc. (https://www.chef.io/)
Need programming knowledge
ā¢ Gatling code is writable & readable š
22
Test ENG doesnāt have enough permission
23
ā¢ Provide Jenkins jobs to create/delete new Test Client Servers (Jenkins Slaves)
Manage Test Clients with Jenkins Pipeline
24
Pipeline Code Sample
25
26
27
28
29
30
31
Collect Test Logs
ā¢ Use stash / unstash DSL
32
Generate Report from Test Logs
ā¢ gatling command w/ --reports-only option
33
Generate Report from Test Logs
34
Publish Test Report
ā¢ Use other Jenkins plugins through pipelineā HTML Publisher Pluginā Compress Artifacts Plugin
35
36
Automate / Improve Test Pipeline
37
Not Coveredby this slide
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)HashiCorp (https://www.hashicorp.com/)Chef Software, Inc. (https://www.chef.io/)
Manage Pipeline of Performance Test with Jenkins
38
Jenkins project (https://jenkins.io/)GatlingCorp (http://gatling.io)
Weāre Hiring Senior QA Engineer!!
Looking for a talented and experienced Senior QA engineer
with over 3 years of experience in software QA who can take on challenging tasks in running
full-scale testing on companyās worldwide services
https://jobs.rakuten.careers/careersection/rakuten_ext_cs/jobdetail.ftl?job=00004405
39