When Dev met Test(The future of software testing)
2016.01 by JungGunhome: genycho.blog.me
role : specialistactor : domain expert
role : generalistactor : test specialist
Developer
Tester
When dev met test
2015 remade in Software testing
This is about…Message 1: Co-Work (3Amigos)
Message 2: Shift-left-testing (early testing activities)
Several cases that test people co-worked with
development team closely
Not only about testing, but also
about dev quality
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
V. Conclusion
Target projectsSmall, Research&Dev project (Trendy technology related)
Video, Data analystics, Network&Security, IoT based projects
Current quality level was low, but the impact was huge
Small(~12people)Internalfunding
Flexiblerequirements
Complex,Trendy tech
Young,Highly educated Human resource
Testability가떨어지는
Low quality concept
Test Operation organizationFlexible test resource managing (Shared Service) for small, multiple projects
Each person has one or two specialized domain area
Big Data VA, IoT Security, Network
Domainstudy
TestStrategy
TestEducation
TestDesign
TestScripting
Testware dev,Automation
TestExecution
QualityReview
0.25 0.25 0.25 0.25 0.5 0.25 0.5 0.25
Project A Project B Project 1 Project 2 a project
TM
TE
TM
TE
TM
TE※ Shared Service
※ Practices & effort
Projects information (2014yr~2015yr)
Mobile authentication Security FIDO - external requests
Admin portal(re-usable asset)
Web application Admin portal 6 external requests
projects technology Target domain Dev size(people)
challenges, difficulty, etc
MES bigdata analyticsData
AnalyticsManufacture Factory
5Verifying statistics functions result and the integration with big data platform
Crypt/DeCrypt algorithmSecurity
(en/decryption)Encryption/decryption
3Complex security algorithm,
Integration with Legacy system
Logistics demand prediction engine
Data
AnalyticsLogistics 4
complex prediction algorithm based of rules
Integration with legacy system
IoT based smart factory(Smart OHT)
IoTSemiconductor manufacturing
6Understanding IoT(hardware+software) technology and its Testability
political issues with customers
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, programming
5. Test-Automation
III. Strategic activities
IV. Other activities
V. Conclusion
Customized quality educationTo prevent defects, project-customized education in the early phase
( An example about outline )
- what test activities will be done and why
- Frequently occurred defects
based on similar doman, technology
- How to write (User) test scenario &
the basic of test design
- Intro about sample unit test code
with very early dev code
- Test Automation introduction with Jenkins
> Education: sharing the test plan- what test activities will be done and why- the whole test cycle, roles&responsibility
- be able to get feedback by sharing with dev
> Education: Test Design guide- the difference with Unit & Integration testing
- the template of test scenario and guide
- basic test design technique
- practical guide with other input documents
> Education: Sharing old defects, cases
- defects and test cases based on the product’s domain, technology
[ defects(Web) ] [ Quality guide for API ]
API DevelopmentStandards on quality
- Logging- Exception handling &error code
- API Spec guild- Unit test checklists
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, programming
5. Test-Automation
III. Strategic activities
IV. Other activities
V. Conclusion
What Test Design
- The thinking process to test effectively and efficiently.
- because the projects were mainly a kind of pure algorithm libray, they were so complex and needed to design test cases well.
Black-box approach
- initial test approach was black-box which design tests with specification
- various documents – requirements, algorithm description, Architecture doc etc
- because they were so vague or insufficient to understand the algorithm, we focused on visualizing the logic to share the understanding with customers
White-box approach
- the real code was the best documents
- made the domain-specific dev-code checklist
- do offline reviews with dev code or unit-testing code (regular pair-programming)
※ Grey-box testing?
- basically do blackbox-testing while checking dev code
※ Wait!, Who is the best person do test design?
- Test Desing should be done by Testers!!!- They can do anyhow, but do they know what custmer really wants?
BusinessAnalyst
Customer
Tester
“ANALYSTS!!!”(pros): knows well the businees and dev both. Need to be reviewed what they analyzed(cons) : not know about testing technique
“TESTERS!!!”(pros) : test experts, experienced in many domain, focused on testing(cons) : need to understand requirements fully
“CUSTOMERS!!!”(pros): best know about business(cons): no time, no tes or IT knowledge
Who shuld do test design?
The answer depends on ‘Test Level’ or ‘type’, but basically everyone should work together.
Step 1) Understanding with documents, interviews, everything as possible
Step 2) Sharing what understood and getting feedback
(Example) Analyzing old test scenarios
find questions to confirm And test conditons
Make checklist, diagrams to share with dev team
Step 1) Understanding with things – documents, interviews
Step 2) Sharing what understood and getting feedback
(Customer)Easy to understand with diagrams
(Developers)With the checklists, I was able to self-define the detail parts
(Product Manager)Visual diagram can help potential stakeholders to understand our product
I can give the team more detailed requirements
>Test Design: Test Coverage reviews
- testers review test coverage (line&branch coverage) on real code
- if needed, they add more test cases or ask developers to do
TC add)
need to test server and client different
version check test
Coverage check)
The code is not reachable(double checking)
Need to delete
※ (Example) Graphical approach to review
If the documents are insufficient, dev code is the best documents.
First, draw any diagrams if it can be helpful, then review the logic, workflow with various stakeholders
소스 코드
구조-Process 처리여부구조-Log 처리구조-Exception 처리입력 파라미터 체크input table 없는 경우 처리output table 없는 경우 처리해당 컬럼명이 없는 경우 처리로직 상의 if 문 조건 나열계산식에서 입력값 짝에 따라 분자 or 분모가 0이 되는경우코드 내에 형 변환(문자 -> 숫자, 숫자-> 문자, Double->int 등)
로직 수행 중 오류 발생 시 에러 테이블에 넣어주고 있는지
Ex 1) the sequence of dev code 2) Code review checklist(a) main structure(b) logging(c) exception handling…(f) if conditions…(j) calculation(k) type casting
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, programming
5. Test-Automation
III. Strategic activities
IV. Other activities
V. Conclusion
Customized Test Code guide
- To minimize the learning curve to write unit-test code
. language - Java, C/C++, JavaScript, Scala(Spark)
. Specific framwork based test code(Spring fr, MapReduce, Spark, Kura…)
Target Guide Type
Complex pure algorithm
Various Test conditons and cases
junit
C language program
Tool research, Gtest pilot GoogleTest
Big DataData-driven testing using Junit, MRUnit
MRUnit(MapReduce),
ScalaTest(Spark)
Complex prediction algorithm
Unit/Intg test code with Junit, CascadingUnit,
Various test casesCascading
IoT basedsoftware unit-testing mocking Hardware parts
Eclipse Kura
Framework
Web applicationvarious test cases on various layers
Spring
Framework
Target toTest
(TC1) basic workflow
[ Basic Test Approach ]
(TC2) code branched workflow
(TC3) specific business workflow
(TC4) exception workflow
…
[ Projects ]
※ (Example) Structured Test code
- To easy-maintain, data-driven testing
- Naming, Comments guide
Test Code Structure
FaroServiceTestCase- 테스트 수행 전 스프링프레임워크의 설정 파일 로딩- 매 테스트 후 테스트 이전 상태로DB 초기화- 특정 쿼리문을 직접 수행 가능한인터페이스 제공 등
SampleServiceTest@Autowired로 객체 주입 후- 기본적인 테스트- 다른 흐름 테스트- 잘못된 키 값 등의 테스트- 필수입력 누락 등의 테스트…
SampleTestData- 테스트에 사용되는 데이터를분리하여 다른 테스트에서도참조할 수 있도록 정의(추가로 엑셀 또는 DB에서가져올 수 있게 보완 가능)
CodeMngService- 코드 전체 조회- 코드 검색- 코드 등록, 수정, 삭제…
AbsractTestData- 테스트 데이터를 DB, 엑셀 등으로부터 관리할 수 있도록 지원하는 추상 클래스
Spring configrationloading, common variables,
utils are in a super Parent class
If the configuration(file) is changed, only one class needs
to be changed
If the data is changed,Only one is to change
Data that can beReferred in other testsare stored in a separate
class
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, programming
5. Test-Automation
III. Strategic activities
IV. Other activities
V. Conclusion
Pair-Testing- not fix bugs, but fix human mistakes
- developer+tester pair in a sprint testing
- developers learn what, how testers think and test
testers learn what the code intended and expected
- (Expected)
better communication
reduced bugs (defect prevention) by learning
testers can find more serious bugs
the product’s quality is better
(Not-Expected)
Developers and testers do not trust each other
complaint that pair-testing is useless and time-consuming job
disappointment about testers, testing
(Example) Pair-testing
- 40minutes pair-testing with 6 develpers
- more than 40 defects and issues (The team needed to define the standard)
Session
dev date targetNotes
(Reminds, Test paths, defects, etc)
1 A dev-2d
mornigRole-management
[역할관리 - 조회](1) 화면 리사이즈시 데이터 영역이 잘리는 현상 있음(하단 스크롤로 끝까지 옮겨도 표시되지 않음)(2) (공통-검토필요) 현재 화면 위치를 왼쪽 메뉴에서 초록색으로 표시해 주는데 F5 새로 고침 시 초록색이 표시되지 않음(3) (권고) 조회조건에 역할의 사용여부로 조회할 수 있는 기능이 있었으면 편할 것으로 보임(4) 검색조건에 특수문자 입력이 가능하거나 타 화면과 달리 trim 처리가 적용/ 또는 미적용된 경우가 섞여 있음(5) (검토예정이라고 함) 역할명, 설명등을 주어진 길이만큼 입력하면 목록에서 ... 으로 표시됨.
툴팁이 표시되면 좋을 것으로 보임 * 수정팝업을 띄우면 표시되니 결함보다는 권고사항성입니다~[역할관리 - 추가](6) 입력 시 id, 명에 trim 처리가 필요할 것으로 보임.(조회시는 권고이나 등록/수정 시에는 trim이 필요)(7) (다른 화면도 체크 필요) 신규 추가시에 사용여부를 N으로 하고 저장해도 Y로 저장 됨 <- 쿼리에 'Y'로 박혀 있음[ 사용자 팝업1,2 ](8) 직급 컬럼 등의 정렬이 타 화면과 일치하지 않음 (가운데 정렬 필요)(9) 조회조건 필드가 특수문자는 막혀 있는데 trim 처리는 하는 등 타 검색 필드와 일관성 필요(10) “그거”라고 했던 건 - 체크박스를 선택하고 페이지를 이동하면 다른 페이지에서 선택하지 않은 건들이 선택되어 있는 등 이상 동작을 하는 경우가 발생 함
2 B dev-2d
morning
Group-management-popups
[업무그룹 메뉴팝업](1) 팝업 화면 타이틀명에 공통적으로 언더바가 포함되어 있음(2) 여러 팝업에서 같은 항목의 정렬이 서로 다름
- 길이가 일정한 코드성 데이터는 가운데 정렬, 이메일, 설명 등과 같이 값이 가볍적으로 바뀌는 값들은 좌측 정렬이 맞을 것으로 보임(3) (확인필요) 화면 및 컬럼 폭 리사이즈가 안 되는데 길이가 길어질 수 있는 항목의 경우 툴팁을 검토할 필요가 있음(4) 메뉴에 대해 사용여부를 N으로 수정해도 타 화면 팝업에서 메뉴가 조회됨(5) (권고) 메뉴 추가에서 기존 추가한 메뉴와 새로 추가할 수 있는 메뉴가 구분이 안 됨. 방안 몇가지 얘기했던 것 검토 요청[사용자 팝업](6) 사용자를 선택하지 않고 선택 버튼을 클릭하면 메시지는 "한 건 이상을 선택해 주세요" 라고 정상 표시되나 바로 팝업 창을 닫아버림. 선택할 수 있도록 팝업 창 유지 필요(7) 각 항목들의 좌우 정렬이 타 화면과 다름(8) 팝업 화면 전체에 공통 발생 - 조회 데이터의 필드들이 모두 불필요하게 editable 함
3 C dev-2d
afternoon
Group-management
[업무그룹관리](1) 업무그룹 ID에 한글이 들어갈 수 있으나 조건조회 해당 필드에 한글이 입력되지 않음(2) (확인필요-일관성) 신규 추가후 리프레시 될 때 추가된 건이 전체 페이지의 제일 하단에 표시되는데 타화면과 기준 논의 필요 <- 사용자 추가의 경우 해당 정렬 위치에
추가되었던 것으로 보임(3) (권고) 조회조건에 업무그룹의 사용여부별로 조회할 수 있었으면 좋겠음(4) 업무그룹ID 등에 trim처리를 해야 할 것으로 보임(5) "그거" 라고 했던, a) 목록 조회에서 전체 조회하여 여러 페이지 조회, b) 조회조건 입력 후 조회버튼이 아닌 끝 으로 이동 선택 => 아무 건도 조회되지 않음, c) 조회조건
삭제하여 전체 조회 => 전체조회는 정상적으로 되나 하단 페이지 숫자가 표시되지 않음.
4 D dev-1d
Morning
Login,User-registration
[회원가입](1) 로그인 ID 값에 대해 trim 처리를 하지 않음(2) ID 입력, 중복체크 한 상태에서 사용자 이름을 입력하지 않고 저장하면 ID 중복 체크를 하라는 메시지가 표시 됨(3) 비밀번호, 비밀번호 체크에 대해 필수입력 표시 누락(4) (검토) 이메일, 전화번호에 대한 영문자 입력 등에 대해 체크 로직 적용에 대해 검토 권고
* ID, 비밀번호 생성 규칙은 반영 예정이라고 함[로그인] - N/A[사용자관리](5) UI 브라우저 리사이즈 시 사용자 관리만 하단 스크롤이 2개 생기는 현상 있음(6) 전체 화면이 아닌 경우 하단 footer가 페이지 네비게이션 영역을 덮는 현상 발생(7) (권고,공통) 왼쪽 메뉴 접기 아이콘이 있는데 접은 상태에서는 화살 방향이 반대여야 펼치는 기능으로 인지하기 쉬울 것으로 보임(8) 조회 조건의 사용여부 콤보박스 값이 직접 선택 및 수정이 가능함(9) 브라우저 리사이즈 시 조회 영역은 정상 확장되나, 하단 데이터 영역이 확장되지 않는 현상 발생(10) 사용자 정보 수정 팝업에서 필수값을 입력하지 않았을 때의 메시지가 ID중복체크를 하지 않았다는 잘못된 메시지 발생(11) (동시성) A 관리자가 사용자 정보를 수정(팝업)하고 있을 때 B 관리자가 해당 사용자를 삭제. 다시 A 관리자가 수정 저장하면 메시지 상으로는 정상적으로 수정되었다고
표시가 됨(해당 사용자는 삭제) - 서버 상에서 수정하려는 id가 없어서 정상 반환하는 현상 때문으로 보임.
5 E dev-1d
afternoon
Menu-management
[메뉴관리](1) 메뉴에 대해 사용여부를 N으로 수정해도 타 화면 팝업에서 메뉴가 조회됨(2) 메뉴 하위의 메뉴페이지 리스트에 대해 N으로 수정해도 Y로 다시 변경되어 조회 됨(3) 메뉴 하위의 메뉴페이지 신규 추가시 사용여부를 N으로 하고 추가하면 추가는 되었다고 메시지 표시되나
실제로는 저장 안 됨(4) 메뉴 페이지를 여러개 등록한 후 삭제하면, 삭제가 1,2건만 되고 이후 삭제가 되지 않음(5) 하나의 메뉴에 대해 동시에 다른 페이지에서 접근하는 테스트에 대해
A가 보고 있는 상태에서 B가 삭제하는 경우 A에서 다른 동작을 하면 "알 수 없는 에러가 발생하였습니다" 라는 상황을 파악하기 어려운 메시지가 표시됨
(6) UI 브라우저 리사이즈 시 조회 영역은 정상 확장되나, 하단 데이터 영역이 확장되지 않는 현상 발생
Early testing, easy to understand and fix
Not just finding defects, but also chances to define the dev standards
in real time
※ Test Code Pair-programming
- developers can learn how to write test code in the specific cases.
testers can know better about dev code
- Testers can talk about quality aspects in dev code level
Well, at first, let’s write the most basic case whichWe can debug what we write.
after that, everytime you need to check your code orDebug, add more tests cover those cases
In someway, we can check the test coverage, it will help us if we should add more tests or not
Should I write test code for this part? That is too much and difficult to synch…
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, programming
5. Test-Automation
III. Strategic activities
IV. Other activities
V. Conclusion
Test Automation plan
- project customized Test automation plan should be established
RDMBS
DAO implementations
DAO interfaces
Service implementations
Service interfaces
Other remote interfaces
Data AccessLayer
ServiceLayer
PresentationLayer
Web UI Test
Controller
REST API
UI
55 √
Controller Test
Service Test
DAO Test
RESTful API Test
√
11
22
33
44 √
TestLevel
Target Tools
Unit
Test
(Sprint)
DAO Junit
Service Junit
REST API SoapUI
Web UI Pair-testing
Sprint
Integration
Test
Internal
Workflow
Selenium
(+Sikuli)
Integration
External/Legacy
Selenium
/ Manual
Integration
Test
Internal/External
WorkflowManual
User
Test
Internal/External
workflowManual
[Example: Test automation strategy ]
Test Automation Infra in early phase
Customer,Other stakeholders
ProductManager
Dev team
Shared & visualizedTest results Okay.
We are coding&testing
My tests are ok.
Junit test reports Test coverages(Method,Line,Bran
ce, etc)
Code Inspection(PMD,…)
Jekins(+SonarQube?)
[Example: Test automation ]
I. Introduction
II. Engineering activities
III. Strategic activities
1. Test Strategy/Plan
2. Test analysis/report
3. Other Professional services
IV. Other activities
V. Conclusion
Test Strategy
(Example 1) Client-Server application
- define ‘Test Levels’ like (1)Unit testing, (2)Integration1(Client, Server each),
(3)Integration2(Client-Server) and each level’s purpose, target, role, method
User Env
UnitTesting
IntgTesting 1
IntgTesting 2
AcceptanceTesting
각 메소드(또는 단위 기능)에 대해다양한 예외 상황을 만들어 면밀히 테스트
서버, 클라이언트 각 영역 안에서여러 단위기능들의 묶음에 대해 기능 흐름을몇 가지로 정의하고 테스트
서버 - 클라이언트 간의 상호 연동 관점에서기능 흐름을 몇 가지로 정의하고 테스트
XXXeaXXXea
XeaXea
XXeaXXea
????
사용자 관점(사업부)에서 제품을 어떻게사용할지 흐름을 정의하고 테스트
Client Server
Mod A
Mod B
Mod C
Mod D
Mod E
Mod F
Unnit Test
Acceptance Testing
Intg 1
Intg 1Intg 2
(Example 2) Cascading Framework considered Unit/Intg Testing
- define Unit / Integration testing cosidering target’s architecture and functional roles
Unit Test: targets one Pipe mainly
Integration Test: targets workflows from Source to Sink(or failed end)
(Example 3) Hardware-Software integratied product
- define various test approach in various dev phase
: Software level Unit/intg testing, Hardware-Software integrated testing, real
environment testing etc
Level Software testing On board testing Simulated Env Real Env
Description
Software Unit/Intg testing for
Java programs
Application interface testing.
Hardware, software integration
checking
Simulated, down-scaled
environment testing
Real Envronment
Methods
Junit With test scenarios With business scenarios With business scenarios
In charge
Dev team(testers) Dev team(testers) Dev team, Customers Customers/Dev team
어플리케이션정보파일
센서,하드웨어
I. Introduction
II. Engineering activities
III. Strategic activities
1. Test Strategy/Plan
2. Test analysis/report
3. Other Professional services
IV. Other activities
V. Conclusion
Test Result Report & Assets
Dev/Quality issues based on test result refine best practices, documents, guides
I. Introduction
II. Engineering activities
III. Strategic activities
1. Test Strategy/Plan
2. Test analysis/report
3. Other Professional aids
IV. Other activities
V. Conclusion
[ Refinement about Test requirements ]
- Functional, Non-Functional requirements
- Test Plan for Non-Functional testing
[ Customized test strategy ]
- Architecture level test plan
- Communications with several group(dev, customer,
etc)
[ Items ]
- Purpose, Levels
- Input/Output, Entry/Exit Criteria
- Test env
- Roles and Responsibility
- Schedule
Test related documents(Plan, Scenario, etc) template, guide and drafts
테스트테스트테스트테스트 계획서계획서계획서계획서
Standized documents template and guide
FunctionalFunctionalFunctionalFunctionalTest Test Test Test
ReportReportReportReport
Standardized and Reliable Test Service
at the end of project
Total Quality reportTotal Quality reportTotal Quality reportTotal Quality report. Functional Testing result
. Performance Testing result. Security Testing result
. Software License verification. Code Inspection result
……
Regularly
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
1. 3 Amigos
2. Code review
3. Code Inspection(tool)
4. Standard Process advisor
V. Conclusion
Testing was a communication channel during the develoment
- readable, visible approach as possible to show and get feedback to customers
Customer
Ahha,
Graphical test case documents Complex workflow to real world
Publish readable TestCase Spec documents with TestCode comments- in-house tool which parse testcode comments and publish test specification on web
…
Customer
Readable, Real-time, Synchronized Test
Specification
@author
@Desc
@Step
@Input
@Expected
@Name
@TC_ID
GenerateGenerate
Writing commentsOther developer Dev lead,
tester
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
1. 3 Amigos
2. Code review
3. Code Inspection(tool)
4. Standard Process guide
V. Conclusion
Dev, Test code review (Online)
- Formal process for devcode review is done, testers mainly review testcode
- reviews testcase’s purpose, assertion and also work with test-coverage by checking
which line or branch covered or not
- if necessary, devcode review perfomed(often needed)
Test code review example
- The test method name should be test{target}_{prupose}_seq.
- Each test method should test one purpose.
- Copy&paste ‘throws Exception’ to each test method. We don’t need to exception handling (try,catch) in test code
- If you want to test the specific Exception occur, just type the exact exception name not general ‘Exception’. And use ‘fail’ statement just after the line you expect for code readers
- External resource(File..) should be referenced in relative path not fixed path in your pc
Dev Code review example
- Not covered toString method in test code.One more thin, Eh,.. It’s better to return at least empty String like “” rather than null…for toString() methods. It can occur NullPointerException if somenone uses and in some case.
- This checking code is not covered in testing. Well, you are double checking, so it looks impossible to reach that code. Need to refactor.
- This several catch clauses are not reachable…In my opinion, you can replace higher exception class than lower ones
- This method will return null, if something’s wrong, but the callers are not prepared. It needs to be considered
Rules and Occurrence
Code Inspection tool drive from the beginning
- Code Inspection tool(PMD) setting in the beginning of the project
- Detailed code guide for Top 10 violations regularly
Top 10, Rule explanation and solution, alternatives
PMD, security, performance,Official process, etc
Quality/TestShared Service
Dev Team
Shared SA
Fortify, white hacking~~
Shared TA
PMDPMD
SecuritySecurity
PerformancePerformance
PMD Rule customizingmeeting
Security testingmeeting
Performance test meeting
. How to measure web client response time(any tools)?
. How to test JavaScript?
. What is the official process to finish the project?
HelpDesk for other quality services
- As a quality member, introduce what to do, how to do in quality
- explain similar project cases and its pros/cons
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
V. Conclusion
1. retrospectives
2. Final message
3. Career path in testing
Retrospective 1
person Good Bad
C With PMD, I could learn a lot about coding. I could have experienced TDD
I failed TDD..
D Anyway finished, it’s good!!!With test cases co-writing, I could learn my code myself.
Too short deadlineDocuments synchrozing
E Test code, PMD helped my coding skill Too short time schedule
G At first, I didn’t believe testers cannot be helpful, but now we together worked for the product’s quality. That is important
Tight time schedulingTest Scenario planning was so good, it should be happened in very early phase.
H(PM)
Well, I have never thought co-working with testers in dev phase. Next time, this approach willbe basic.
“I thought they would do nothing because they knew nothing about algorithm,but what they did was that helping me understanding my product well”
Retrospective 2
[ Pair-testing ]
(Good) Lots of bugs could’ve been found. Pair-testing in Web UI was much more effective than
test code programming. It was good to hear other projects, other developers cases
(Bad) 30 or 40min is too short, I want to do more.
[ Test Scenario(cases) co-review ]
(Good) Well, I learned how you approach. It was much easier to think how to test.
(Bad) We changed requirements and test cases, that contents are not shred.
[ testcode pair-programming ]
(Good) It was good to show how Junit test code is written in detail. It was so easy after checking what you made.
(Bad) I needed to be prepared. I didn’t know much about Junit itself
[ Other words? ]
(Good) Time-boxing for every activity (30min, 40min) was really good.
(Bad) Well, I think there should be sessions that we talk what we should develop and test together.
Only with us, developers, we say similar things with similar views. Let’s work together.
Final Message
Dev TeamDev Team TestTest CustomerCustomer
테스트 강화로 제품의 기능& 품질 확보
관련 프로세스,산출물 가이드로 과제 자체에 대해 더 집중가능
코드 리뷰, 단위테스트 방안 가이드로개발 관점의 품질활동 수행
개발 시작부터 3자(사용자) 관점에서지속적인 피드백으로 협업
다양한 테스트 프랙티스에 대한 적용 경험 축적(소규모 과제)
개발 마지막에만 하는 테스트가 아니라개발 초기부터 능동적이고 적극적으로하는 테스트 수행
개발팀-고객 사이에서 가교 역할
기본 품질이 확보된제품 딜리버리
테스트 내용 (실시간)공유를 통해 제품에 대한 상세한 파악가능
제품에 대한 고객 관점의 피드백 제공 가능
win-win win-win
Easy Understanding
User centric product delivery
Easy Understanding
User centric product delivery
Synergy by working together
Tester’s Job shifting
DEVTEST
CUSTOMER
Test doesn’t knowhow much we are Burdened (crying.)
Dev doesn’t know what I want(Crying..)
No documents. Everyone’s busy (Crying..)
AS-IS
DEV TESTCUSTOMER
TEST
Test repsents us to dev(Thnx~)
Test came to help me(smile~)
My job is helpful for dev(smile~)
Became active (real) member To deliver the right product(smile~)
TO-BE
Various Test careerpaths
- as many as various work needed, as many as various job needed
methodolgist,
SA,TA,DA,…
developer
PM
PL
Tool engineer
Dev project
Tester Test Lead
Test Engineer Test Architect
Test Manager
Tester
TM
TL
Test project
Test Strategy
Education, guide
Manual, AutomationSupport
Q & ADo you think testers can write business test scenario?
Why testers care about Unit-testing?, It is the job for developers.
The activities are too many, do you mean we should do them all?
Basically, every member(including customers) should participate in writing business test scenario and evolved during the whole project period.
Who is in charge to write the draft depends on project’s conditions
Well, Unit-testing is the developer’s job. That’s right. However as quality people, we can check anywhere the quality exists, and
give them feedback with our profession, quality.Moreover we can learn much about product internally.
Of course not. The message is just delivering co-work with dev.Each person(testers) has different background, skills, interests, I just wanted toIntroduce various activities and make them as one, each career path
T.H.A.N.K Y.O.U