Post on 29-Aug-2018
transcript
Testing Professional Development
Framework
“Raising competency, or testing clones?”
Mike Jarred & Dr. Luke Avsejs
Slide 2 © IDBS 2012
Agenda
◦ Objectives, and constraints
◦ IDBS – Delivering Software to Regulated Environments
◦ What is the Testing PDF?
◦ Why did we do it?
◦ How have we implemented the PDF
◦ Some things we had to think about…
◦ Mutual Benefit
◦ The good, the bad and the beautiful…
◦ Closing thoughts – Q&A
Slide 3 © IDBS 2012
Objectives and Constraints
Objectives of this session
◦ It’s a ‘show and tell’ – our journey to date, not an attempt to sell a generic
approach
◦ Keep it interactive – feedback and thoughts from other practitioners in
the community
Garner feedback and improvement suggestions. This is under continuous
improvement, all ideas welcome
Constraint
◦ Our framework is not publically available, but check out the BCS SFIA
plus for a good starting point if interested
http://www.bcs.org/category/7849
Slide 4 © IDBS 2012
Delivering Software to Regulated
Environments
IDBS is a global provider of innovative enterprise data management, analytics
and modelling solutions
◦ Used by more than 200 pharmaceutical companies, major healthcare providers, global
leaders in academic study, and high tech companies
◦ Our products enable increased efficiency, reduced costs and improved productivity of
industrial R&D and clinical research
◦ Our platforms are enabling translational medicine – personalised medicine (right drug, right
patient, right time)
Our customers work in highly regulated environments
◦ GxP (GLP, GCP, GMP etc…) FDA regulated, 21 CFR Part 11, EU Annexe 11
Customers audit IDBS to ensure risk of non-compliance introduced by our
products is mitigated through robust software development practice
The Test Group are under particular scrutiny as we provide the safety net...
Slide 5 © IDBS 2012
IDBS Testing Group
IDBS operate a “Test Centre of Excellence”
Standardised Testing Methodology, incorporating company best practices and tools
Centralised resource pool providing risk mitigation across product portfolio ◦ 35 Team Members located globally - Guildford, London, Portland US, China
Structure
◦ I report to VP of Product Delivery, who is part of the executive leadership team reporting to the CEO.
◦ Director of Test for corporate test strategy, and developing group capability ◦ 2 - Test Managers responsible for tactical delivery across products ◦ 4 - Test Leads specialising in product lines ◦ Technical roles for cross product service lines, Test Automation, Security Testing, Test
Environments & DBA, Load & Performance Testing, Toolset Administration & Support
Primarily science domain knowledge, 10 PhDs
4 - IT career testers to augment domain expertise
Test follow a professional development framework to ensure necessary competencies / skills and certification (ISTQB) are available to support product development.
Slide 6 © IDBS 2012
So, what is the Testing PDF?
It’s a mechanism for specifying roles, career paths, competencies, skills,
skill levels, training and qualifications required to deliver testing services
to our projects
Competencies and skills are categorised as Technical (what we want
people to do) and Transferable (how we want people to be, inc. soft skills
/ behavioral attributes)
Framework facilitates a consistent & logical approach to staff
development through benchmarking of knowledge, experience,
training and qualifications
Under continuous improvement – updated regularly as new competencies
and skills are identified as the demands on testing change
Lets take a look at it in a little more detail…
Slide 7 © IDBS 2012
Career Path
Senior Tester ISTQB Advanced Technical Test Analyst
/ Test Analyst
Tester 1/2 ISTQB Foundation
Tester 0 ISTQB Foundation
Test Manager ISEB Practitioner / ISTQB Advanced /
PRINCE2 Practitioner
Test Lead / Test Team Lead ISTQB Advanced Test Manager / ISEB
Practitioner
Test DBA ISTQB Advanced Technical Test Analyst
/ Test Analyst
Automation Test Engineer ISTQB Advanced Technical Test Analyst
Career Level Project Experience Training
Senior Testing Experience should include participation of Test Strategy definition and tactical
implementation of Testing Processes on at least one large project and two medium
projects.
Writing Test Strategies
Writing Test Plans
Web Application Security Tester
Automation Test Lead ISTQB Advanced Technical Test
Analyst
Security Tester ISTQB Advanced Technical Test Analyst
/ CSTA
Load & Performance Tester ISTQB Advanced Technical Test Analyst
Slide 8 © IDBS 2012
Example Competency Skills Matrix
Competency Skill Tester 1 Tester 2 Senior Tester Example Evidence / Indicators
Environment &
Data
Administration /
Coordination
Able to define test environment
hardware, software and data
requirements
2 2 3
Test Environment Plan
Updated Project Issues Log
Able to support test team with
Oracle issues relating to
strategy and infrastructure
1 1 2
Able to maintain Oracle
instances 1 1 2
Able to give environment
management advice 1 2 2
Able to assist developers with
Oracle related problems. 1 1 1
Able to assist with database
enhancements 1 1 2
Programming with Java 1 1 3
Test Design &
Maintenance
Test Script Creation 3 3 4 Fully Accepted Test Script
Checklists
Test Coverage Report
Updated Regression Test Packs
Test Script Maintenance 3 3 4
Test Model Creation and
Implementation 1 2 2
Use of Oracle, SQL and
PL/SQL within test scripts 1 2 3
Skill Level (0-5)
Skill Level Description
1 - Basic Basic knowledge has been acquired through limited training. Very limited experience, through classroom and/or case studies, or on-the-job training. Regular direction and assistance
is required to perform duties
2 - Intermediate Broader base of training has been received. A few successful experiences have been completed.
3- Senior Extensive training has been received. Several successful experiences have been completed. Performs almost all actions associated with this skill without assistance - may
occasionally require assistance from a more senior individual for only the most difficult aspects of this skill.
4 - Expert Very extensive training has been received. Training may be provided by this individual within their organisation. Many repeated successful experiences have been completed in a
variety of complex settings. Performs actions associated with this skill without assistance. Recognised within the organisation as "the person to ask" when difficult problems arise.
5- Master Very extensive training has been received. Is sought after to train others. Many repeated successful experiences have been completed in the most complex settings. Performs
actions associated with this skill without assistance. Is sought after, from both within and outside their organisation, when the most difficult problems arise.
Slide 9 © IDBS 2012
So, why did we do this…?
Organisational Drivers
Demonstrate to customers how we holistically maintain a competent testing group (we are always asked under
audit)
Demonstrate internally that we are driving professionalism / increasing capability in test group
Enable clear objective setting and structured appraisal reviews
Provide clarity regarding competencies and skills needed in the test group
Demonstrate the Career Paths open to individuals in the Test Group – there was an org structure, but no
visibility of how to progress
Assisting in formalising competency based recruitment
Slide 10 © IDBS 2012
How have we applied the PDF?
Awareness & Training Implementation
Overview of PDF, benefits, risks and pilot
implementation plan provided to Exec team –
secured approval and sponsorship
Team members skill levels for competencies are validated through the appraisal mechanism. Evidence of skill levels required
Test Management contributed and reviewed PDF
content
Outputs from appraisal are linked to Personal
Development Plans
Test Group received PDF overview as part of
change management
Experience, skills and qualifications are continuously
reviewed as part of personal development meetings
Initial 6 month pilot to capture improvement
suggestions
Once a level on the PDF is established, further
opportunities for development will be provided where
possible as part of project delivery and departmental
initiatives
Slide 12 © IDBS 2012
Mutual benefit
Benefits to Testers Benefits to Company
A more structured & professional approach to career development
Supports Continuing Professional Development (CPD)
Increased sales and retention of existing business through successful audits
Visibility to all staff of how to progress within
testing
Employee retention / recruitment advantage
Increase skill levels and broaden experience Consistent appraisals across the department, equitable &
quantitative scoring
Gain Industry recognized qualifications Increased awareness of training needs to budgeting and
planning.
Commitment from IDBS to their development Identification of skills gaps, succession planning, coaching
& mentoring
Opportunities to cross skill into a different
framework e.g. Development (once rolled out to
other groups)
Assurance that recruitment targets area of greatest value
Slide 13 © IDBS 2012
The Good, the Bad and the Beautiful…
Lots of positive feedback (within, and outside the Test
Group)
Training economies of scale
Competency based recruitment, and succession planning
Increasing team flexibility
Some scepticism over value of PDF and our ability to
support career aspirations
Evidencing skill levels has proven time consuming, but is a
one off activity – consensus is it’s worth investing in
Evidencing transferable skills is difficult
Heat map / Pie Charts (management information is beautiful)
Slide 14 © IDBS 2012
Example heat map & pie charts
COMPETENCY SKILL Job Role 1 Job Role 1 Job Role 2 Job Role 2
Competency 1
Competency Skill 1 2 2 3 4
Competency Skill 2 1 2 2 2
Competency Skill 3 1 1 1 2
Competency Skill 4 2 2 2 3
Competency Skill 5 3 1 2 4
Competency Skill 6 1 3 2 4
Competency Skill 7 1 1 2 4
Competency 2
Competency Skill 1 2 1 2 2
Competency Skill 2 2 2 3 4
Competency Skill 3 1 1 3 4
Competency Skill 4 0 1 2 3
52%
30%
18%
Test Team Competency
Attaining
Not Meeting
Exceeding
51%
25%
24%
Tester 2 Competency
Attaining
Not Meeting
Exceeding
Slide 15 © IDBS 2012
Closing thoughts from our team…
“I find that it opens the discussion about what your line manager
expects of you in your role, and allows me to identify areas where I
could improve prior to having that meeting. It also provides a good
backing for the appraisal, which is pretty invaluable, as you tend to
forget the things you’ve done throughout the year”
“It provides a structure in line with the company organisation to allow us
to see the different paths for professional development; giving a clear
yet open idea about career progression within the team”
“I think it is useful. The only downside is that the number of fields does
make it lengthy to complete, it seems some fields could be consolidated
or perhaps a few removed that don’t seem relevant to a testers role”