Past Performance Information
John SavickyArizona State University
May 2013
2
PIPS Best-Value Process
Identification of PotentialBest-Value
PHASE 1
Pre Planningand
Quality Control
PHASE 2
Award | Measurement &Documentation
PHASE 3
3
PIPS Has Multiple Filters• No single step will complete minimize
all risk. Process has several steps that act as filters.
• Filters do not limit competition, but make it difficult for non-performers to be competitive.
• Looking for dominant information
4
What is the Objective of PPI?• PPI should not be used as the sole factor to ensure that the vendor can
perform on a current project.
• Communicate to the vendors that the owner will purchase based on value (performance and price).
• Identify if a vendor has capability to perform.
• Encourage vendor to use and partner with the best people / best subs
4
5
How Should We Collect PPI?1.Survey all of the vendors past
projects?2.Survey all projects done for just your
organization?3.Select a random number of past
projects (good and bad)?4.Survey the last ten projects?5.Survey all the projects completed in
the last 2 years?6.Survey only projects that are similar
to the type of project being procured?
6
In PIPS, Vendors Are Allowed To Only Select Their Best
Past Projects
7
Is This Any Different Than The Traditional Approach?
8
What Questions Should We Ask?• Ability to manage cost• Ability to manage schedule• Ability to communicate• Ability to manage subcontractors• Ability to follow facility rules• Ability to minimize change orders• Ability to minimize claims• Ability to minimize punch list items• Knowledge of codes• Management abilities• Quality of submittals• Quality of workmanship• Professionalism
8
Response time to emergencies Response time to warranty work Knowledge of regulations Willingness to do value engineering Contractor’s level of honesty Overall construction knowledge Overall construction quality Overall performance of contractor Customer public relations Your overall customer satisfaction Was job completed on time Skill of craftsmen
9
What Questions Should We Ask?
9
10
Statistical Analysis• Objective:
– Determine if survey questionnaire could be minimized
• Methodology: – Analyzed 175 contractor PPI databases (GC, Elec, Fire Alarm)– Analyzed 37 criteria– Perform statistical analysis
• Results:– Results indicated 8 criteria would generate same results as asking 37– Re-tested 8 criteria on 50 randomly selected contractors. – Further tested on 29 randomly selected HVAC, Plumbing, and Painting
vendors.
10
11
Survey Form
11
12
Collecting Surveys
• Option 1 – Client collects surveys on all vendors
• Option 2 – Client hires a third party to collect surveys
• Option 3 – Vendors collect their own surveys
13
PPI Analysis• If Client collects surveys, be prepared for additional work
– Must call to confirm– Must have several machines to accept surveys / paper– Must have people to contact regarding surveys– Must enter data and double check for errors– City of Peoria = over 8,000 surveys– University of Minnesota = over 4,000 surveys
14
In PIPS, Vendors Are Required to Survey, Collect,
and Submit Their Own Surveys
15
DETAILED STEPS
Past Performance Information• PPI will be collected on the critical Entities:
– Firm– Project Manager (Individual)– Key Subs or Suppliers
• Each Entity must prepare and submit a list of references, customer surveys, and Past Performance Information Scores
16
VendorENTITYPrepare and Send Survey Questionnaires to Past Clients
Step 2
Step 3Collect/Receive Completed Surveys
Prepare Reference ListStep 1
Enter data into Reference ListStep 4
Package all material (Reference List and Surveys) and SubmitStep 5
Reference List • Maximum number of references that can be submitted is 10 for each Entity.
• The past projects must be installed and operational.
• The reference list must contain different projects.
• The reference list must contain different clients.
• The client must complete the survey (no consultants or third parties)
17
Survey Questionnaire • Each Entity must prepare, send out, and
collect survey questionnaires.
• All returned survey MUST be evaluated AND signed by the client. If a survey is not signed, it will NOT be counted or considered.
• Returned surveys must be submitted with Proposer's proposal (Proposer should keep a copy of all returned surveys).
18
PPI Score• Proposer shall input the survey scores,
and then average all of the responses to obtain the “Overall Average Score”.
• Proposer shall also count the “Total Number of Returned Surveys”.
• The Client may contact the reference to clarify or to obtain additional information. If the reference cannot be contacted, the survey may be deleted.
19
No Criteria
Surv
ey 1
Surv
ey 2
Surv
ey 3
Surv
ey 4
Surv
ey 5
Average
1Satisfaction with the staff assigned to the project
10 9 10 10 9.8
2Ability to meet your goals and expectations
10 9 10 10 9.8
3Ability to integrate and interface with any existing systems
10 9 9 10 9.5
4Ability to communicate and document system functionality
10 9 9 10 9.5
5 Overall quality of the installed product 10 9 9 10 9.5
6Overall quality of the company’s service
10 9 10 10 9.8
7 Overall customer satisfaction 10 9 10 10 9.8
8Ease of implementing new requirements into the system
10 9 9.5
9.6
4
Overall Average Score:
Total Number of Surveys Returned:
20
Summary• PPI is only one step in the process.
• PPI is not used as the sole factor to determine future performance.
• The PBSRG has collected PPI in many different ways:– Sent surveys to past owners, called owners,
collected data– Allowed vendors to send surveys, but collected the
data– Allowed vendors to send and collect surveys
• The goal is to be efficient. Research has shown that the most optimal way to collect PPI is to place the vendor at risk and allow them to be completely responsible for their PPI.
21
Comments / Questions
W W W . P B S R G . C O M