Design of a Large Industrial Project Risk Tool – Calculating Risk and

Opportunity Adjusted ROI

Ken J. Kallaher, P.E.Corporate Fellow

ken.kallaher@exp.com1-412-616-8854

Large Project Risk Management

2

•

Aviation

Bridges

Brownfield Development

Chemical

Civic + Cultural

Commercial + Retail

Education

Entertainment + Sports

Federal

Food Processing

Forest Products

Healthcare

Highways + Roads

Hospitality + Gaming

Hydroelectric

Hydrology + Hydraulics

Justice + Correctional

Manufacturing

Mining

Mission Critical

Nuclear

Petrochemical

Pipelines

Ports + Marine

Rail + Transit

Renewables + Alternative Energy

Science + Technology

Solid Waste + Recycling

Transmission + Distribution

Urban Development

Waste Water Collection + Treatment

Water Supply, Treatment + Distribution

•

Picture

Picture

Designs and implements

Project Management Systems and

Processes

What does our group in exp do?

• Engineering Management• Planning & Scheduling• Cost Management

• Contract Management (Goods and Services)

• 20 other fields of expertise

• Pre-Project Phase, from Idea to Project funding

• Execution Phase

• Start-up

Manages Projects across

the entire Project life

cycle

Develops and manages

Operations Readiness

Plans

Provides tailored Project

Management Services

Sales and implementation of specialized

project management

software

5

« Specialized »• Development and Implementation of Project Selection

and Delivery Models (PSDM)

• Development and Implementation of Project

Management Systems (Procedures)

• Development and Management of Operations

Readiness Plans (ORP)

• Commissioning and Start-up Management

• Development and Implementation of Project

Governance Structure

• Review and Audit of Project Controls

• Project Finance Modeling Assistance

• Management (or support) of Owners activities

• Procurement and Logistics Management

• Plant shut-down Management

• Portfolio Management

• «VIPs»

• Claims Analysis

• Bankable Studies

Areas of competency« Traditional »

PMI type of services• Project management training and coaching

• Management of Pre-Project Phase

• Risk Management

• Value Engineering

• Project Direction

• Detailed study coordination

• Construction Management

• Scope Management

• Project Planning and Scheduling

• Cost Control

• Performance Management

6

Picture

Picture

A few of our valued clients

7

Megaproject Failure Rate: 56%

8

Current Condition• Client X >$20B/yr. revenue, >80 countries• Largest BU competes in highly competitive markets

requiring large projects to be built on tight schedules • Other BUs - smaller highly custom/specialized• Highly distributed / independent regions / BU with

strong central control over funding – multiple cultures• Centralized process – not consistently followed, pencil

whip “check the box” attitude prevalent with some until an Executive (or up and comer) asks tough questions

• Very successful company with many “wins” & great projects + some very significant overruns & late

9

Gaps

• Qualitative risk analysis done too early (insufficient data) with strong regional bias

• No formal opportunity analysis done or pursued• Quantitative analysis done by EPC to protect EPC only• Capital allocation between competing projects done by

Sponsor credibility & ability to withstand “questioning”• Historically decentralized project organization with

pockets of excellence and pockets of inexperience (can’t change central process, minimal impact on local)

10

Target Condition – “Top 7 Slides”• Consistent, repeatable process based on *GBP• Easy to do qualitative analysis very early in project

development to screen for show stoppers & focus *RMP• One tool to capture and prioritize risks and opportunities

from geographically diverse & highly independent groups• Facilitated qualitative analysis to create *RIRR from top

20 (maximum) independent risks / opportunities• Graphical “easy to visualize” results usable by domain

experts and top executives• Can’t do “top 10” in a 50 minute presentation – review

first principals first, then tool

11

* LegendGBP - Global Best PracticesRMP - Risk Management ProcessRIRR - Risk Adjusted IRR

Modern Risk Management

• AS/NZS 4360:1995• AS/NZS 4360:2004• ISO 31000:xxxx• PMI, AACEi standards• Pick 1• Don’t reinvent the wheel

12

1

13

Identify Risk Analyze Risk

Response Planning

Tracking

ControlIterate!!

Statement of risk

List of risks(risk register)

Risk evaluation

Risk Classification

Risk prioritization

Risk Mitigation plans

Risk Acceptance Rationale

Risk Tracking Requirements

Risk status report

Risk mitigation plans

Risk decisions

13

Risk Management Process – Iterate!2

Iterate

• This is a consistent a weakness I observe• During each monthly iteration, look at triggers and

leading indicators for changes• What new risks have materialized?• What risks have increased or decreased in probability

or consequences?• Doesn’t need to be long or bureaucratic (software can

make this much less painful)

14

2

Use Appropriate Risk Treatment• Owners with limited/no megaproject experience often

try to transfer all risks without understanding risk premium• Party with greatest influence/control over risk should own it

15

3

“What if” Planning• Traditional approach:Identify potential risksCapture in risk registerAssign someone to each risk (typically a junior person)Check risk management off of “must do to get $$ list”Move on to “more important” tasks

• Ask Integrated Project Team (project risks) and Steering Team (strategic risks) how they will respond to hypothetical but realistic scenarios:What if detailed engineering progress is not on track to

meet the (politically sensitive) start of construction date?For a location sensitive project, what if the primary

location is suddenly unavailable (Trinidad example)?

16

4

Standardized Frameworks Help

• Consider a complete graphical risk breakdown structure to organize risks and to prompt thinking about areas that are sometimes missed

• Strongly consider a standardized probability and consequences matrix that allows all risks (project and strategic) to use the same scale and be captured in one unified project risk register

17

5

Picture

Picture

18

Risk Breakdown Structure5

##

19

Judgment Frequency Experience

5Almost certain

orFrequent

Expected to occurVery High, may occur at least several times per year

A similar outcome has arisen several times per year in local operations

4Likely

orProbable

More likely to occur than not occur.

High, may occur about once a year

A similar outcome has arisen several times per year in the company worldwide or broader industry

3Possible

orOccasional

As likely to occur as not to occur.

Possible, may occur at least once in a one to ten year period

A similar outcome has arisen at some time previously in local operations

2Unlikely

orRemote

Not impossible, more likely not to occur than to occur.

Not impossible, likely to occur during the next ten to twenty five years

A similar outcome has arisen at some time previously in the company worldwide or broader industry

1Rare

orImprobable

Very unlikely to occur.Very low, very unlikely during the next twenty five years

No experience of this happening in the broader worldwide industry but is theoretically possible

Rating Basis for Probability Rating

##

20

ConsequencesHealth & Safety Environment Regulatory Image & Reputation Financial Impact Facility Integrity Project Performance Employees

5CRITICAL

Fata l i ty of s taff, contractor or the publ ic

Long term environmenta l damage (5 years or longer), requiring >$5 mi l l ion to s tudy or correct or in penal ties

Regulatory intervention and prosecution poss ible

Damage to corporate reputation at international level ; ra ised in international media .Major loss of shareholder, pol i ti ca l or community support

Direct loss or increased cost > $20 mi l l ionEstimating error or capi ta l loss > $20 mi l l ionFraud > $1 mi l l ion

Major unacceptable system, asset, integri ty or condition problemFai lure to achieve cri tica l system, asset or performance goals

Time-cri tica l project misses major mi lestone or deadl ine > 6 monthsFa i lure to achieve cri tica l system, asset or performance

A large number of senior managers or experienced employees leave the company.

4MAJOR

Serious injury or occupationali l lness (non-recoverable) or permanent major disabi l i ties (acute or chronic)

Medium-term (1-5 yr) envi ronmenta l damage, requiring $1 to 5 mi l l ion to s tudy or correct

Breach of l i censes , legis lation, regulation or corporate mandatory s tandards

Damage to corporate reputation at national level ; ra ised in national media . Signi ficant decrease in shareholder, pol i ti ca l or community support

Direct loss or increased cost of $5 – 20 mi l l ionEstimating error or capi ta l loss of $5 – 20 mi l l ionFraud $0.5 - 1 mi l l ion

Fai lure to achieve some system, asset, integri ty or condition targetsFa i lure to achieve some performance targets

Time-cri tica l project misses major mi lestoneor deadl ine by 3-6 monthsFa i lure to achieve some performance targets

Some senior managers or experienced employees leave. High turnover of experienced employees . Company not perceived as an employer of choice.

3MODERATE

Lost time or restricted duties injury or occupational i l lness (recoverable)

Short-term (<1 yr) envi ronmenta l damage, requiring up to $1 mi l l ion to correct

Breach of s tandards , guidel ines or impending legis lation. Subject ra ised as corporate concern through audit findings or voluntary agreements

Adverse news in s tate or regional media . Decrease in shareholder, pol i ti ca l , or community support

Direct loss or increased cost of $1 – 5 mi l l ionEstimating error or capi ta l loss of $1 – 5 mi l l ionFraud $0.25 - 0.5 mi l l ion

Some reduction in system, asset,integri ty or condition Some reduction in performance

Time-cri tica l project misses major mi lestoneor deadl ine by 1-3 monthsSome reduction in performance

Poor reputation as an employer. Widespread employee atti tude problems. High employee turnover.

2MINOR

Medica l Treatment or Fi rs t Aid InjuryNo lost time or occupational i l lness

Environmenta l damage, requiring up to $250,000 to s tudy or correct

Breach of internal procedures or guidel ines

Adverse news in loca l media . Concerns on performance ra ised by shareholders , government or the community

Direct loss or increased cost of $0.25 – 1 mi l l ionEstimating error or capi ta l loss of $0.25 – 1 mi l l ionFraud $0.1 - 0.25 mi l l ion

Minor system, asset, integri ty or conditiondegradationMinor performance degradation

Time-cri tica l project misses major mi lestone or deadl ine by <1 month. Minor performance degradation.

Genera l employee morale and atti tude problems. Increase in employee turnover.

1INSIGNIFICANT

No Injury Negl igible envi ronmenta l impact, managed within operating budgets

No breach of l i censes , s tandards , guidel ines or related audit findings

Publ ic awareness may exis t, but there i s no publ ic concern

Direct loss or increased cost below $250,000 Negl igible estimating error or capi ta l lossNegl igible fraud

Negl igible system, asset, integri ty or condition impactNegl igible performance impact

Negl igible mi lestone or deadl ine delay. Negl igible performance impact

Negl igible or i solated employee dissatis faction.

Rating

Opportunities (Positive Risks)

• Use the same level of systematic capture, assignment, strategy and tracking as for negative risks

• Decide which ones (probability x consequence) are worth investing time and resources to go after

• Sometime pursuit of an opportunity requires a significant project change and approval of key stakeholders

• Must identify and decide to pursue before end of FEL 2 to avoid high risk FEL 3 scope changes

• Good article this topic in Dec. 2012 Project Management Journal: “Challenging Classic Project Management: Turning Project Uncertainties into Business Opportunities”

21

6

Operations Readiness

• This is frequently a weakness in megaprojects particularly for greenfield

• New region/country amplifies the issues• Integrated Project Team (is supposed to) have systems,

discipline, project controls tracking, etc.• Operations are often left on their own• Global Best Practice is full integration of Operations into

Project Development and Execution System (PDS)

22

7

Industrial Project Delivery Model

Start Construction

RFAFull Funds

Advance Check Off

Agreements Signed

MOU Basic EngineringCompleted

Screening Development Delivery

Benefits

FEL 1Concept

FEL 2Pre-Feasibility

2D 3B

StrategicResources

Assessment

Test

ing

Pre-

notic

e

Basic Engineering

Prepare RFQ Documents

Tests Under Load

Production Ramp-Up Production on Specs

(Quantity / Quality)

Warranty Period

Facilities Mechanical Acceptance Completed

Bid Period

Decis

ion

by

Revie

w Bo

ard

Turnover

X weeks Notice

Star

t Me

chan

ical

Acce

ptan

ce

Final Acceptance

Joint Action Plan

ProjectAssessment

EPCM

Con

tract

Awa

rd

Industrial Acceptance

Star

t Me

chan

ical A

ccep

tanc

e

EPCM

FEL 3Feasibility

Fund

s Av

ailab

le

Integrated Project Plan

Preliminary EngineeringConcept

Project Business Case Development

Plans & Programs Development

Proj

ect C

harte

r

Proj

ect S

tate

men

t

Oppo

rtuni

ty S

tate

men

t

Operations

4C

Class 3 Class 4 FinalClass 5 Estimate Types Class 2 Class 1

Concept

Study by Review Board

Members

Engineering and other Service Providers contribution

RFA & Information

Memorandum

Funds Appro-priation

RFA

Docs

Issue

d

Preliminary Eng. Basic Engineering

Construction

Release Release Release

Start Turnover

EPCM Bidding & Award Process

Star

t Det

ailed

En

gine

erin

g

ORP Level 0

ORP Level 1

Operations Readiness Plan Level 2

1B 1C 2A 2B 2C 3A 3C 3D

Operations Readiness Plan Level 3

Ideation

Star

t of I

deat

ion

Stag

e

FEL 0Pre-Concept

Acceptance Process

Execution Commissioning Ramp-up Productionon Specs.

4A 4B

Pre-Op. Testing

Dry Tests

FRONT END LOADING (FEL)

Major Gate

Sub-Gate

1A

Star

t ORP

Im

plem

ent.

Bids Analysis & Negotiation

Detailed Eng.

Procurement

By Operations & EPCM

Class 4Initial

The Contracting Process to secure Engineering and other Services at appropriate time to satisfy requirements of FEL stages (and beyond) is not illustrated in this diagram.

Class (AACE) Designation

54 Initial4 Final321

FEL 0 (Pre-Concept)FEL 1 (Concept)FEL 2 (Pre-Feasibility)FEL 3 (Feasibility)DefinitiveFinite

Accuracy(Greenfield)

(-30%,+60%)(-20%,+30%)(-15%,+25%)(-10%,+10%)(Better than ±10%)(Better than ±5%)

Type of Estimates

Accuracy(Brownfield & Mining)(-30%,+60%)(-20%,+30%)(-15%,+25%)(-10%,+15%)(Better than ±10%)(Better than ±5%)

23

7

EMENTATION

PRE-FEASIBILITY FEASIBILITY EXECUTION

Final Acceptance per Specifications

Full Funds Appropriation

High level Plan setting the course of

action in view ofoperations

Baseline Plan including manpower deployment

and departments’ implementation

strategy

Detailed OR Plan by business process,

done with department’s personnel

Progressive deployment of Operations resources, and execution of activities and deliverables according to OR detailed Plan

ORP Level 1 ORP Level 2 ORP Level 3 ORP Level 3 DEVELOPMENT &IMPLEMENTATION

MONITORING / ENFORCEMENT

Start Construction

COMMISS. & START-UP

Mechanical Completion

Preparation Enforcement• Align OR and project

key milestones

• Identify major categories of OR deliverables

• Draft of organizational structure

• Develop OR master schedule

• Develop preliminary OR and operations budgets

• Identify OR work by Owner and by others

• Develop OR control schedule

• List of business processes by department

• Refine / upgrade ORP documents initiated in ORP Level 1, to satisfy requirements of project funding and IRR calculation

• Develop short-term detailed OR plan

• Develop detailed OR schedule and synchronize with project schedule

• Develop / Implement OR progress monitoring system

• Validate OR budget

• Initiate schedule progress reporting and budget control functions

• Assist Owner’s managers with their ORP implementation

• OR schedule and progress monitoring

• Maintain OR schedule ¨evergreen¨according to OR progress and project evolution

• OR budget monitoring

• Act as facilitators to managers in their Plan’s implementation / execution and dealings with other departments

• Assistance in development and implementation of procedures for facilities tests, commissioning, start-up and acceptances

Implementation

Project Life Cycle and ORP Evolution Processex

p.’s

Key

Con

tribu

tions

24

7

• Operations and Maintenance• Business Management• Procurement / Outsourcing• Sales and Marketing• Information Systems and Technology• Commissioning

• Operations Readiness (transitional department)• Environment, Health and Safety • Human Resources• Community Relations• Finance• Legal Affairs

• List of major categories of OR deliverables by department• Preliminary Owner’s organization, roles and responsibilities• OR master schedule by department (See box below for example)• List of OR work packages (Owner’s & others)• Preliminary effort assessment and resources deployment plan• Preliminary OR budget• Operations and maintenance philosophy• Commissioning and start-up strategy/model• Ramp-up strategy/model

PRE-FEASIBILITY

ORP Level 1

Example of departments covered by an ORP:

Typical ORP Level 1 Deliverables

25

7

Solution for Client X• Solution included a process that built on client’s current practices• Excel based tool with all quantitative analysis done in @RISK• Input: simple cash flows copied direct from client’s financial model• Each tab of the tool dedicated to specific function or information

(Consequences tab, Graphical RBS tab, cash flow inputs, etc.)• Build a bridge between current condition and global best practices

26

Simple Early Qualitative Tool

27

• Bridge to client’s current process• As more is known add standard 5 x 5 probability & consequences • Before full funding, quantitative analysis (beyond EPC) is added

Risk register input IRR per line Output

28

• Single register eliminates qualitative vs. quantitative redundancy• Can’t see from this sanitized view – Capex / Opex selection• Best / expected / worst used as parameters for @RISK distribution• 15 Level 2 entries in RBS, Level 1 grouped automatically - graphs

29

30

how

31

X

22% chance Sponsor gets fired (WACC=11%)5.7% chance of exceeding BU hurdle rateExperts saw it & immediately began: what if….

32

33

Questions and answers

34

Backup Slides

Qualitative Risk Analysis• This consists of using the probability and impact matrix tool

(PIM) to prioritize and rank the risks contained in the risk register. By doing this, you are able to focus your management time and effort on the most important risk areas. In effect, you are able to apply the 80/20 rule – 20% of the risks will cause 80% of the threats to your project objectives, hence you need to focus on those.

• Each risk would be evaluated for it’s probability and impact using a numbered ranking system such as low, low to medium, medium, medium to high, and high. Or perhaps using a 1 to 10 scoring system. Each risk will have the numbers for probability multiplied together to get a priority score so that these can be ranked. Other data could be captured for each risk such as urgency or proximity, and the category (for example hardware, software, commercial, design, and so on).

36

Quantitative Risk Analysis• As the name might suggest, this is quantifying (assigning a

value = quantity) the ranked risks often done in terms of time or cost. There are several tools that help in this. One is by using decision trees to arrive at a monetary amount for each risk (the extra cost incurred or time delay if the risk happened).

• Another tool is expected monetary value analysis. If a risk would incur an extra $1000 and it has a 25% probability, then the risk cost value could be seen as $250. If this was done for all the risks, then simply adding up each risk cost value could be used as a risk budget to help fund the aggregated risks.

• A risk is an uncertain event, that if it did occur, it would have an impact on the project’s objectives. But a risk can be a negative impact threat or a positive impact opportunity. Both have uncertainty which is what makes them both examples of a risk.

37

Risk Response“Negative” Risk Responses• Transfer - Reallocate the risk to others• Accept - Do not develop mitigation strategies. May develop

contingency plan if needed• Watch - Monitor risk attributes; establish metrics; identify

contingency strategy if needed• Mitigate - Eliminate or reduce likelihood of occurrence or impact;

identify contingency plan• Avoid - Find a way around risk

38

Bankable Quality Projects

• Bank’s engineer will typically want to see the risk register monthly and will expect changes (iteration)

• Risk Management is a pervasive theme with exceptionally high pressure to transfer all risks

• Partnership projects significantly complicate risk management process (not linear with # of partners)

• Consider project from Banker’s perspective (typical Banker’s Checklist slide)

39

Covered by IM Chapter No.

1. A credit risk rather than an equity risk is involved. 72. A satisfactory feasibility study and financial plan have been prepared. 4 & 73. The cost of product or raw material to be used by the project is assured. 4 & 54. A supply of energy at reasonable cost has been assured. 4 & 55. A market exists for the product to be produced. 36. Transportation is available at reasonable cost to move the product to the market. 5 & 67. Adequate communications are available. 4 & 58. Building materials are available at costs contemplated. 49. The EPCM contractor is experienced and reliable. 4 & 510. The Operations Team is experienced and reliable. 4, 5 & 611. Management personnel are experienced and reliable. 412. Unproven technology is not involved. 413. Contractual agreement among joint venture partners, if any, is satisfactory. 214. A stable and friendly political environment exists; licences and permits are available. 4 & 615. The risk of expropriation has been properly considered 816. Country risk is satisfactory. 817. Sovereign risk is satisfactory. 818. Currency and foreign exchange risks have been addressed. 719. The key promoters have made an adequate equity contribution. 2 & 720. The project has value as collateral. 721. Satisfactory appraisals of resources and assets have been addressed. 422. Adequate insurance coverage is contemplated. 823. Force majeure risk has been addressed. 824. Cost over-run risk has been addressed. 825. Delay risk has been considered. 826. The project will have an adequate ROE, ROI and ROA for the investors. 727. Inflation rate projections are realistic. 728. Interest rate projections are realistic. 729. Environmental Impact has been addressed. 430. Socio-Economic Impact has been addressed. 4

Check Items

Sample Banker’s Checklist

40

##

Delivery PhaseFEL 3 Feasibility

3A 3BAgreements

SignedBasic Eng.Completed Full Funds

Studyby DRB

Members

Fundsappro-

priation

Dec

isio

nby

DR

B

Fund

s Av

aila

ble

Project Related Documents

Operations ReadinessPlan (ORP)

IRR, Funding& Financing

End Product Value

3AxxFEL 3 Tech. Doc.

3AxxFEL 3 Study

Report

3AxxIntegrated Project Plan 1st Ed.

3AxxFunding Agreement Draft

3AxxEPCM RFQ Doc.

3Axx

3BxxClass 3

Project Estimate

3BxxProj. Business Case - Final

3BxxEPCM Select. Summary

3BxxProject Charter Final

3BxxRaw MaterialsSupply Plan

3BxxProj. EHSProgram

3BxxProduct & Marketing Plan

3BxxORP – Level 2

3BxxProj. Delivery Agreement3Bxx

Integ. Project Plan– Final

Project auth

2Dxx

Energy Contract

EPC

/M C

ontr

act A

war

d

Stud

y Te

am a

nd

Engi

neer

ing

Serv

ices

Pro

vide

r M

obili

zatio

n

2D

2DxxLand Agreements

Lenders & Partners Financial Requirements - Final

Partnership Agreements

Ban

kabl

eD

ocum

ents

–3B

xx

2Dxx

Proj. Business Case - FEL2

41

Strategic Risk Mgnt Plan

Bankable Quality Projects

42

FinancingStructure Partners

OperationsReadiness

Balance Of

Stage GateDeliverables

Bankable Cost

Estimate &Schedule

ProjectRationale

& Economics

EnablingFramework&

Government

People

Bankable Quality – True North

43

Alcoa Confidential

Strong well staffed Owner’s team Supported by a quality engineering service provider

(size, balance sheet, recent track record, experienced people)

Tier 1 EPC/M team utilizing proven systems and procedures

Minimal risk of changes in team or loss of key personnel (incentives); financial penalties to EPC/M for loss of key people

Plan and commitment for quality, experienced construction supervision and workforce

Quantity and quality of team are key to success (“A” team not “C” team). Expect careful scrutiny of key team CV’s for experience both within the industry as well in projects of similar size.

People

Bankable Quality – True North

44