COL ALEKSANDAR GEORGIEV
ANALYSIS AND LESSONS LEARNED SECTION HEAD
CMDR COE
Office: +359 2 92 24731
NATO LESSONS LEARNED
PROCESS
EXERCISE “CRNA GORA 2016”
LESSONS IDENTIFIED CONFERENCE26 January 2017
NATO UNCLASSIFIED
2
• LL Definition & Purpose
• LL Capability
• NATO LL Process- NATO “ODCR” Format
- Analysis – From Observation to Lessons Identified
- Staffing Lessons Identified to Lessons Learned
- Lessons Learned Information Sharing
• What is in place?
• Conclusion
Agenda
NATO UNCLASSIFIED
3
NATO UNCLASSIFIED
Lessons Learned Definition
“ A term used to describe people, things, and activities
related to the act of learning from experience to
achieve improvements.”
“ …In a military context, this means reduced operational risk,
increased cost efficiency, and improved operational
effectiveness.”
NATO Lessons Learned Handbook (2016)
4
NATO UNCLASSIFIED
LESSONS LEARNED PURPOSE
“The purpose of a Lessons Learned procedure is
to learn efficiently from experience and to
provide validated justifications for amending the
existing way of doing things, in order to improve
performance, both during the course of an
operation and for subsequent operations.”
AJP 3(B) Allied Joint Doctrine for the Conduct of Operations (March 2011)
5
NATO UNCLASSIFIED
“A LL capability provides a commander with the structure, process and tools necessary to capture, analyse and take remedial action on any issue and to communicate and share results to achieve improvements.”
Bi-SC Directive 80-6, 06 July 2011
LESSONS LEARNED CAPABILITY
6
NATO LL PROCESS
L E S SON
I DENT I F I ED
Understand root causes
Propose: Remedial Actions to resolve the problem
Action Body to implement the Remedial Action.
ANALYSIS
Problem
Procedure
Good practice
OBSERVE
LLLIAnalysis Phase Remedial Action Phase
Commander’sGuidance
Improvedcapability
OBS
Post
Publish
Promote
DISSEMINATE
Command:Endorses Remedial ActionsCommits resourcesTasks an Action body
ENDORSE & TASK
L E S SON
L E ARNE D
AB: prepares Action Plan, implements RA, reports progress
Tasking authority: monitors progress
IMPLEMENT VALIDATE
Verify results
SHARESHARE
Validation
NATO UNCLASSIFIED
7
NATO LL PROCESS (2)
L E S SON
I DENT I F I ED
Understand root causes
Propose: Remedial Actions to resolve the problem
Action Body to implement the Remedial Action.
ANALYSIS
Problem
Procedure
Good practice
OBSERVE
LLLIAnalysis Phase Remedial Action Phase
Commander’sGuidance
Improvedcapability
OBS
Post
Publish
Promote
DISSEMINATE
Command:Endorses Remedial ActionsCommits resourcesTasks an Action body
ENDORSE & TASK
L E S SON
L E ARNE D
AB: prepares Action Plan, implements RA, reports progress
Tasking authority: monitors progress
IMPLEMENT VALIDATE
Verify results
SHARESHARE
Validation
NATO UNCLASSIFIED
8
OBSERVATION AND NATO “ODCR” FORMAT
Title
• Observation Description: “what happened?”
• Discussion: “why did it happen?”
• Conclusion: “what can we learn from this?”
• Recommendation: ”who can do what about it?”
NATO UNCLASSIFIED
An Observation - this is a comment based on something someone has
heard, seen or noticed that has been identified and documented as an
issue for improvement or a potential best practice.
9
WHEN AN OBSERVATION IS REQUIRED?
NATO UNCLASSIFIED
Key Factors…..
• Facts and Sequence of Events;
• Supporting Statistics;
• Involved Personnel;
• Levels of Experience, Training,
Qualification, Competence;
• Key Decisions etc.
Lack of …
• Communication;
• Training, Familiarity
with Doctrine, SOP
etc;
• Resources;
• Equipment;
• Preparation etc.
What was supposed to happen
Doctrine, Scenario,
SOPs, Cmd’s
Intent, etc.
10
“ODCR” FORMAT EXAMPLE
Observation Description: During Exercise Cooperative Archer it was observed that there is a
lack of medical equipment, individual protective equipment, training/qualification on such
equipment and training in aeromedical evacuation procedures. (What happened?)
Discussion: During the exercise it became apparent that there was no individual protective
equipment for personnel working on the flight line or on board aircraft, despite the fact that even
ERO (Engine Running Offload) operations were being conducted. No doctors were qualified as
flight surgeons, and other medical personnel had no specific training in the areas of flight
medicine. There was a lack of electro medical equipment for use on the aircraft. Ambulance
drivers were unfamiliar with flight line procedures such as the approach to aircraft. Overall there
was insufficient familiarity with, and provision of, acceptable levels of aeromedical evacuation
equipment during the exercise. (Why did it happen?)
Conclusion: There was insufficient equipment and training for personnel involved in the
aspects of aeromedical evacuation during Exercise Cooperative Archer. (What can we learn
from this?)
Recommendation: Investment should be made in individual protective equipment and training
for personnel involved in flight-line and in-flight procedures. Key medical personnel require
training in flight medicine, aeromedical evacuation and flight physiology. Electro medical
equipment (subject to airframe certification and airworthiness) should be obtained for use on
board aircraft, and personnel should be trained in its use. (What can we do about it?)
Observation Title: PfP Interoperability in NATO aeromedical evacuation exercises
NATO UNCLASSIFIED
What should we avoid when drafting Observations?
11NATO UNCLASSIFIED
• Avoid using Observations to air complaints;
• Avoid to prepare observations about some issues which can be quickly
resolved within departments;
• Make sure the issue is applicable.
• Make sure that the Description includes all the key facts.
• Avoid introducing new data in Conclusion.
• Ensure that your analysis allows you to describe the root causes, not the
symptoms. Remedial Action which solves the symptoms only, will not solve the
issue.
• Example: If you have a terrible headache from dehydration,
Aspirin ® will only fix your pain: it will not address
your dehydration.
12
NATO LL PROCESS (3)
L E S SON
I DENT I F I ED
Understand root causes
Propose: Remedial Actions to resolve the problem
Action Body to implement the Remedial Action.
ANALYSIS
Problem
Procedure
Good practice
OBSERVE
LLLIAnalysis Phase Remedial Action Phase
Commander’sGuidance
Improvedcapability
OBS
Post
Publish
Promote
DISSEMINATE
Command:Endorses Remedial ActionsCommits resourcesTasks an Action body
ENDORSE & TASK
L E S SON
L E ARNE D
AB: prepares Action Plan, implements RA, reports progress
Tasking authority: monitors progress
IMPLEMENT VALIDATE
Verify results
SHARESHARE
Validation
NATO UNCLASSIFIED
ANALYSIS – FROM OBSERVATION TO LESSONS IDENTIFIED
13NATO UNCLASSIFIED
What is Analysis?
According to NATO Doctrine AAP-6 (2014) analysis is:
“The study of a whole by examining its parts and their interactions.”
or …
“The application of critical and creative thinking to organizational problems”.
LIAnalysisOBS
14
Command Analysis – within the command is needed to:• understand easy problems
• Root cause and solution are obvious
• Problem is not frequent
• Problem and solution lie within the command
• identify potential best practices
• identify lessons from a familiar situation
Joint Analysis – involving external entities is needed to:• understand difficult problems
• Complex
• Repetitive
• Cross-Command
• understand/prove best practices
• understand new situations
Which Analysis Approach?
NATO UNCLASSIFIED
Key Features:
Quantity
Quick
Subjective
Cheap
Key Features:
Quality
In-Depth
Objective
Costly
Analysis techniques
15NATO UNCLASSIFIED
Problem
Root Cause
Solution
Brainstorming/Starbursting
16NATO UNCLASSIFIED
• Six Ws
• Who
• What
• When
• Where
• Why
• HoWObservation
• Used to generate questions that guide
analysis
• Helps to ensure Discussion and
provides necessary level of detail
Problem
Root Cause
Solution
Visualization/Flow Diagram
17
• Used to visualize the facts
• Can compare what was supposed to happen with what actually happened
• Can identify points in process where problems occurred
NATO UNCLASSIFIED
5 Times Why
18
Observation
Why?
Why?
Why?
Why?
Why?
The vehicle will not start. (the problem)
The battery is dead. (first why)
The alternator is not functioning. (second why)
The alternator belt has broken. (third why)
The belt was well beyond its useful service
life and not replaced. (fourth why)
The vehicle was not maintained according
to the recommended service schedule.
(fifth why, a root cause)
NATO UNCLASSIFIED
Problem
Root Cause
Solution
Want to know more?
19
www.mindtools.com
Part 2 of the Joint Analysis Handbook
Any Operational Analysis/Research
Methods text book
NATO UNCLASSIFIED
20
NATO LL PROCESS (4)
L E S SON
I DENT I F I ED
Understand root causes
Propose: Remedial Actions to resolve the problem
Action Body to implement the Remedial Action.
ANALYSIS
Problem
Procedure
Good practice
OBSERVE
LLLIAnalysis Phase Remedial Action Phase
Commander’sGuidance
Improvedcapability
OBS
Post
Publish
Promote
DISSEMINATE
Command:Endorses Remedial ActionsCommits resourcesTasks an Action body
ENDORSE & TASK
L E S SON
L E ARNE D
AB: prepares Action Plan, implements RA, reports progress
Tasking authority: monitors progress
IMPLEMENT VALIDATE
Verify results
SHARESHARE
Validation
NATO UNCLASSIFIED
21
L E S SON
I DENT I F I ED
Understand root causes
Propose: Remedial Actions to resolve the problem
Action Body to implement the Remedial Action.
ANALYSIS
Problem
Procedure
Good practice
OBSERVE
LLLIAnalysis Phase Remedial Action Phase
Commander’sGuidance
Improvedcapability
OBS
Post
Publish
Promote
DISSEMINATE
Command:Endorses Remedial ActionsCommits resourcesTasks an Action body
ENDORSE & TASK
L E S SON
L E ARNE D
AB: prepares Action Plan, implements RA, reports progress
Tasking authority: monitors progress
IMPLEMENT VALIDATE
SHARESHARE
ValidationNATO UNCLASSIFIED
And the result is.....
Verify results
A Lesson Learned
and improved capability!
22
NATO Lesson Learned
NATO UNCLASSIFIED
“Results from the implementation of
a remedial action that produced an
improved performance or increased
capability”(NATO LL Policy)
LESSONS LEARNED INFORMATION SHARING
23NATO UNCLASSIFIED
“There is no reason to send troops into the fight and get them killed when a Lesson
Learned the month before could be sent to a commander who could have used it for
training….”
General James N. Mattis USMC, Retired, US Secretary of Defense
WHAT IS IN PLACE?
24
NATO LL Policy (08 Sep 2011)
Bi-SC Directive 080-006 Lesson Learned (10 Jul 2013)
ACO Directive 080-001 Lesson Learned (08 Apr 2013)
NATO LL Organisation
LL Portal & LL Database (UNCLASS / NS)
LL Community of Interest
Joint Analysis Handbook & NATO LL Handbook
NATO LL Staff Officer Course (Mar / May / Nov )
JALLC Analyst Training Course (May / Oct )
Lessons Learned OPR Online Course (ADL )
Annual NATO LL Conference in Lisbon
Annual NATO COE LL Workshop
NATO UNCLASSIFIED
LL Portal & LL Database
25
http://www.jallc.nato.int
NATO UNCLASSIFIED
The Shark and Bite Fish
26
NATO UNCLASSIFIED
COL ALEKSANDAR GEORGIEV
ANALYSIS AND LESSONS LEARNED SECTION HEAD
CMDR COE
Office: +359 2 92 24731
