Safety Critical Design -

Understand the Capabilities and Behaviors

of the Conscious and Automatic Minds

-- Houston HFES Symposium --

.

April 26. 2019

Tom Shephard CAP, PMP

Human Cognition

• Autonomic (amygdala) Ex: fight/flight response

• Conscious processes aka, working memory/attention, System 2

• Sensory receptors and pre-processing

• Automatic processes aka, unconscious, System 1

…..the collective product of interdependent and very different subsystems:

Cognitive Subsystem Reaction Times *

• Autonomic: Very Fast 20 milliseconds (ms) (Edwards 2005)

• Conscious mind: Slow 285 ms (Carter 2014 p 121)

• Automatic mind: Fast 70-150 ms (Sträter 2005, p85,128, Carter 2014 p 121)

Reaction times and the profound differences in subsystems enable a wide range of situation-specific responses.

* 1 perception cycle, indicative times

Mental Models (MMs) – Stored knowledge of things/systems and how they work, prototype representations (schemata) and associated action sequences (scripts).

(Reason 2008 p 34, Endsley 2012 p 21-3, Woods et al 2010 p104--111)

Terms and Abbreviations

Attention – Conscious/Automatic controlled executive function that directs WM to focus on an object, internal or external. Attention can be directed to one object at a time, only! (Reason 2008 p 42, Woods et al 2010 p 115-6)

Working Memory (WM) – cognitive resources that enable conscious processing, and constrained by short-term memory that has a very limited store capacity and duration. (Reason 1990 p 32-3, Colom et al 2008)

LTM – Long term memories, e.g., semantic, episodic, MMs

AUTOMATIC PROCESS CONSCIOUS PROCESS

Normal operation

Continuous, automatic, effortless (Kahneman 2011 p20, 105)

Perceptions and responses shaped by the past (LTM/MM). (Reason 2008 p17)

A recognition engine, continuously compares input stimulus to LTM/MMs seeking a match. (Reason 2008 p18-9)

If a match, auto selects the schema and associated action response. If not, triggers a conscious process to resolve. (Endsley 2012 p21-4, Kahneman 2011 p24)

Active actor ~ 5% of the time (Mlodinow 2012 p34, )

Runs concurrent to automatic processes. (Reason 1990, p132-4)

Serial, step-wise processing cycles.

Highly effortful. ‘Lazy’ tendencies. (Kahneman 2011, p21, Ch3)

Default is a least effort mode if not explicitly tasked. (Kahneman 2011 p24, Mlodinow 2012 p34)

AUTOMATIC PROCESS CONSCIOUS PROCESS

Observability

None, hidden (Mlodinow 2012)

Process/product perceived as an intuition or ‘gut feel’…degrades when unhappy or uncomfortable. (Kahneman 2011 p 11, 69)

Mostly. Visibility into the object of one’s directed attention, decisions, etc. (Reason 2008, p 12)

Hidden – effects of priming, goal conflicts and beliefs, confirmation bias, task switch errors, etc. (Kahneman 2011 p24,103, Reaso n 1990 p11-2, Wickens 2015)

Decision capability

None

Analytical capability Limited, e.g., ability to estimate averages but not sums. No statistical capability. (Kahneman 2011 p 92-3)

Yes. Powerful decision and analytical capability

Caveat – results affected by biases, WM errors, faulty mental models, inappropriate short-cuts, etc. (Kahneman 2011)

AUTOMATIC PROCESS CONSCIOUS PROCESS

Ability to detect risk

None (Sylvestre 2017 p66-73)

Yes, if activated and tasked (Sylvestre 2017 p66-73)

Ability to detect danger

Yes - fast, continuous, automatic (Sylvestre 2017 p66-73)

Danger reflects prior experience.

Limited. Effortful, duration limited (Reason 2008 p18)

May be interrupted / suppressed if sudden danger detected. (Kahneman p 35, Sylvestre 2017 p66-73)

Validate response before acting

No Impulsive behavior

Recognition product is all there is (past experience) (Kahneman 2011 p 85-6)

Yes, with intended effort. Else, default validity checks are: (Kahneman 2011 p 46, 80-6, 99, 105)

- Superficial effort to validate input data or planned actions - No effort to see if key data is missing - Seeks confirming information only (confirmation bias)

AUTOMATIC PROCESS CONSCIOUS PROCESS

Direct LTM/MM access

Yes

Retrieval process: Initially seeks a similarity match to memory call criteria. If several possibilities, selects the most frequently used. (Sträter 2005, p110, Reason 1990 p98, 130-147, 2008 p12 -25)

No

Access capabilities limited to: (Reason 1990 p35, 131, 135, 144 , 2008 p12, Sträter 2005 p85, 119)

- Define memory call criteria - View the product of retrieval process - Assess/accept/reject retrieval product

Ability to self-monitor/correct

None *

* The next cognitive cycle may trigger a different response if inputs change. (Kahneman 2011 41-2, 105)

Yes, if activated and not overtasked

Provides the only means to change attentional focus, behavior, beliefs, goals, work pace or emotions. (Kahneman 2011, p24-5, 41, 103, Sträter 2005, p119)

AUTOMATIC PROCESS CONSCIOUS PROCESS

Access to all sensory data

Yes, all senses

Data rate: >11 million bits/second (BPS)

Pre-processed data: Yes, all senses. (Reason 2008, 12)

Max input data rate: 16-50 BPS (Mlodinow 2012, p33)

Focused attention can cause change blindness, e.g., invisible gorilla. (Kahneman 2011 p23)

Span of control Once ‘learned’ a skill or habit is an automatic process requiring very brief periods of attention, e.g., < 25 ms.

Every new skill or habit begins as a consciously controlled activity. (Reason 2008 p 13-14, Kahneman 2011 p 35)

With repetition, skills and habits become automatic processes in 2-6 months. Before that, a sliding mix of conscious/automatic processes.

(Reason 2008 p13-4)

AUTOMATIC PROCESS CONSCIOUS PROCESS

Time tracking:

Track event sequence: Yes

Ability to track clock time: None Time distortion can occur with changes in mood, age, situation, etc.

Elapsed time: Limited. With experience one’s MM provides the ‘gist’ on when a future event may occur, e.g. fast or slow. This improves with increasing expertise.

Event sequence: Yes, subject to WM and Attention-related issues *

Clock time: Limited. Can accurately track clock time for periods to 30 seconds, then progressively less reliable.

Elapsed time: See automatic process

*Time tracking consumes and fully relies on WM/Attention…limited resources negatively affected by internal and external factors.

Humans are not reliable clocks!

The human response to a safety-critical task or event depends on which mind responds….

…..conscious or automatic…

..each provides different answers and responses.

This concludes the presentation.

Thank You!

For further detail and application see:

- A Cognitive Approach to Safety Critical Task Design (Poster)

- Introduction and Advanced Workshops: Apply Situation Awareness

and Cognitive Science to Safety Critical Functions

http://shephardpost.com

Author Bio – Tom Shephard CAP, PMP

• A passionate seeker of best practice tools & methods

• 37 years - Operating and engineering companies: O&G, refining, midstream, terminals and pipeline

• Technical safety department: management and member

• Functional safety lead on many projects

• Automation project manager / project Lead

• Corporate standards and practice development

• Certified Automation Professional (CAP)

• Certified Project Management Professional (PMP)

• A lifetime of hands-on safety work. All project phases and activities.

Recently retired: Wood Group Mustang

References

Carter, R., Aldridge, S., Page, M., Parker, S., 2014, The Human Brain Book, 2nd Ed, DK Publishing, New York

Colom, R. Abad, F.J., Quiroga, M.A., Shih, P., Flores-Medoza, C., 2008, working memory and intelligence are highly related constructs, but why?, Intelligence 36 (2008) 584-606

Edwards, S.P., 2005, The amygdala: the body’s alarm circuit, Brainwork, May 2005, Dana foundation (dana.org)

Endsley, M.R., Jones, D.G., 2012. Designing for situation awareness: An approach to user-centered design, 2nd Edition, CRC Press

Kahneman, Daniel, 2011, Thinking, Fast and Slow, Farrar, Straus and Giroux

Mlodinow, Leonard, 2012, Subliminal: How Your Unconscious Mind Rules Your Behavior, Vintage Books (Div of Random House Inc.), 1st Edition

Reason, J., 1990. Human Error, Cambridge: Cambridge University Press

Reason, James, 2008. The Human Contribution, Unsafe Acts, Accidents and Heroic Recoveries, Ashgate Publishing Ltd,

Sanders. Mark S., McCormick, Ernest J, 1993, Human Factors in Engineering and Design, McGraw-Hill Inc, 7th Ed

Sträter, O., 2005. Cognition and safety: an integrated approach to systems design and assessment, Ashgate Publishing Ltd,

1st Ed

Sylvestre, Christian, 2017, Third Generation Safety: The Missing Piece, ISBN 978-0-648 1200-0-1, National Library of

Australia Cataloging-in-Publication entry

Wickens, C. D., et.al., (2015), Discrete task switching in overload; a meta-analyses and a model, Int. J. Human-Computer

Studies (2015), http://dx.doi.org/10.1016/j.ijhcs.2015.01.002

Woods, D.D., Dekker, S., Cook, R., Johannsen, L., Sarter, N., 2010. Behind Human Error, Ashgate Publishing, 2nd Ed