10/16/02copyright Brian Williams, 20021 courtesy of JPL Diagnosing Multiple Faults Brian C. Williams...

transcript

1copyright Brian Williams, 200210/16/02courtesy of JPL

Diagnosing Multiple Faults

Brian C. Williams16.412J/6.834JOctober 16th, 2002

2copyright Brian Williams, 200210/16/02

Outline

Properties of Diagnosis Diagnostic Agents Diagnosis using Conflicts Estimating Modes Finding the Most Likely

Conflict-directed A*

10/16/02 copyright Brian Williams, 2002 3

Failures may be hidden: STS-93Symptoms:

• Engine temp sensor high• LOX level low• GN&C detects low thrust• H2 level low (???)

Problem: Liquid hydrogen leakEffect:

• LH2 used to cool engine• Engine runs hot• Consumes more LOX

Failures may be detected by qualitative arguments

• Mars Observer

courtesy of JPL

Failures are Often Novel

Multiple Faults Occur

courtesy of NASA

• Quintuple fault occurs (three shorts, tank-line and pressure jacket burst, panel flies off).

• Power limitations too severe to perform new mission.

• Novel reconfiguration identified, exploiting LEM batteries for power.

APOLLO 13

• Mars Observer• . . . .• Mars Climate Orbiter• Mars Polar Lander• Deep Space 2

courtesy of JPL

Failures Are Subtle,Frequent and Remote

Mars Polar Lander Diagnosis:

• Legs deployed during descent.

• Noise spike on leg sensors latched by software monitors.

• Laser altimeter registers 50ft.

• Begins polling leg monitors to determine touch down.

• Latched noise spike read as touchdown.

• Engine shutdown at ~50ft.

Failures Involve Complex Software/Hardware Interactions

Candidate: Assignment to all component modes.

10Adder(i): G(i):

Out(i) = In1(i)+In2(i) U(i): 12

Diagnosis identifies consistent modes

Candidate = {G(A1) & G(A2) & G(M1) & G(M2) & G(M3)}

Given Model and Observations:

Diagnosis identifies consistent modes

Adder(i): G(i):

Out(i) = In1(i)+In2(i) U(i):

Candidate: Assignment to all component modes. Diagnosis D: Candidate consistent with model Phi and

observables OBS.

Diagnosis = {G(A1) & U(A2) & G(M1) & G(M2) & G(M3)}

Given Model and Observations:

Outline

senseP(s)

observations actions

AGENT Diagnostic Agent:

• Monitors & Diagnoses

• Repairs & Avoids

• Probes and Tests

Consistency-based From Symptoms

Goal: Support programmers with Embedded Languages thatmonitor, diagnose and repair

(Model-based Programming)

Programming Diagnostic Agents

System Model

Control Program

RMPL Model-based Program

Control Sequencer

Deductive Controller

CommandsObservations

Titan Model-based Executive

State goalsState estimates

ModeEstimation

ModeReconfiguration

Trackslikely

plant states

Tracks least cost goal states

Executes concurrently Preempts Queries (hidden) states Asserts (hidden) state

ClosedClosed

ValveValveOpenOpen StuckStuck

openopen

StuckStuckclosedclosed

OpenOpen CloseClose

0. 010. 01

0.010.01

inflow = outflow = 0

Generates target goal statesconditioned on state estimates

Example: The model-based program sets engine = thrusting, and the deductive controller . . .

Determines that valveson the backup engine

will achieve thrust, andplans needed actions.

Deduces that a valve failed - stuck closed

Plans actionsto open

six valves

Fuel tankFuel tankOxidizer tankOxidizer tank

Deduces thatthrust is off, and

the engine is healthy

Outline

Representing Inconsistent Candidates Compactly

Conflict

• A set of component modes M that are inconsistent with the model and observations.

• Every subset of a conflict is a conflict

• Only need conflicts that are minimal under subset

• Logically, not M is an implicate of model & Obs

Kernel Diagnosis

= {U(A2) & U(M2)}

Representing Diagnoses Compactly

Partial Diagnosis: A set of component modes M all of whose extensions are diagnoses.

• M removes all symptoms

• M entails Model & Obs (implicant)

Kernel Diagnosis: A minimal partial diagnosis K

• M is a prime implicant of model & obs

Using Conflicts to Divide and Conquer

Given model Phi and observations OBS1. Find all symptoms and

diagnose each symptom separately

2. Merge diagnoses

General Diagnostic Engine[de Kleer & Williams, 87]

Using Conflicts to Divide and Conquer

Given model Phi and observations OBS1. Find all symptoms and

diagnose each symptom separately find minimal conflicts and constituent diagnoses

2. Merge diagnoses find kernel diagnoses

General Diagnostic Engine[de Kleer & Williams, 87]

1. Find Symptom

Symptom: F is observed 10, but should be 12 if M1 & M2 & A1 are okay.

1. … and its Conflict

Symptom: F is observed 10, but should be 12 if A1, M1& M2 are okay.

Conflict: G(A1) & G(M1) & G(M2) is inconsistent

Symptom: F is observed 10, but should be 12

Conflict: G(A1) & G(M1) & G(M2) is inconsistent

U(A1) or U(M1) or U(M2) removes conflict.

1. Find Another Symptom

Symptom: G is observed 12, but should be 10 ...

Symptom: G is observed 12, but should be 10

Conflict: G(A1) & G(M2) & G(M1) & G(M3) is inconsistent

U(A1) or U(A2) or U(M1) or U(M3) removes conflict

Conflict not just upstream from symptom

Kernel Diagnoses =

2. Merge Diagnoses

U(A1) or U(M1) or U(M2) remove conflict 1.

U(A1) or U(A2) or U(M1) or U(M3) remove conflict 2

“Smallest” sets of modes that remove all conflicts

Constituent Kernel Diagnoses

Kernel Diagnoses = {U(A1)}

3. Merge Diagnoses

Kernel Diagnoses = {U(M1)}{U(A1)}

3. Merge Diagnoses

Kernel Diagnoses = {U(A2) & U(M2)} {U(M1)}{U(A1)}

Kernel Diagnoses = {U(M2) & U(M3)}{U(A2) & U(M2)}{U(M1)}{U(A1)}

3. Merge Diagnoses

Worst Case Complexity

Given model Phi and observations OBS Generate all minimal conflicts

An instance of Implicate generation - NP Hard

Generate kernel diagnosis from conflicts An instance of Implicant generation - NP Hard

Performance is a pragmatic question: How large is the set of conflicts and kernels? Is it faster to go through conflicts,

than to go directly to the kernels?

Outline

Diagnosis With Only the Unknown

Inverter(i): G(i): Out(i) = not(In(i)) U(i):

X YA B C0 00 0

Nominal and Unknown Modes

• Isolates surprises• Doesn’t explain

Diagnosis With Only the Known

Inverter(i): G(i): Out(i) = not(In(i)) S1(i): Out(i) = 1 S0(i): Out(i) = 0

X YA B C0 00 0

Exhaustive Fault Modes

• No surprises• Explains

Solution: Diagnosis as Estimating Behavior Modes

Inverter(i): G(i): Out(i) = not(In(i)) S1(i): Out(i) = 1 S0(i): Out(i) = 0 U(i):

X YA B C0 00 0

Nominal, Fault and Unknown Modes

• Isolates surprises• Explains

Example Diagnoses

X YA B C0 01

Diagnosis: [S1(A),G(B),U(C)]

Sherlock[de Kleer & Williams, 89]

Example Diagnoses

X YA B C0 01

Diagnosis: [S1(A),G(B),U(C)]

Kernel Diagnosis: [U(C)]

X YA B C0 0??

Sherlock[de Kleer & Williams, 89]

1. Find Symptoms & Conflicts

Conflict:

not G(A), G(B) and G(C)

X YA B C0 0

1 0G G

More Symptoms & Conflicts

Not S1(A), G(B), and G(C)

X YA B C0 0

1 0S1 G

not S0(B) and G(C)

X YA B C0 0

not S1(C)

X YA B C0 0

All Conflicts

< S1(C) >

< S0(B), G(C) >

< S1(A), G(B), G(C) >

< G(A), G(B), G(C) >

2. Constituent Diagnoses from Conflicts

< S1(C) >=> G(C),S0(C) or U(C)

< S0(B), G(C) >

=> G(B),S1(B),U(B),S1(C),S0(C) or U(C)

< S1(A), G(B), G(C) >

=> G(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C) or U(C)

< G(A), G(B), G(C) >

=> S1(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C) or U(C)

3. Generate Kernel Diagnoses

[U(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]

[G(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C),U(C)]

[S1(A),S0(A),U(A),S1(B),S0(B),U(B),S1(C),S0(C),U(C)]

[U(C)] [S0(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]

3. Generating Kernel Diagnoses

[U(C)] [S0(C)]

[U(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]

[U(C)] [S0(C)]

[U(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]

[S1(B),G(C)]

[U(C)] [S0(C)]

[U(B),G(C]

[S1(B),G(C)]

[U(A),G(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]

[U(C)] [S0(C)]

[U(B),G(C]

[S1(B),G(C)]

[U(A),G(B),G(C)]

[S0(A),G(B),G(C)]

[G(C),S0(C),U(C)]

[G(B),S1(B),U(B),S1(C),S0(C),U(C)]

3. Generate Kernel Diagnoses

Outline

Computing Likelihood Conflict-directed A*

Diagnoses: (42 of 64 candidates)

Fault Models [G(A),G(B),S0(C)] [G(A),S1(B),S0(C)] [S0(A),G(B),G(C)]

. . . Fault Free [G(A),G(B),U(C)] [G(A),U(B),G(C)] [U(A),G(B),G(C)]

Partial Diagnoses [G(A),U(B),S0(C)] [U(A),S1(B),G(C)] [S0(A),U(B),G(C)]

X YA B C0 00 0

Changes to Event Space

Candidates withUNKNOWN failure modes

Candidates withKNOWN failure modes

Good Good

Candidate Initial (prior) Probabilities

p(c) p(m)mc

p(G) .99 .99 .99

p(S1) .008 .008 .001

p(S0) .001 .001 .008

p(U) .001 .001 .001

p([G(A),G(B),G(C)]) = .97

p([S1(A),G(B),G(C)]) = .008

p([S1(A),G(B),S0(C)]) = .00006

p([S1(A),S1(B),S0(C)]) = .0000005

Assume FailureIndependence

00.20.40.60.8

S1(A),

X YA B C0 00

Posterior Probability, after Observation x = v

P(x=v|c) estimated using Model:

If previous obs, c and Phi entails x = vThen p(x = v | c) = 1

If previous obs, c and Phi entails x <> vThen p(x = v | c) = 0

If Phi consistent with all values for x Then p(x = v | c) is based on priors E.g., uniform prior = 1/m for m possible values of x

p(c | xv) p(x v | c)p(c)

p(x v)Bayes’Rule

Normalization Term

Observe out = 1: C = [G(A),G(B),G(C)] P(C) = .97 P(out = 1 | C) = ? = 1 P(C | out = 0 ) = ? = .97/p(x=v)

p(x v)

X YA B C 10

Observe out = 0: C = [G(A),G(B),G(C)] P(C) = .97 P(out = 0 | C) = ? = 0 P(C | out = 0 ) = ? = 0 x .97/p(x=v) = 0

p(x v)

X YA B C 00

X YA B C0

p(S1) .008 .008 .001

p(S0) .001 .001 .008

p(U) .001 .001 .001

How do the single faults change?• which are eliminated?• which predict observations?• Which are agnostic?

Priors for Single Fault Diagnoses:

00.20.40.60.8

S1(A),

X YA B C0 00

S1(A)S

X YA B C0 00 0

Top 6 of 64 = 98.6% of P

Outline

When you have eliminated the impossible, whatever remains, however improbable, must be the truth.

- Sherlock Holmes.

The Sign of the Four.

Exploring the Improbable

Focusing on Probable Candidates: Conflict-directed A*

GenerateCandidate

TestCandidate

Consistent?Keep

ComputePosterior p

BelowThreshold?

RecordConflict

DoneYes No

Yes No

Leading CandidateBased on Priors

IncreasingCost

Feasible

Infeasible

IncreasingCost

Feasible

Infeasible

IncreasingCost

Feasible

InfeasibleConflict 1

IncreasingCost

Feasible

IncreasingCost

Feasible

Conflict 1

IncreasingCost

Feasible

Conflict 1

IncreasingCost

Feasible

Infeasible

Conflict 3

Conflict 2

Conflict 1

IncreasingCost

Feasible

Infeasible

Conflict 3

Conflict 2

Conflict 1

• Feasible regions described by the implicants of known conflicts (Kernel Assignments)

•Want kernel assignment containing the best cost candidate

Given Current Conflicts: Generate best-cost kernel of conflicts Extend best kernel to best candidate Test consistency

Failure: new-conflict.

Assume Independent failures:

PG(mi) >> PU(mi)

Psingle >> Pdouble

PU(M2) > PU(M1) > PU(M3) > PU(A1) > PU(A2)

Example: Diagnosis

Conflicts / Constituent Diagnoses none

Best Kernel: {}

Best Candidate: ?

First Iteration

?(M1) & ?(M2) & ?(M3) & ?(A1) & ?(A2)

G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

Select most likely value for unassigned modes

Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

Extract Conflict and Constituent Diagnoses:

Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

Not [G(M2) & G(M1) & G(A1)]

Test: G(M1) & G(M2) & G(M3) & G(A1) & G(A2)

Not [G(M2) & G(M1) & G(A1)]

U(M2) or U(M1) or U(A1)

Conflicts / Constituent Diagnoses U(M2) or U(M1) or U(A1)

Best Kernel: U(M2)

Best Candidate: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

Second Iteration

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

Extract Conflict:

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

Extract Conflict:

Not [G(M1) & G(M3) & G(A1) & G(A2)]

Test: G(M1) & U(M2) & G(M3) & G(A1) & G(A2)

Extract Conflict:

Not G(M1) & G(M3) & G(A1) & G(A2)

U(M1) or U(M3) or U(A1) or U(A2)

Conflicts / Constituent Diagnoses U(M2) or U(M1) or U(A1)

U(M2) or U(M3) or U(A1) or U(A2)

Best Kernel: U(M1)

Best Candidate: U(M1) & U(M2) & G(M3) & G(A1) & G(A2)

Second Iteration3

Test: U(M1) & G(M2) & G(M3) & G(A1) & G(A2)

Consistent!

U(A2) U(M1)

U(M3)U(A1)U(A1), U(A2), U(M1), U(M3)

U(A1) U(M1) U(M2)

U(A1) U(M1) U(M1) & U(A2) U(M2) & U(M3)

Conflict-Directed A*: Generating Best Kernels

U(A1), U(M1) , U(M2)

Constituent Diagnoses

• Kernel assignments found by minimal set covering

• To find the best kernel, expand tree in best first order

Outline

Research in Model-based Diagnosis

Methods exist for:

• Focusing on likely diagnoses

• Active probing

• Diagnosing dynamic systems and monitoring behavior

• Repairing and compensating for failures

• Diagnosing hybrid discrete/continuous systems

• Performing distributed diagnosis

10/16/02copyright Brian Williams, 20021 courtesy of JPL Diagnosing Multiple Faults Brian C. Williams...

Documents