Smart Embedded Systems Group University of Osnabrück

- 1 -

Uncertainty and Trust Estimation inIncrementally Learning Function Approximation

Andreas Buschermöhle, Jan Schoenke and Werner Brockmann

Institute of Computer ScienceSmart Embedded Systems Group

Catania, July 9th 2012

Smart Embedded Systems Group University of Osnabrück

- 2 -

Overview

Motivation

Uncertainties in Learning Function Approximation

Uncertainty Estimation

Results

Conclusion and Summary

Smart Embedded Systems Group University of Osnabrück

- 3 -

Motivation

• Upcoming tasks are of increased complexity

Operation in natural and changing environments

Not every situation can be planned in advance

Unforeseeable interaction behavior

• Disturbances influence the system dynamically

Sensor faults, Actuator faults

Unobserved influences

…

Changing behavior Incremental learning

Disturbances Uncertainties

Safe operation always crucial

?

Smart Embedded Systems Group University of Osnabrück

- 4 -

Uncertainty in Learning Function Approximation

• Five sources influence incrementally learning function approximation

i. Uncertain training input data

ii. Uncertain training output data

iii. Varying target function / unobserved variables

iv. Expressiveness of the approximator

v. Sparsity of training data

• At the output this results in two effects

1. No data is available to evaluatethe approximation (ignorance)

2. Multiple output values are possible,i.e. the target output varies (conflict)

i

ii

iii

iv

v

Smart Embedded Systems Group University of Osnabrück

- 5 -

Related Work

• Gaussian processes [Rasm2006], [Dall2009]

Local data density reflected in output variance

Separate input variance possible

• Evidence theory [Deno1997], [Peti2004]

Local data density reflected in output bounds

Bound for training outputs

• RBF networks [Leon1992]

Local data density by counting for each RBF

Approximation uncertainty by cross-validation

v

v

ii

v

iv

Every approach is dataset-based

Not all sources of uncertainty are covered

i

Smart Embedded Systems Group University of Osnabrück

- 6 -

Approach

• Different measures to observe the influence

Mapped to trust signals

no trustworthiness

1 full trustworthiness

Fuzzy operators for fusion of trust signals

• Application to a zero-order Takagi-Sugeno fuzzy system

Local linear interpolation

• Learning by normalized gradient descent

Smart Embedded Systems Group University of Osnabrück

- 7 -

Approach

• Local Measures:

Frequency of learning stimuli

Activity sum of learning stimuli

Mean node adaptation

Mean absolute node adaptation

Direct incremental rating of node adaptation

Smart Embedded Systems Group University of Osnabrück

- 8 -

Evaluation

• Two measures to investigate the performance

Trust-weighted MSE Mean trust level

• Two exemplary target functions

• Simulated disturbances for cases i-iv

i. Normally distributed noise on training input

ii. Normally distributed noise on training output

iii. Additive disturbance of with unobserved input

iv. Lowered number of fuzzy rules for approximation

v. Implicitly covered in progress of learning

i

ii

iii

iv

v

∑𝑗=1

𝑁𝑡 𝜗 (𝑥 𝑗 )𝑁𝑡

( 𝑓 (𝑥 𝑗 )−~𝑓 (𝑥 𝑗)) ² ∑

𝑗=1

𝑁𝑡 𝜗 (𝑥 𝑗 )𝑁𝑡

Smart Embedded Systems Group University of Osnabrück

- 9 -

Resultsno disturbances

Smart Embedded Systems Group University of Osnabrück

- 10 -

Resultstraining input noisei

Smart Embedded Systems Group University of Osnabrück

- 11 -

Resultstraining output noiseii

Smart Embedded Systems Group University of Osnabrück

- 12 -

Resultsunobserved variableiii

Smart Embedded Systems Group University of Osnabrück

- 13 -

Resultsexpressiveness of the approximatoriv

Smart Embedded Systems Group University of Osnabrück

- 14 -

Resultsall sources of uncertaintyvi

Smart Embedded Systems Group University of Osnabrück

- 15 -

Conclusions

• It is possible to estimate all influences of uncertainty on-line

• Different measures account for different influences

Activity sum of learning stimuli () ignorance

Mean absolute adaptation () long term conflict

Direct incremental rating of adapt. () short term conflict

• Combination of these through trust management

redundantnon

redundant

Good combined results for all influences of uncertainty

Smart Embedded Systems Group University of Osnabrück

- 16 -

Combined Uncertainty Measureall sources of uncertaintyvi

Smart Embedded Systems Group University of Osnabrück

- 17 -

Summary

Integral view on all sources of uncertainty inincrementally learning function approximation

Measuring ignorance and conflict combined through trust management

Combined trust signal represents• all uncertainties at the output• locally• incrementally• gradually

Smart Embedded Systems Group University of Osnabrück

- 18 -

Smart Embedded Systems Group University of Osnabrück

- 19 -

References

[Dall2009] Dallaire, P., Besse, C., Chaib-Draa, B.: Learning Gaussian Process Models from Uncertain Data. In: Proc. Int. Conf. on Neural Information Processing, pp. 433-440. Springer, Bangkok (2009)

[Deno1997] Denoeux, T.: Function Approximation in the Framework of Evidence Theory: A Connectionist Approach. In: Proc. Int. Conf. on Neural Networks, pp. 199-203. IEEE Press, Houston (1997)

[Leon1992] Leonard, J.A., Kramer, M.A., Ungar, L.H.: Using Radial Basis Functions to Approximate a Function and its Error Bounds. IEEE Transactions on Neural Networks, vol. 3, no. 4, 624-627. IEEE Press (1992)

[Peti2004] Petit-Renaud, S., Denoeux, T.: Nonparametric Regression Analysis of Uncertain and Imprecise Data Using Belief Functions. In: Int. J. of Approximate Reasoning 35, pp. 1-28. Elsevier (2004)

[Rasm2006] Rasmussen, C.E.,Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)