asArchitecture des Systèmes d’Information

Bayesian Networks and Statistical Learning

Applications to complex system modellingand diagnosis

Philippe LERAY, Olivier François, Ahmad Faourcontact: Philippe.Leray@insa-rouen.fr

PSI (Perception, Systems and Information) Laboratory – FRE CNRS 2645

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 1/12

asArchitecture des Systèmes d’Information

Structural learning – complete data

The DAG space has a super-exponential size → heuristics !

Constraint based methods (IC, PC, BN-PC. . .)Score based methods

complete search in Tree space (MWST)greedy search in DAG space, with node ordering (K2)or without (GS)greedy search and Markov equivalence (GES)

Conferences : François & Leray RJCIA 03 (french), RFIA 04 (french)

Journal : JEDAI 04 (french)

MWST = good performances vs. computation timeMWST for GS initialisation = robust initialisation

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 2/12

asArchitecture des Systèmes d’Information

Structural learning – incomplete data

Few methods deal with incomplete data

Usual principle = applying EM to score based methodsgreedy search in DAG space (SEM = GS+EM)

Conference : François & Leray EGC05 (french) [subm. to ECSQARU 05] :

MWST+EM = MWST + score estimation with EMMWST+EM for SEM initialisation = robust initialisation

Perspectives :

greedy search and Markov equivalence = GES+EMconstraint based methods and incomplete data

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 3/12

asArchitecture des Systèmes d’Information

Structural learning – latent variables

Combinatorial explosion

Where are the latent variables in the DAG ? Cardinality ?→ new operators in SEM→ space restriction : hierarchical latent class model (HLC)

Conference : Leray & al. ECML03 Workshop (PGM for classification)

Tree augmented HLC

Perspectives :

SEM+EM = dealing with incomplete data and latent va-riable discovery

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 4/12

asArchitecture des Systèmes d’Information

Structural learning – a priori knowledge

Using a priori knowledge to simplify the search space

Perspectives :

Dynamic bayesian networks (2TBN) = 2 structures :intra-slice (t) and inter-slice (t → t + 1)

Oriented object bayesian networks (OOBN), Multi-agentbayesian networks, ...

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 5/12

asArchitecture des Systèmes d’Information

Complex system modelling and diagnosis

Discovering handwriting strategies of primary schoolchildren

I. Zaarour PhD thesis (completed in feb. 2004)Collaboration with a psychology lab (PSY.CO Rouen)

Conferences : ECML03 Worshop - IGS03 - RFIA04 (french)

Journal : IJPRAI 04

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 6/12

asArchitecture des Systèmes d’Information

Complex system modelling and diagnosis

Intrusion detection in computer networks

A. Faour PhD thesis (begin sept. 2004)Collaboration with a network security expert

Conferences : EGC05 Worshop (french)

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 7/12

asArchitecture des Systèmes d’Information

Complex system modelling and diagnosis

Bayesian networks for classificationO. François PhD thesis (end envisaged in dec. 2005)

Journal : RIA 2004 (french)

Dysfunction detection and localisation in a chemicalreactor

Collaboration with a chemical process engineering lab (LRCP

Rouen)

Conference : SFGP 2005 (french)

Micro-wave transistor thermical modellingProject with Thales Air Defense and a aero-thermochemistry lab

(CORIA Rouen) financed by Haute-Normandie Region

G. Mallet MSc thesis (feb-july 2005) followed by a PhD

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 8/12

asArchitecture des Systèmes d’Information

Scientific animation

French workshop on bayesian networks :June 2001 – first workshop, Paris (co-organisation)March 2003 – second workshop, Rouen.Jan. 2005 – French PGM workshop during EGC 2005conference, Paris.

Software : BNT Toolbox for Matlabcode contributionsresponsable for structure learning packagefrench BNT website and documentationhttp ://bnt.insa-rouen.fr

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 9/12

asArchitecture des Systèmes d’Information

International activities

Members of PASCAL (Pattern Analysis, StatisticalModelling and Computational Learning) europeannetwork of excellence

CollaborationsCausal networks and structural learning – S. Meganck& B. Manderick, Computational Modeling Lab, VrijeUniversiteit Brussel (VUB), Belgium.

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 10/12

asArchitecture des Systèmes d’Information

Selected bibliography (in english)

http ://asi.insa-rouen.fr/˜pleray/publisRB.php

International journals :

Zaarour, I. et al. (2004). Clustering and bayesian network approaches for

discovering handwriting strategies of primary school children. International Journal

of Pattern Recognition and Artificial Intelligence, 18(7) :1233-1251.

International conferences :

Leray, P.et al. (2003). A bayesian model for discovering handwriting strategies of

primary school children. In Working Notes of the Workshop on Probabilistic

Graphical Models for Classification, ECML/PKDD-2003, 49-57.

Zaarour, I.et al. (2003). A bayesian network model for discovering handwriting

strategies of primary school children. In 11th Conference of the International

Graphonomics society (IGS 2003), 178-181.

Misc :

Leray, P. and Francois, O. (2004). BNT structure learning package : Documentation

and experiments. Technical report, Laboratoire PSI.

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 11/12

asArchitecture des Systèmes d’Information

Selected bibliography (in french)

Books :

Naïm, P., Wuillemin, P.-H., Leray, P., Pourret, O., and Becker, A. (2004). Réseaux

bayésiens. Eyrolles, Paris.

French journals :

Leray, P. and Francois, O. (2004). Réseaux bayésiens pour la classification -

méthodologie et illustration dans le cadre du diagnostic médical. Revue

d’Intelligence Artificielle, 18/2004 :169-193.

François, O. and Leray, P. (2004). Etude comparative d’algorithmes

d’apprentissage de structure dans les réseaux bayésiens. Journal électronique

d’intelligence artificielle, 5(39) :1-19.

French conferences :

Francois, O. and Leray, P. (2005). Apprentissage de structure dans les réseaux

bayésiens et données incomplètes. In Proceedings of EGC 2005 (to appear), 1-6.

Faour, A. and Leray, P. (2005). Réseaux bayésiens pour le filtrage d’alarmes dans

les systèmes de détection d’intrusion. In Proceedings of EGC 2005 Atelier

Modèles graphiques probabilistes (to appear), 1-8.

Francois, O. and Leray, P. (2004). Evaluation d’algorithmes d’apprentissage de

structure pour les réseaux bayésiens. In Proceedings of 14ème Congrès

Francophone Reconnaissance des Formes et Intelligence Artificielle, RFIA 2004,

1453-1460.

Bayesian Networks and Statistical Learning Applications to complex system modelling and diagnosis – p. 12/12