Information Modeling

Fast forward

Five posters

− Four introduced here

− One on Haptics project introduced by Jonas Unger

Three additional papers

Look-ahead Control

It is shown that an energy formulation allows

− simple integration routines

− interpolation on coarse grids

A residual cost is proposed

− derived from engine and driveline characteristics

Proper physical model of gear shift

Hellström, E., Åslund, J., and Nielsen, L. Design of an efficient algorithm for on-board look-ahead control. Solicited for Control Engineering Practice.

Look-ahead Control

In many modern engines the fuel map is non-linear due to the different control strategies that are used today

Different formulations of the fuel minimization problem are studied in this paper and consequences of the non-linearities are analyzed

Ivarsson , M., Åslund, J., and Nielsen, L. Optimal Speed on Small Gradients –Consequences of a Non-Linear Fuel MapIFAC World

Regression on ManifoldsFrom Shells to Temperature

14 M

ay-0

9

Chemical compositionmeasurements

Reconstructed water temperature

Shells from an archaeological site

• Henrik Ohlsson and Lennart Ljung. Gray-Box Identification for High-Dimensional Manifold Constrained Regression. In the15th IFAC Symposium on System Identification, SYSID 2009.• M. Bauwens , H. Ohlsson, K. Barbe, V. Beelaerts, F. Dehairs and J. Schoukens. On Climate Reconstruction using Bivalve Shells: Three Methods to interpret the Chemical Signature of a Shell, In

the7th IFAC Symposium on Modelling and Control in Biomedical Systems. 2009.• M. Bauwens, H.Ohlsson, K. Barbe, V. Beelaerts, J. Schoukens and F. Dehairs. Three Ways to do Temperature Reconstruction Based on Bivalve-Proxy Information. In the 28th Benelux Meeting on

Systems and Control, 2009.• Henrik Ohlsson, Jacob Roll and Lennart Ljung. Manifold-Constrained Regressors in System Identification. In Proceedings of the 47th IEEE Conference on Decision and Control, 2008.• Henrik Ohlsson, Jacob Roll, Anders Brun, Hans Knutsson, Mats Andersson and Lennart Ljung. Direct Weight Optimization Applied to Discontinuous Functions. In Proceedings of the 47th IEEE Conference

on Decision and Control, 2008.• Henrik Ohlsson. Regression on Manifolds with Implications for System Identification, Licentiate Thesis no. 1382, 2008.

5Nonlinear System Identification Using EM

A new algorithm for nonlinear system identification based on

•EM algorithm

• Particle smoother

Computed using a particle smoother

Composition parameters as a general representation of mixture components

A DAE Formulation For Multi-Zone Thermodynamic Models and It’s Application to CVCP Engines

• A DAE formulation for multi-zone in-cylinder models is extended with change of composition and the possibility to handle liquid components.

• The concept of composition parametersis introduced and it is shown that using these the same set of equations can be used to model the two fundamentally different cases of a Well Stirred Mixerand Well Stirred Reactor.

• It is shown that for all Well Behaved Gases the DAE has a unique solution for all pressure, temperatures, and compositions as long as all zones have non zero mass.

Addition of liquid zone

Extension with change of composition

Torque Modeling for Optimising Fuel Economy in Variable Compression Engines.Ylva Nilsson, Lars Eriksson, and Martin Gunnarsson (2008)International Journal of Modeling, Identification and Control (IJMIC), vol 3(no 3).

Fuel optimal control of a variable compression engine is studied and it is shown that a crucial component is the model for the engine torque.

The modeling and validation is performed on a VCR using two fuels with different octane ratings.

A model for the produced work that captures the important effects of ignition and compression ratio is proposed. The main task for the model is to be a mean for determining the fuel optimal control signals, for each requested engine torque and speed.

The contribution is a model suitable for finding this optimal combination. This model consists of well-known components, and the novelty lies in the compilation and validation of the control oriented efficiency model for a variable compression engine.

Despite the models simplicity, it describes the indicated work with good accuracy, and suits its purpose of finding optimal control signals.

In the evaluation it is shown that a fuel optimal controller based on the proposed model captures the optimal IMEP to within 1.2 % This corresponds to a loss in engine efficiency that is in the range of 0.5 percentage units or less.

Model Engine

Fault diagnosis

Structural analysis is a tool to identify submodelssuitable for design of diagnostic tests

The conventional structural analysis is extended to nonlinear models with inequality constraints

New tests are designed that can improve fault isolability properties of the diagnostic system

Åslund, J., and Frisk, E. Structural analysis for fault diagnosis of models with constraintsSafeProcess Symposium, Barcelona, June 2009

We can see where you are looking from measurements of your brain activity!

14 M

ay-0

9

Lots of signals Where you are lookingReal-time fMRI

• Henrik Ohlsson, Joakim Rydell, Anders Brun, Jacob Roll, Mats Andersson, Anders Ynnerman and Hans Knutsson. Enabling Bio-Feedback Using Real-Time fMRI. In

Proceedings of the 47th IEEE Conference on Decision and Control, 2008.

left

rightt t

Using Real-Time fMRI to Control a Dynamical System

Accepted at the conferenceInternational Society for Magnetic Resonance in Medicine

(ISMRM), Honolulu, April 2009Anders Eklund, Henrik Ohlsson, Mats Andersson, Joakim Rydell, Anders Ynnerman, Hans Knutsson

Brain computer interfaceA neural network classifies the brain activity every second

The classification is used as a force on the inverted pendulumThe subject balanced the inverted pendulum during 7 minutes

Using Real-Time fMRI to Control a Dynamical System

by Brain Activity ClassificationSubmitted to the conference

Medical Image Computing and Computer Assisted Intervention (MICCAI)

Anders Eklund, Henrik Ohlsson, Mats Andersson, Joakim Rydell, Anders Ynnerman, Hans Knutsson

Longer version of ISMRM abstractThe subject could balance the pendulum with imagined activity

Concurrent Volume Visualization of Real-Time fMRI

Nguyen Tan Khoa, Henrik Ohlsson, Anders Eklund, Willem J. Frishert, Patric Ljung, Mats Andersson,

Hans Knutsson, and Anders Ynnerman

Submitted to IEEE Visualization 2009

Clinical Applications for Real-Time fMRISubmitted to the conference

Medical Image Computing and Computer Assisted Intervention (MICCAI)

Anders Eklund, Nguyen Tan Khoa, Willem Frishert, Henrik Ohlsson, Mats Andersson, Anders Ynnerman, Hans Knutsson

Interactive functional brain mapping as surgical aidBio-feedback, e.g. learn to suppress your pain

Stroke and Parkinson rehabilitation

Free Form Incident Light FieldsJonas Unger, Stefan Gustavson, Per Larsson and Anders Ynnerman

VITA, Linköping University

Computer Graphics Forum Vol. 27 Nr. 4, 2008The Eurographics Association and Blackwell Publishing Ltd.

Special issue EGSR '08,

Capture Processing Rendering

HDR image stream

Tracking information

ILF scene description

Extracted illuminantsBackground ILF

Free Form Incident Light Fields

• A technique for photo-realistic rendering using captured real world illumination

• Illumination capture using High Dynamic Range video imaging

• Projection onto a set of 2D surfaces in the scene

• A novel data structure for efficient random access of the 4D radiance data suitable for image synthesis

• Algorithm for photo-realistic rendering of synthetic objects placed in the captured real world scene

z

x

y

HDR Light Probe Sequence Resampling for Realtime Incident

Light Field RenderingJoakim Löw, Anders Ynnerman, Per Larsson, Jonas Unger

VITALinköping University

● Image Based Lighting– HDR measurements of

real world illumination

● Resampled to grid of spherical harmonics– Realtime lighting

– Memory efficient

HDR Light Probe Sequence Resampling for Realtime Incident Light Field Rendering

The Impact of Feedback Designon Haptic Volume Visualization

Karljohan Lundin Palmerius Camilla Forsell

Norrköping Visualization and Interaction StudioLinköping University, Sweden

Dynamic Deformation Using Adaptable, Dynamic Deformation Using Adaptable, Linked Asynchronous FEM RegionsLinked Asynchronous FEM Regions

UmutUmut KocakKocak, , KarljohanKarljohan LundinLundin PalmeriusPalmerius, Matthew Cooper, Matthew Cooper

MOVIII workshop2009-05-14

Prof Fredrik GustafssonDr Thomas SchönDr Umut OrgunerDr Gustaf HendebyDr David TörnqvistLic Jeroen HolKarl GranströmJonas CallmerFredrik Lindsten

MOVIII: Information Integration Publications 2008

fredrik@isy.liu.se

MOVIII workshop2009-05-14

Plenary presentation

Fusion of data from different sources

Author: T. Schön

Published: Proceeding of the 10th International Symposium on 3D Analysis of Human Movement (3DMA), 2008.

The H.J. Woltring lecture at 3DMA

MOVIII workshop2009-05-14

Two new courses

Two courses using MOVIII data/core problems

Graduate course: Dynamic vision, Poster

Undergraduate course: Sensor fusion, Poster

SLAM LAB

Rotation invariant visual odometry project

MOVIII workshop2009-05-14

Theses

Jeroen Hol, licentiate thesis: Pose estimation and calibration algorithms for vision and inertial sensors

David Törnqvist, PhD thesis: Estimation and detection with applications to navigation

MOVIII workshop2009-05-14

Particle filter theory

A basic convergence result for particle filtering

Authors: X. L. Hu, T. Schön, L. Ljung Published: IEEE Transactions on Signal Processing, Vol 56, pp

1337-1348, 2008.

Risk-Sensitive Particle Filters for Mitigating Sample Impoverishment Authors: U. Orguner, F. Gustafsson

Published: IEEE Transactions on Signal Processing, Vol 56, 2008.

Journal papers:• A new convergence proof of the PF with stronger results (the state estimate converges, not only bounded functions of the state).• An new alternative design and interpretation of jitter noise in the PF in terms of risk minimization.

MOVIII workshop2009-05-14

Particle smoothing in system ident.

Particle Filter Approach to Nonlinear System Identi¯ cation under Missing Observations with a Real Application

Authors: R. B. Gopaluni, T. Schön, A. G. Wills

Published: Proceedings of the 15th IFAC Symposium on System Identification (SYSID), 2009.

Parameter Estimation for Discrete-Time Nonlinear Systems Using EM

Authors: A. Wills, T. Schön, B. Ninness

Published: Proceedings of the 17th IFAC World Congress, 2008.

MOVIII workshop2009-05-14

Target tracking

Improved Target Tracking with Road Network Information Authors: U. Orguner, T. Schön, F. Gustafsson

Published: Proceedings of IEEE Aerospace, 2009. POSTERTarget tracking using delayed measurements with implicit constraints Authors: U. Orguner, F. Gustafsson

Published: Proceedings of 11th International Conference on Information Fusion, 2008.One of five papers selected for best paper awardStorage efficient particle filters for the out of sequence measurement problem

Authors: U. Orguner, F. Gustafsson Published: Proceedings of 11th International Conference on Information Fusion, 2008.

Illustration: Propagation delayed measurements give state dependent time delay in measurement equation

MOVIII workshop2009-05-14

Collision avoidance

IFAC symposium Safeprocess, Barcelona, 2009.

Authors: Fredrik Lindsten, Per-Johan Nordlund, Fredrik Gustafsson

Poster

MOVIII workshop2009-05-14

Calibration of camera and IMU units

A New Algorithm for Calibrating a Combined Camera and IMU Sensor Unit

Authors: J. Hol, T. Schön, F. Gustafsson

Published: Proceedings of the 10th International Conference on Control, Automation, Robotics and Vision (ICARCV), 2008.

Relative Pose Calibration of a Spherical Camera and an IMU

Authors: J. Hol, T. Schön, F. Gustafsson

Published: Proceedings of the 7th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), 2008.

MOVIII workshop2009-05-14

Other camera related issues

Detecting Spurious Features using Parity Space

Authors: D. Törnqvist, T. Schön, F. Gustafsson Published: International Conference on Control,

Automation, Robotics and Vision, 2008.

Separate presentation

Indoor photorealistic 3D mapping using stereo images from SLR cameras

Authors: V. Kolbe, F. Isaksson, T. Beckman, T. Schön

Published: Swedish Symposium on Image Analysis, 2009.

Awarded second prize for best industrial contribution

Ego-Motion and Indirect Road Geometry Estimation Using Night Vision

Authors: T. Schön, J. Roll Published: Proceedings of the IEEE Intelligent

Vehicle Symposium (IV), 2009.

MOVIII workshop2009-05-14

SLAM

Utilizing Model Structure for Efficient Simultaneous Localization and Mapping for a UAV Application

Authors: R. Karlsson, T. Schön, D. Törnqvist, G. Conte, F. Gustafsson Published: Proceedings 2008 IEEE Aerospace Conference, 2008.

Tree of Words for Visual Loop Closure Detection in Urban SLAM Authors: J. Callmer, K. Granström, J. Nieto, F. Ramos

Published: Proceedings of the 2008 Australasian Conference on Robotics & Automation, pp 8, 2008.

Learning to detect loop closure from range data Authors: Callmer, J.; Granstrom, K.; Nieto, J.I.; Ramos, F.T

Published: Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA)

MOVIII workshop2009-05-14

UT/UKF

On Nonlinear Transformations of Stochastic Variables and its Application to Nonlinear Filtering

Authors: G. Hendeby, F. Gustafsson

Published: International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008.

Knowledge Processing Middleware

■ Principled and systematic software framework for bridging the sense-reasoning gap in a physical agent

UAV State Estimation

Image Processing

Qualitative Spatial Reasoning

Chronicle Recognition

GPS

Pan-tilt unit

Color camera

Thermal camera

IMU

Anchoring

Heintz, Kvarnström and Doherty. Knowledge Processing Middleware. 1st International Conference on Simulation, Modeling and Programming of Autonomous Robots (SIMPAR), 2008.

Heintz, Kvarnström and Doherty. Bridging the Sense-Reasoning Gap: DyKnow – A Middleware Framework for Knowledge Processing. IROS Workshop on Current Software Frameworks in Cognitive Robotics Integrating Different Computational Paradigms, 2008.

Qualitative models

Knowledgefusion

Mixed models

Informationfusion

Quantitative models

Sensor data

Sensorfusion

DyKnow Federations

■ DyKnow supports information processing within a single UAV■ A DyKnow federation allows UAVs to share and merge

streams of information on many levels of abstraction■ Example: Detecting traffic situations using multiple UAVs

Platform 1

Gatewayagent

expo

rtpr

oxy

impo

rtpr

oxy

FIPA ACL

Directcommunication

Interfaceagent

DyKnowfederation

agent

Efficient streaming,low latency

Set up connectionusing speech acts

Platform 2

Gatewayagent

expo

rtpr

oxy

impo

rtpr

oxy

Interfaceagent

DyKnowfederation

agent

Heintz and Doherty. DyKnow Federations: Distributing and Merging Information Among UAVs.11th International Conference on Information Fusion, 2008.

Planning and Execution Monitoring

■ We use automated planning to achieve complex goals■ Plans may fail■ Monitor plan execution!

■ DyKnow generates state sequences■ Constrained by temporal logic formulas■ operator attach(uav, crate)

:monitor executing-attach(uav, crate) until [0,5000](carrying(uav, crate) until executing-detach(uav, crate))

Doherty, Kvarnström and Heintz. A Temporal Logic-based Planning and Execution Monitoring Framework for Unmanned Aircraft Systems.Journal of Autonomous Agents and Multi-Agent Systems (JAAMAS), 2009.

Kvarnström, Heintz and Doherty. A Temporal Logic-Based Planning and Execution Monitoring System. 18th International Conference on Automated Planning and Scheduling (ICAPS), 2008.

Fly Attach Detachcarrying(uav,crate)

…

at most 5000 ms

Stream Reasoning in DyKnow

■ DyKnow represents information about the world as streams.■ By incrementally reasoning about the world as new information becomes

available it is possible to draw conclusions with minimal delay.■ Two examples of stream reasoning in DyKnow:

■ Evaluating temporal logical formulas over streams of states using progression. executing-attach(uav, crate) until [0,5000] (carrying(uav, crate) until executing-detach(uav, crate))

■ Detecting complex situations described by simple temporal networks of events over streams of event occurrences using chronicle recognition.

Doherty, Kvarnström and Heintz. Stream Reasoning in DyKnow: A Knowledge Processing Middleware System. ECSW Workshop on Stream Reasoning (SR), 2009.

A B C

D E

[0, 2000] [500, 1000][0, 1000]

[1000, 1000][1000, 2000]

A Reconfigurable Diagnosis System

■ Problem formulation:■ multiple fault diagnosis (MFD)■ real-time requirements■ dynamic systems

■ Solution:■ tests with thresholded residuals■ accurate MFD requires many tests,

however all tests are not needed at all times

■ we propose FlexDx, a reconfigurable diagnosis system only running testsneeded for the current diagnoses which starts tests in the past if necessary

Heintz, Krysander, Roll and Frisk. FlexDx: A reconfigurable diagnosis framework. 19th International Workshop on Principles of Diagnosis (DX), 2008.

Krysander, Heintz, Roll and Frisk. Dynamic test selection for reconfigurable diagnosis. 47th IEEE Conference on Decision and Control (CDC), 2008.

■ Key contributions:■ How to select and initiate tests

given the current diagnoses, and

■ how to implement FlexDx using the stream-based DyKnow middleware.

TestTestResidualResidual

DiagnosesTestSet

System

system observations residuals

last fault free time

test set

diagnoses initial diagnosis

conflict set

ResidualGenerator

ConflictSetMediator

CreateTests

ResidualMonitor

Positioning Communication Relays

■ UAVs used for monitoring of important targets■ High volume transmission requires line-of-sight■ Problematic in urban or mountainous terrain

■ Use chain of intermediate relay UAVs – must find suitable placement!■ Two new algorithms developed, based on discretization

■ Experiments: Faster by a factor of 10–50

Burdakov, Doherty, Holmberg, Kvarnström and Olsson. Positioning Unmanned Aerial Vehicles as Communication Relays for Surveillance Tasks. Robotics: Science and Systems (RSS), 2009.

Burdakov, Doherty, Holmberg and Olsson. Optimal placement of communications relay nodes.23rd European Conference on Operational Research (EURO), 2009.

Logical Reasoning

■ Logical reasoning is powerful, in theory■ We want to make it practical

■ FCTP algorithm for logical reasoning with assumptions■ Sound and complete for a well-defined class of theories■ Fully interruptible

Magnusson, Kvarnström and Doherty. Abductive Reasoning with Filtered Circumscription. 8th Workshop on Nonmonotonic Reasoning, Action and Change (NRAC), 2009.

FCTP

Communication

■ Complex real-world scenarios involve cooperation■ Cooperation requires communication

■ Logical reasoning about belief and commitment■ Planning speech acts and cooperation

Magnusson and Doherty. Planning Speech Acts in a Logic of Action and Change.25th Workshop of the Swedish Artificial Intelligence Society (SAIS), 2009.

Execution

■ Plans break down when executed

■ Fully automatic execution monitoring■ Fully automatic plan revision

Magnusson, Landén and Doherty. Planning, Executing, and Monitoring Communication in a Logic-based Multi-agent System. 18th European Conference on Artificial Intelligence (ECAI), 2008.

Magnusson, Landén and Doherty. Logical Agents that Plan, Execute, and Monitor Communication. 2nd Workshop on Logic and the Simulation of Interaction and Reasoning (LSIR-2), 2009.

Interactivity

■ We have autonomous cooperating robot agents■ Now if only we could communicate with them!

■ Natural language question answering■ Interactive computer game application

Magnusson and Doherty. Logical Agents for Language and Action.4th Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2008.

Who are you?

I am Smith.

Loops

■ Logical reasoning is powerful, in theory■ We’ve made it (more) practical■ But is it still powerful?

■ Automated induction proofs■ Infinite UAV surveillance plans■ Compact representation using fixpoint logic

Magnusson and Doherty. Deductive Planning with Inductive Loops.11th International Conference on Principles of Knowledge Representation and Reasoning (KR), 2008.

Non-GPS Navigation

■ The outcome of this research is a navigation system which combines complementary vision-based techniques together with inertial sensors to navigate a UAV in case of GPS outages

■ The approach proposed relies on geo-referenced imagery information stored in the platform and it is based on probabilistic techniques

■ The navigation system developed has been implemented and tested on-board an experimental unmanned helicopter platform

Conte and Doherty. Vision-based Unmanned Aerial Vehicle Navigation Using Geo-referenced Information. Accepted for publication: EURASIP Journal of Advances in Signal Processing, 2009.

Vision-based Indoor Navigation

A method for accurate vision-based state estimation which allows for autonomous indoor navigation of micro-scale UAS’s has been implemented and tested in real-flight closed-loop experiments.

The system consists of vision-based pose estimation, sensor fusion by means of a 9 state Kalman filter, and control executed onboard the UAS. Autonomous flight capability is achieved without communication with external entities.