INFOTECH OULU Annual Report 2015 1 BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG) Professor Juha Röning and Dr. Heli Koskimäki, Faculty of Information Technology and Electrical Engineering, and Professor Seppo Vainio, Faculty of Biochemistry and Molecular Medicine juha.roning(at)oulu.fi, heli.koskimaki(at)oulu.fi, seppo.vainio(at) oulu.fi http://www.oulu.fi/bisg Background and Mission Biomimetics and Intelligent Systems Group (BISG) is a fusion of expertise from the fields of computer sci- ence and biology. In BISG, our basis are intelligent systems and our research areas include data mining, machine learning, robotics, and information security. More precise research topics vary from data mining algorithm development and optimization of industrial manufacturing processes all the way to environmental monitoring with mobile robots. Bringing expertise from ICT and Biotech together we will reach the skills to make use of the mechanisms common in information processing and the biological data processing system and extrapolate this to intelli- gent solution making in ICT. One important goal of this program is to be able to physically link living cells via identified signaling systems to establish learning complex that involves Bio and ICT in a unified bifunc- tional interactive machine. The group consists of four sub-groups: Data Analysis and Inference Group, Developmental Biology, Robot- ics and Secure Programming We have conducted basic research in intelligent sys- tems and developmental biology for over ten years as individual groups. Now we have joint our efforts. Our team consists of two professors, 10 post-doctoral re- searchers and 15 doctoral students. The annual external funding of the group is more than two million Euros, in addition to our basic university funding. There have been two completed doctoral degrees from the group. From the research of the group, eleven spin-out com- panies have been established so far: Codenomicon, Clarified Networks, Hearth Signal, Nose Laboratory, Nelilab, Atomia, Indalgo Probot, Aquamarine Robots, Radai and IndoorAtlas. We co-operate with many international and domestic partners. In applied research, we are active in European projects. In addition, several joint projects are funded by the Finnish Funding Agency for Technology and Innovation (Tekes) and industry. We were a research partner in the, Internet of Things (IoT), SIMP and CyberTrust SHOKs. Prof. Juha Röning was selected as ACO (Academic coordinator) of the Cyber Trust pro- gram. We are active in the scientific community. For exam- ple, Prof. Juha Röning is acting as visiting professor of Tianjin University of Technology and as the Robot Science Adviser of Tianjin Science and Technology Center for Juveniles. He served as a member of the Board of Directors in euRobotics and as a member of the SAFECode International Board of Advisors, and as a chief judge in the euRathlon 2015 (air+land+sea) competition, which took place Piombino, Italy, from the 17th to 25th September. He chaired the euRathlon / SHERPA Summer School 2015 in Oulu, Finland, 1 st to 5 th of June. It was a five-day course to provide partici- pants with a full overview and hands-on experience with multi-domain real robotic systems. He also chaired with prof. Othmane the First International Workshop on Agile Development of Secure Software (ASSD’15) in Toulouse 24 th of August 2015. In Tekniikan päivät (Tampere 23-24.10.2015) he lectured about Cyber Security. Prof. Seppo Vainio has been the chair in the Personalized Medicine day (2015), course on 580402S Biomedical imaging methods (1-4 ECTS; 2015) and BIG data seminar (2015. Prof. Seppo Vainio is part of a European nanotechnology ”HyNanoDend” network. During the reporting year, the group organized the 8th International Crisis Management Workshop and Winter School (CrIM’15), which brought together both Finn- ish and international information security experts. The group also organized Big Data Seminar 14 th of Sep- tember bringing together people interesting genome, and morphogenesis. Scientific Progress Intelligent Systems Incorporating Security Within the Biometics and Intelligent Systems Group, the Oulu University Secure Programming Group (OUSPG) has continued research on security and safety in intelligent systems. Security and safety challenges in intelligent systems are threefold: increasing complexity leads to unforeseeable failure modes, quality is not the priority and awareness is lacking. We have approached the challenges from these three directions in our re- search. Complexity - Model Inference and Pattern Recogni- tion: we work under the premises of unmanageable growth in software and system complexity and emer- gent behaviour (unanticipated, not designed) having a major role in any modern non-trivial system. We have worked on natural science approaches to understanding artificial information processing systems. We have developed and applied model inference and pattern
Transcript
INFOTECH OULU Annual Report 2015 1
BIOMIMETICS AND INTELLIGENT SYSTEMS GROUP (BISG)
Professor Juha Röning and Dr. Heli Koskimäki, Faculty of Information Technology and Electrical
Engineering, and Professor Seppo Vainio, Faculty of Biochemistry and Molecular Medicine
and criminal enterprise have proven that software and
system security is not just an add-on, despite the past
focus of the security industry. Instead, security, trust,
dependability and privacy have to be considered over
the whole life-cycle of the system and software devel-
opment, from requirements all the way to operations
and maintenance. This is furthermore emphasized by
the fact that large intelligent systems are emergent and
do not follow a traditional development life-cycle.
Building security in not only makes us safer and se-
cure, but also improves overall system quality and
development efficiency. Security and safety are trans-
formed from inhibitors to enablers. We have developed
and applied black-box testing methods to set quantita-
tive robustness criteria. International recognition of the
Secure Development Life Cycle has provided us with a
way to map our research on different security issues.
Awareness - Vulnerability Life Cycle: Intelligent sys-
tems are born with security flaws and vulnerabilities,
new ones are introduced, old ones are eliminated. Any
deployment of system components comes in genera-
tions that have different sets of vulnerabilities. Tech-
nical, social, political and economic factors all affect
this process. We have developed and applied processes
for handling the vulnerability life-cycle. This work has
been adopted in critical infrastructure protection.
Awareness of vulnerabilities and the processes to han-
dle them all increase the survivability of emergent
intelligent systems for developers, users and society.
These research goals are reached through a number of
research activities.
Secure Software Development Lifecycle as a part of
the Cyber Trust project - we approach all three goals
by researching practical ways of building security into
Secure Platforms, Cloud Computing services and Criti-
cal Infrastructure, from the design phase to actual oper-
ational use (Figure 1).
Figure 1. Dependencies of a single cloud based web service visualized by technology and location.
Situational Awareness in Information and Cyber Secu-
rity aims to understand critical environments and accu-
rately predict and respond to potential problems that
might occur. Networked systems and networks have
vulnerabilities that present significant risks to both
individual organizations and critical infrastructure. By
anticipating what might happen to these systems, lead-
ers can develop effective countermeasures to protect
their assets (Figure 2).
Figure 2. Port scanning visualized in an industrial auto-mation network.
Coverage based robustness testing: Modern web
browsers are feature rich software applications availa-
ble for different platforms ranging from home comput-
ers to mobile phones and modern TVs. Because of this
variety, the security testing of web browsers is a di-
verse field of research. Typical publicly available tools
for browser security testing are fuzz test case genera-
tors designed to target a single feature of a browser on
a single platform. This work introduces a cross-
platform testing harness for browser fuzz testing, called
NodeFuzz. In the design of NodeFuzz, test case gen-
erators and instrumentation are separated from the core
into separate modules. This allows the user to imple-
ment feature specific test case generators and platform
specific instrumentations, and to execute those in dif-
ferent combinations. During development, NodeFuzz
was tested with ten different test case generators and
six different instrumentation modules. Over 50 vulner-
abilities were uncovered from the tested web browsers
during the development and testing of NodeFuzz
Identification of a protocol gene: this research, PRO-
TOS-GENOME, approaches the problems of com-
plexity and quality by developing tools and techniques
for reverse-engineering, and identification of protocols
based on using protocol genes - the basic building
blocks of protocols. The approach is to use techniques
developed for bioinformatics and artificial intelligence.
Samples of protocols and file formats are used to infer
structure from the data. This structural information can
then be used to effectively create large numbers of test
cases for this protocol. In 2015, the project further
developed the existing methodology, resulting in im-
provements in efficacy and discovering a number of
vulnerabilities in web browsers.
Internet of Things studies security and privacy issues in
large-scale sensor networks. Topics of interest are
alternative ways of authentication, such as proof of
work and cryptocurrencies, secure update mechanisms,
INFOTECH OULU Annual Report 2015 3
software defined networking and related service-level
agreements for data centres. The work brings together
our research themes of Quality, Complexity and
Awareness to an application area where resource limits
are combined with global connectivity.
Privacy and Security and Online Social Networks
Exploiting Social Structure for Cooperative Mobile
Networking (SOCRATE), a two -year (2015-2016)
Tekes funded project under the Wireless Innovation
between Finland and U.S. programme WiFiUS
[http://wifius.org/], is a collaboration between the Uni-
versity of Oulu (co-PI Dr Ulrico Celentano), VTT,
Aalto University, Arizona State University, and Uni-
versity of Nevada. During 2015, BISG has focused on
architecture and on privacy and security issues in
online social networks data mining.
Knowledge about the social structure of network users
can be exploited to optimise radio network operations
(Figure 3).
SOCIAL CONTEXT INTERFACE
ODS1 OSN1 CRN
ESTIMATION, PREDICTION, INFERENCE
... ...
WIR
EL
ES
S N
ET
WO
RK
S
ODS1 OSN1
EN
HA
NC
EM
EN
TS
/AM
EN
DM
EN
TS
SO
CIA
L-A
WA
RE
PR
OT
OC
OL
S
PR
OF
ILE
S
... ...
Smart caching
Social-aware protocols
Social-aware resource allocation
User behavior
Location and context
Mobility pattern
Social interests
Social relationship
Events
Environmental data
WIRELESS NETWORKS
SOCIAL NETWORKS
OPEN DATA
(Bulletins, Road
traffic, …)
Information for
network
reconfiguration,
contents
dissemination
Consensus
Figure 3. A conceptual framework supporting optimisa-tion of mobile networks by exploiting social structure of physical users. (Celentano et al. 2016.)
To this end, personal and sensitive information about
users should be collected. Clearly, this needs to be
done in the fullest respect of the users’ privacy. Privacy
protection is important not only to protect individuals’
intimacy, but also to prevent identity theft: related
threats may finally expose the system to denial of ser-
vice or sabotage. The solution to overcome the above
and minimise the risks associated with potential
threats, includes properly specifying the data structure
by partitioning attributes, managing access rights and
the related keys, and designing the topology of reposi-
tories (Figure 4).
access manager
...
R0
contents manager
R1 Rm-1
S1S0 Sn-1
...
......
Figure 4. Data distributed across repositories and parti-tioned into domains, with access and content manage-ment. (Celentano and Röning 2015.)
Results in this area have been presented at WF-IoT
2015 (Celentano and Röning 2015). Furthermore, the
University of Oulu was the editor of a project-wise
joint magazine article (Celentano et al., Manuscript)
currently under review.
Intelligent Systems Incorporating Machine Learning and Data Mining
GlobalRF: Two-year Tekes funded project, the Global
energy expenditure estimation and cognitive load when
Figure 9. Principles of basic leave-one-person-out cross-validation in activity recognition and three ways for train-validate-test approach: single division, 10-fold division and double leave-one-person-out cross-validation. The bias in basic leave-one-person-out cross-validation is due the back propagation marked with slashed red arrow. If there are no back propagation the basic scheme is adequate.
INFOTECH OULU Annual Report 2015 7
starting to solve a new machine learning problem. She
also considered the special restrictions and require-
ments that need to be taken into account when applying
regular machine learning algorithms to time series
data.
Intelligent Systems Incorporating Robot-ics and Cybernetics
euRathlon Summer School
The euRathlon SHERPA summer school 2015 was
organized from the 1st to 5th of June mainly by the
robotics group members at the department of Computer
Science and Engineering. The summer school was
attended by 42 students, mostly doctoral, originating
from 10 different countries (Figure 10). Also, eight
invited lecturers, mostly from SHERPA, held lectures
during the summer school. Overall, based on the satis-
faction survey after the event, the participants were
especially pleased with the overall organization of the
event.
Figure 10. Early attendees in the euRathlon SHERPA 2015 summer school organized at the University of Oulu.
This year, the euRathlon summer school was a joint
operation between different autonomous mobile robot-
ics fields, the aerial (SHERPA, Smart collaboration
between Humans and ground-aErial Robots for im-
Proving rescuing activities in Alpine environments),
the land and marine robots (University of Oulu, Probot
Ltd. and Aquamarine Robots Ltd., Coppelia Robotics
GmbH). In total, the summer school lasted for four and
a half days consisting roughly 50% of lectures and 50%
of practical exercises. At the beginning of the summer
school, the students could choose their preferred field
to study; aerial, marine or land. The students following
the land robot exercises needed to implement control
algorithms for the land robots with the aid of the intui-
tive Coppelia Robotics V-REP simulator before testing
the algorithms in the real world environment. The robot
server interface, where control algorithm software
client implemented by the students in Python language
connected, was the same for both the simulator and the
real robots. The drag and drop robot in the simulator
world and the real robots could then be controlled by
the same implemented client side control software
(Figure 11–12).
Figure 11. V-REP Simulator environment running on Porteus Linux and the Navigation UI client program connected to the virtual robot.
Figure 12. The outdoor land robots that were assem-bled for testing the control implementations made by the students; Mörri on the left side and C-frame built from modular robot building blocks, provided by Probot Ltd, on the right side.
The quadrotor aerial drone exercises consisted of im-
plementing control algorithms for the drone (Figure
13), which was operated through software running on
ground station PC. The control code was implemented
in C# programming language, using predefined primi-
tives, such as Goto, RotateYaw and TakePicture. The
ground station communicated with the aerial drone via
a Wi-Fi link, using MAVlink protocol. The students
also had access to high level information of the quad-
copter, such as drone altitude, position in the NED
(North-East-Down) frame, status of the flight battery
and more.
Figure 13. The flying drone utilized in the aerial robot exercises.
INFOTECH OULU Annual Report 2015 8
In the aquatic scenario, four teams were instructed to
design path tracking and station-keeping algorithms for
an unmanned surface vehicle. The challenges for the
aquatic control algorithms followed from an unknown
control model and varying environmental conditions
(wind). In order to obtain successful implementations
and to pre-test their algorithms, after some literature
review to the state-of-the-art control algorithms, stu-
dents started to implement their algorithms in a PC
class room with simulator environment. In simulations,
students were using Aquamarine Robot’s GUI to fol-
low how their algorithms perform with different wind
conditions. After their implementation was accepted in
simulator environment, each team tested their imple-
mentation and made some modifications with marine
robot Dolphin in a small lake (Figure 14).
Figure 14. The UI of the Aquamarine Robots Dolphin robot utilized in the marine robot exercises and the image of the Dolphin captured from above autonomous-ly by the SHERPA flying drone (bottom picture).
The exercises were aimed to implement control algo-
rithms for the robots in order to execute a co-operation
scenario where the robots could perform joint actions
using a central communications server. The scenario
area was selected to be the nearby bay area (Kaijonlah-
ti), where the marine, land and aerial robots could op-
erate. The area used was approximately 350 * 350
meters, consisting mainly of a water area and of a
grassy field with no trees (Figure 15).
Robotics Research
In 2015 BISG had a wide range of research in the area
robotics, including industrial safety, aerial data gather-
ing, battery life management and control of complex
wheeled land robots.
ReBorn
In the EU funded ReBorn project, having participants
from 17 industrial and academic institutions from 10
different countries, research related to improving the
recyclability of industrial robots has been carried out.
Figure 15. The outdoor robot exercise area viewed from the Google Earth application.
The main focus of the University of Oulu in the project
has been in the identification of typical use cases, the
related standardization and the identification of poten-
tial improvement areas in the existing standardization.
In industrial manufacturing, the work scenarios are
becoming progressively more human-robot interaction
and collaboration oriented. Therefore, the safety re-
quirements in human-robot collaborative scenarios
must be clearly defined and standardized to minimize
the possibility of user injuries (Figure 16). Especially
in collaborative scenarios, proper safety equipment and
user aware control algorithms must be in place to elim-
inate the possibility of injury to humans operating
inside collaborative work spaces. In addition to en-
cations require new virtual, cloud based and robot
capability aware technologies to be employed for im-
proved manufacturing interaction. These technologies
can also enable faster product development cycles and
facilitate higher level of product customization options
on the production line.
Figure 16. One possible risk potential evaluation sce-nario of collaborative work spaces, each object is as-signed a risk potential based on possibility and severity of injury. The total spatial risk potential is used in evalu-ation of the constraints used during robot movement planning and operation.
Matine
Study of ionizing radiation detection and sample col-
lection with a DJI Inspire 1 quadcopter is being carried
out in co-operation with STUK (Radiation and Nuclear
INFOTECH OULU Annual Report 2015 9
Safety Authority), University of Helsinki and Finnish
Defence Research Agency in a MATINE (Maan-
puolustuksen tieteellinen neuvottelukunta) funded
project. In initial phases, the quadcopter has been
equipped with a Kromek GR1-A gamma-ray spectrom-
eter (Figure 17–18) which is connected to a
smartphone via USB. This smartphone is equipped
with an application that was developed to collect radia-
tion data from Kromek GR1-A, and send it to database
via WLAN/3G (Figure 19). This combination is used
for identification of radiation sources in outdoor envi-
ronments. Initial testing has been performed for testing
the feasibility of the suggested approach and the field
tests may begin in the early 2016.
Figure 17. 3D printed casing implemented for attaching a gamma-ray detection sensor onboard a DJI Inspire 1.
Figure 18. Test spectrums collected with Kromek GR1-A spectrometer from low radiation (10 μCi) sources from a 1 cm distance. The main photon energy spike (662 keV) detected from a Cs-137 sample is shown in 1. The photon energy spikes at 1.17 MeV and 1.33 MeV detected from Co-60 sample are seen in 2. and 3.
Battery Management
First functional prototypes of the modular intelligent
battery modules are being tested with mobile modular
robot units developed with Probot Ltd. With hot-
swappable, modular intelligent battery energy storages
Figure 19. An Android application was developed to gather data from Kromek GR1-A gamma-ray spectrom-eter and send it to database.
(Figure 20–21), the energy can be transferred to the
mobile platform modules from the switched payloads
also during operation, enabling continuous uninter-
ruptible operation. Battery state awareness and auto-
mated operational efficiency optimization are being
developed related to these platforms for implementing
more energy efficient mobile robot units for challeng-
ing environments. A general artificial neural network
based Li-ion battery State-of-Charge estimation model
has been developed to be used in battery energy man-
agement applications (Figure 22–25). Research is also
being carried out for utilizing deep-learning neural
networks in conjunction to predictive route planning
for the mobile robot mission execution algorithms.
Deep-learning based environment classification could
be especially useful in co-operation applications of
mobile ground platforms and unmanned aerial units
that can survey and classify the ground robot operating
area from above.
Control of Complex Wheeled Robots
Pseudo-omnidirectional robots with individually steer-
able wheels offer a good balance between payload,
robustness and mobility. However, the non-holonomic
nature of the regular wheels and the often redundantly
actuated structure of these robots make their control a
complex issue. This complexity of control is further
exacerbated when the wheels are not rigidly connected
to the robot body but are instead connected via actuated
chains which allow the wheels move relative to the
body. BISG has developed control algorithms for such
Articulated Wheeled Vehicles (AMW). The control
algorithms are mathematically simple closed-form
analytical functions and are thus computationally light
INFOTECH OULU Annual Report 2015 10
Figure 20. Insides of the intelligent battery module with a separable DC-DC converter module used for battery charging that can be located inside or outside of the battery module. Internal charger is useful for improving system flexibility while external charging schemes are more cost efficient solutions.
Figure 21. Real-time measurements done by the intelli-gent battery module onboard a mobile robot.
Figure 22. Basic normalized measurement inputs ob-tainable from a Li-ion battery system; voltage (blue), temperature (red), power (green). This data was col-lected from a 2012 model Mitsubishi I-Miev full electric car. Later, dataset 1. was used as training data and 2. as validation and testing data.
Figure 23. Artificial neural network based State-of-Charge (SoC) estimation system that uses basic meas-urement inputs available from most battery measure-ment systems. In addition to a neural network, input space is expanded with battery voltage behavior related operators f.
Figure 24 Battery state estimation model output êSoC (blue) vs. the post calculated real SoC (green).
Figure 25. Error between model estimated and real SoC.
but are currently limited to planar cases. The computa-
tional load is only linearly dependant on the number of
wheels making the developed control algorithm suita-
ble for multi-wheel configurations and/or low-powered
embedded MCUs. The control algorithms synchronize
the rolling and steering velocities of complex planar
robots (example in Figure 26) with freely located
wheels forming fixed or variable (Figure 27) footprints.
The rolling and steering velocities remain synchronized
even with very complex motions of the robot (Figure
28). With the developed control algorithms, the tra-
versable path, robot’s heading on different points of the
path and the path velocity can be controlled separately,
thus offering great freedom on how to control the robot
on a given practical task. The control algorithms do not
in practice suffer from representation singularities
which are a common problem in wheeled control. The
control algorithms also compensate for the proximity
of mechanical singularities by adjusting the robot’s
path velocity according to the maximum capabilities of
INFOTECH OULU Annual Report 2015 11
its wheels’ steering and rolling actuators. In fact the
developed control algorithms are time optimal in a
sense that at any given moment the robot is either trav-
ersing with maximum allowed path velocity or at least
one of its steering or rolling actuators is turning at its
maximum velocity (Figure 29), i.e. the robot traverses
the given path in the given way with the given velocity
restrictions as fast as it possibly can.
Figure 26. Example of complex wheeled planar robot.
Figure 27. Example of wheels’ motion with respect to the robot body. The colored lines depict the wheel paths as the robot body traverses a straight line.
Figure 28. Simulation run of Figure 26’s robot traversing a given path (blue dots) while keeping its front directed at all times to a point of interest (green larger dot). The path was 82 meters in length.
Figure 29. (Top) wheel rolling speeds, (Middle) wheel steering speeds and (Bottom) robot path velocity for the first 30 seconds of Figure 19’s simulation run.
In summary, the developed control algorithms can be
used in a wide range of robot configurations and sce-
narios with low computational cost. The control algo-
rithms are currently limited to planar surfaces and can
cause sudden and large changes in velocity (Figure 28
bottom) and the control algorithms are being extended
to work also with uneven surfaces and limited accelera-
tions.
Magnetic Field Localization and SLAM
The objective of our indoor localization research is to
develop methods for exploiting indoor magnetic field
variation in positioning and mapping. The idea is based
on analysis with various indoor magnetic field datasets
showing that indoor magnetic fields provide sufficient
spatial variation and temporal stability to permit infer-
ence about sensing locations, given noisy measure-
ments. In recent years, we have published various pa-
pers studying magnetic field localization and related
methods in robotic and human contexts (Figures 30 and
31).
In 2015 we continued our magnetic field SLAM explo-
ration studies. SLAM exploration refers to the methods
where we try to find optimal ways to collect magnetic
information for mapping purposes, meanwhile, simul-
taneously use this information for localization purpos-
es. We have developed new ways to model magnetic
field information using spectral Gaussian processes.
We also have developed methods for efficient action
selection using environmental partition based on in-
formation similarities. The results will be published on
autumn 2016.
INFOTECH OULU Annual Report 2015 12
Based on the magnetic field localization studies, a new
start-up company, called Indoor Atlas Ltd., was found-
ed in 2012. This company offers indoor positioning
technologies for various application areas. The compa-
ny has generated high interest in international technol-
ogy magazines.
In our research on magnetic field simultaneous locali-
zation and mapping (SLAM), we have put a strong
emphasis on light-weight methods running entirely on
mobile platforms, such as Android smartphones and
tablets. Compact map representation and effective
algorithms are essential when using devices with very
limited resources, and we have developed methods to
tackle the problems arising from very sparse data and
high uncertainty levels produced by low-cost and noisy
smartphone sensors. Our work is continuing toward an
autonomous mobile robot system based solely on
smartphone sensors that is able to intelligently build a
map of the magnetic environment (Figure 30).
Figure 30. Magnetic landscape (bottom) of University of Oulu Discus entrance hall (top-left). The landscape illustrates the spatial variation of the magnetic field that allows magnetic field-based localization and mapping. The map is created by an iRobot Create robot (see Figure 31) in a simultaneous localization and mapping (SLAM) experiment.
Figure 31. An iRobot Create collecting data for simulta-neous localization and mapping (top-left) and the result-ing map of the magnetic field (top-center). The bottom row visualizes that the robot can correct its path by using the magnetic information (bottom-left). Without magnetic data, the odometry is highly erroneous (bot-tom-center). If equipped with similar hardware, such as smartphones or tablet computers, both humans and robots can utilize the same magnetic maps (right).
Social robot scenarios are particularly difficult because
of the dynamic (often crowded) environment. Magnetic
field localization is not affected by surrounding people
like laser scanners and cameras for example, and it is
therefore very promising in these kinds of scenarios.
While our method is used to localize the robot, the
other sensors can be assigned to handle the social tasks.
We have also developed localization methods that are
usable by both robots and humans equipped with simi-
lar mobile devices (Figure 31).
Efficient Systematic Sampling from a Discrete Distribution
In order to improve the sampling process in the mag-
netic field SLAM motion model, we have developed an
efficient and low-variance Systematic Alias Sampling
(SAS) method based on the alias method by Walker.
The method produces batches of samples from an arbi-
trary discrete distribution by systematically drawing
from the alias table structure (see Figure 32). SAS
produces samples up to 20 times faster than the com-
pared sampling method in a popular Java mathematics
library (Apache Commons Math). In addition, the
Cramér-Von Mises statistic shows that SAS produces
much more fit empirical distributions than i.i.d. sam-
pling, if the problem of almost divisibility between bin
and sample count is carefully taken care of.
Figure 32. Top: Discrete 101-valued approximation of the standard normal distribution with support of [−4,4] denoted as N101. Middle: Alias table structure generat-ed from N101. The probabilities of selecting the aliased values are depicted as the upper part of the bins. The colors correspond to values in the topmost figure. The black dots are an example batch of 16 systematic sam-ples. Bottom: The empirical distributions defined by the 16 systematic alias samples (SAS) and 16 i.i.d. sam-ples compared to the true cumulative distribution func-tion.
INFOTECH OULU Annual Report 2015 13
Evolutionary Robotics
During 2015 we published the results of study consid-
ering the Lego Mindstorms NXT platform’s suitability
for evolutionary robotics by using a genetic algorithm
to evolve a neural network-based controller for Lego
sumo wrestling. The genetic algorithm was able to
evolve a controller with simple but effective strategy
for the task. The emerged behavior is illustrated in
Figure 33. A video of the evolution can be found at:
https://www.youtube.com/watch?v=PaOADqerpWU.
Furthermore the Lego platform has been utilized in
collaboration with local schools and students, e.g. in
the form of workshops. The experiments on the
platform show that the tools already used in school
education can be utilized to create hands-on
experiences e.g. on the principles of evolution for the
students.
Figure 33. Evolutionary robotics on Lego NXT platform. A typical example of the emerged behavior during gen-eration 100. Both robots start scanning the environment by turning left. Soon the red one gets the yellow one in sight and is able to charge towards the opponent, push-ing it outside the ring, resulting in an easy victory.
The Evolutionary Active Materials
The Evolutionary Active Materials (EAM) project,
which is funded by the Academy of Finland, is a joint
effort between the Computer Science and Engineering
laboratory (CSE) and the Microelectronics and Materi-
als Physics laboratories. The aim of the EAM project is
to develop novel, evolutionary computation (EC) based
design methods for active and versatile materials and
structures. The first components are being developed
through a novel holistic design process utilizing con-
stantly increasing computation power, the development
of multi-physics simulators, and EC techniques, such
as genetic algorithms.
During 2015, the height and the top diameter of Cym-
bal type piezoelectric actuator were optimized by ge-
netic algorithm and FEM modelling. From the opti-
mized results, maps of electromechanical capabilities
of different structures were generated. The blocking
force of the actuator was maximized for different val-
ues of displacement by optimizing the height of the cap
and the length flat region of the end cap profile. By
using values obtained from a genetic algorithm optimi-
zation process, a function was formulated for design
parameters. Using the function, a map of displacement,
the steel thickness and the height of the end cap the
optimized length of flat region was constructed (Figure
34). A similar map with the length of the flat region for
the optimized height of end cap was created. The re-
sults will be published at 2016.
Figure 34. The top diameter of the steel cap as a func-tion of steel thickness and displacement for Cymbal.
New type of actuator called Mikbal (Figure 35) was
invented, optimized with genetic algorithm and real-
ized. Mikbal was developed from Cymbal by adding
additional steel structures around the steel cap to in-
crease displacement and save the amount of used pie-
zoelectric material. The best displacement to amount of
used piezo material ratio was achieved with 25 mm
piezo material diameter in the case of 40 mm steel
structures, and lower height and top diameter of the cap
increased the displacement. The results will be pub-
lished during 2016.
INFOTECH OULU Annual Report 2015 14
Figure 35: The von Mises stresses in Mikbal actuator under 500 V voltage.
Also optimization of the end cap structure of the Cym-
bal type energy harvester was done with genetic algo-
rithm and FEM modeling software Comsol Multiphys-
ics. The aim was to improve harvested power levels
from human walking (Figure 36). The power produced
by the energy harvester was increased by allowing the
algorithm to modify thickness in certain regions as
grooves in the end cap. By evolution of the structure,
power produced by the harvester increased by 38 %
compared to traditional linear type Cymbal harvester
which was also optimized by the algorithm. Increase in
power was obtained by change of mode in mechanics
of the harvester by grooves.
Figure 36. Cymbal type energy harvester in a shoe and an optimised profile for the harvester. In the profile piezoceramic disc is depicted in yellow and steel cap in grey. The grooves shown in the left side of the profile have been found by the genetic algorithm.
Intelligent Systems Incorporating Bio-IT Solutions
Developing novel real-time biosensors for glucose
monitoring. For developing “second generation biosen-
sors” we have taken use of our skills to purify and
culture the skin derived progenitor cells that are re-
sponsible in skin renewal and regeneration. We ob-
tained for the project a Tekes strategic opening fund-
ing. With this support we have advanced the work to
develop of a novel biosensor strategy (Figure 37).
Figure 37. Novel biosensor strategy. Donor skin renew-ing cells are set to culture and a specific responsive component is engineered to target a tag to the 3´end of the coding sequence in the genome. Such a cell is then implanted to the donor to serve as a measure for a given physiological parameter.
By now we have been able to conduct the proof of
principle set up in the sensor construction. These indi-
cate that the skin is indeed responsive to the changes in
certain serum constituents. The data also indicated that
the cells with in the skin can also be engineered and be
converted genetically to serve as biosensors. We have
screened in selected biological phenomena with the
proteomics and transcriptomics the respective mediator
signal transduction pathways serving as a read out. We
identified wealth of candidate factors whose genes are
currently being engineered to convert the respective
protein into an isoform whose activity can read with an
external device.
We have also tested the capacity to culture of FACS
purified cells of the skin and also if such cells can be
transplanted with a fluorescent tagged cells to the do-
nor so that the cells indeed become incorporated. We
assayed the stability of the sensor transplant. The data
suggest that a syngeneic host suggesting that the aimed
biosensor strategy is feasible accepts the skin progeni-
tor graft.
In collaboration with VTT we have also developed the
electronic unit that has the capacity to measure the
changes in the skin basal progenitor cell integrated
sensor. We are currently in a process of filing a patent
of these biotechnological avenues with VTT.
Developing an ex vivo supernatural personal mobile
biosensor device. To advance the goal to develop
wearable sensory devise we started to assemble first
via a HILLA funded project a micro fluidistic set up
that will be converted to a bio recognition tool. During
the research period several micro fluidistic prints were
planned, made and tested. Out of these a configuration
was obtained that collected successfully the skin asso-
ciated fluids as depicted by the presence of color dye in
the fluidistic chamber (Figure 38). A patent search of
the strategy has been conducted.
INFOTECH OULU Annual Report 2015 15
We also developed capacity to the micro fluidistic set
up to monitor specific biomolecules present in the skin
fluids. We are currently advancing the research line
and aim to obtain more throughput capacity. To
achieve this we are planning and printing array format
fluidistic champers. In parallel to these developments
to be able to read the fluorescence that is revealed by
specific antibodies against certain factors is currently
developed in collaboration with VTT. A mobile phone
based micro fluidistic reader was achieved and the
reliability of the phone based reading with associated
programs was developed during 2015 there in.
Screening of electromagnetic and optogenetic respons-
es in organs generated from stem cells. The genetic
engineering offers opportunities to developed technol-
ogies where the cellular in or output signals can be
regulated by certain wavelengths in the electromagnetic
spectrum. Alternatively the cellular actions can be
genetically constructed so that a signal will be trans-
mitted to a biosensor that will convert it to a form read-
able by an electric device.
Figure 38. A micro fluidistic print design is able to col-lect the skin-associated fluids as depicted by the accu-mulation of a blue indicator dye in the chamber.
During 2015 we developed novel tissue engineering
technologies that do enable introduction of specific
gene expression constructs to individual cells of the
model organ such as the mammalian kidney. Here the
organ primordia is dissociated to single cells, the genet-
ic construct encoding the protein of interest such as the
optogenetic or the radio responsive component is
transduced to such a cell with a reporter for read out.
There after the organ is let to self-assemble and placed
for a long-term culture (Figure 39).
Figure. 39. An organ primordia can be dissociated to single cells, the constitute cells transduced with a ge-netic construct to acquire optogenetic and radio genetic guidance capacity to the morphogenetic cells ex vivo.
With the developed model systems we have taken use
of the image analysis technologies to visualize how the
morphogenetically active cells behave in three dimen-
sion (3D). To achieve this we applied defined pressure
to the assembled organ primordia in ex vivo setting.
We found that under a defined pressure the organ flat-
tens towards two dimension (2D) but yet morphogene-
sis progressed (Figure 40). This novel set up has made
it possible follow the fate of individual cells is the cells
are constricting a detailed manner while the natural
form.
Figure 40. The 3D kidney organ primordia develops under a mild pressurize in ex vivo conditions converted towards the 2D configuration. The setup has enabled to identify pressure sensors in the cells and also to develop novel organ pressure monitoring tools.
To target the detailed dynamics by which the form is
assembled in a model organ we took use of the genet-
ically engineered Wnt4CreGFP knock in mouse model.
This was crossed to the floxed Rosa26 Yellow Fluores-
cent Protein (YFP) transgenic mice. In this genetic
crossing the stem cells that generate whole of the neph-
ron will become labeled with the YFP.
We have captured 3D movies from the developing
kidney with the confocal microscope in a time-lapse
INFOTECH OULU Annual Report 2015 16
setting. We are in a process of analyzing the detailed
cell behavior via the machine learning/computer based
image analysis with Prof. Janne Heikkilä. With Dr. Jari
Juuti we aim to construct a specific device that allows
detailed measure of the pressure forced encountered by
the tissue undergoing morphogenesis. These novel
capabilities now allow analysis in great detail the mode
by which the spatial and temporal organization of the
cells go on to construct natural form that is open at
present in any developing organ system. We will use
models to identify the pressure sensors from the cells
with the OMICS technologies.
Developing high throughput robotic aided platforms to
screen complex cellular responses to magnetic/electric
fields via signaling pathway reporters. To advance the
strategies to measure in a high through put manner the
cellular responses to stimuli we have assembled a bio
robotic workstation. Here an Operetta confocal micro-
scope was obtained and this was coupled to a hood that
contains an automated plate-cargo arm, a rack for the
plates with a bar code reader and incubator for long
term exposure of the cells to compounds such as drugs
or specific electromagnetic spectral radiation (Figure
41). The Operetta confocal microscope has machine
learning/image analysis capacity for wealth of meas-
urements to be conducted from the cells.
Figure 41. Operetta confocal workstation coupled to a robotic set up and an incubator was assembled. A) A holder for plates and transported by the robotic arm (B) and the cells with in will be transported to an incubator (C ). The whole set up is inside a hood (D) and the robotic arm transports the plates to the Operetta confo-cal semi-high throughout microscope fluorescent read-er. The data is analyzed by wealth of machine vi-sion/image analysis programs present with in the as-sembled bio robotic set up. The bio robotic core facility will be used to screen with a library of live indicators cellular response to specific frequencies in the electro-magnetic spectra.
To take use of the set up a yeast cell library was ob-
tained and three replica clones from it was generated
and stored for later use. The library is composed of cell
where each of the 3´end of each of the yeast gene was
targeted by a green fluorescent protein (GFP) tag. The
next goal is to obtain capacity to start to use the set up
to define the oscillating properties of the cellular genes
and to use it as live measures for screening responses
to stimuli such as those mediated by the opsins for the
visible light frequencies.
Intelligent Systems with cohort data sets: Cohort data
set is a special data set from the medical domain, which
has not been studied with a machine learning approach
before. The data set, Northern Finland Birth Cohort
1966 (NFBC 1966), is a unique data set with over 14
000 original variables in various yet heterogeneous
formats (numerical, ordinal, categorical, images, text
etc.) from a population of over 12 000 mothers and
their children without any complete data points. The
amount of variables raises to millions if genetics and
epigenetics are considered (p >> n).
There are two extremely important aspects of modeling
this type of data: confidence of the predictions made
with the model and model interpretability. Steps to-
wards instance level confidence estimates have been
made in our previous work (see above) and we will
continue to pursue this goal, along with keeping model
interpretability in focus also, when we start digging
into this fascinating data set. Our goal is to use a ma-
chine learning approach to make novel discoveries
from the data that traditional data analysis approach
has not yet uncovered.
Elders are an increasingly large fraction of the popula-
tion in developed countries. From one hand people
expect an independent life also in presence of more or
less important diseases. On the other hand the treat-
ments to care those diseases, often together with co-
morbidities, imply larger costs. To respond to both
these goals, the disease progress should be kept as low
as possible (see Figure 42), which means early disease
detection, deinstitutionalisation and personalised medi-
cine, striving to allow a better quality of life, a more
cost-efficient healthcare system and a more inclusive
access to healthcare both in developing countries and
in remote areas in developed countries.
0
Healthy state
1
Degeneration starts,
no noticeable impact
on everyday life
2
Mild impacts on
everyday life
3
Disturbs appear
evident or important
4
Severe degeneration
D
Death (complications,
accidents, suicide)
C
Daily care needed
B
Assistance needed
A
Normal life (almost)
preserved
D
Medical
Doctor
H
Hospital
Active
H
osp
ita
lise
d
Cost-
eff
ective
E
xp
ensiv
e
Figure 42. Progress of a disease (left), outcome (right) and access to healthcare. (Celentano and Röning 2015.)
INFOTECH OULU Annual Report 2015 17
Novel Bio-ICT technologies are needed to achieve
these targets and BISG is active in this area in many
fronts summarised below.
By tracking health status of large groups and including
in the analysis a wealth of metrics and parameters,
large amounts of data are generated. On the other hand,
by downscaling biology-based technologies down to
the nanoscale including sensing biological parameters
directly from living cells, potential security threats are
correspondingly moving into human bodies, but prom-
ising tools are offered for personalised medicine and
treatments, including tight biological interaction, pros-
theses and their control (Celentano and Röning 2015).
BISG is strong in all these areas (data analysis, security
and robotics) and it is therefore pushing itself among
the world leaders in this growingly important area.
Towards a Holistic Self-awareness in Humans and AI
Self-awareness is involved in a number of cognitive
functions of the human brain and correspondingly its
disturbances are part of a number of disease of various
statistical relevance. Self-awareness may also improve
the efficiency in robotic systems by a more conscious
execution of tasks (Celentano and Röning 2016).
Interaction among concurrent cognitive entities further
expands the model in Celentano (2014). For multi-
robot systems, various cognitive and social inputs can
be used for self/nonself discrimination, including the
observation of the self, of the environment and of
neighbour entities, as well as the exploitation of social
interaction among agents (Figure 43).
Figure 43. Self-awareness through observation of the self and of the environment, top, and communication among agents, bottom. Even if the final outcome (e.g., relative positions) may look similar, it is important to discriminate what I am doing from what the others are doing. Sharing knowledge among agents further im-proves awareness. (Celentano and Röning 2016.)
In line with BISG strategy, cognitive functions in hu-
mans and in artificial entities are in this research col-
laboratively studied side by side to allow exploiting the
results in both domains bridging neuroscience and
artificial intelligence.
Exploitation of Results
Within the framework of its Bio-IT theme, see above,
BISG has recently started a cooperation with the SpA-
