Lecture Notes in Computer Science 11728
Founding Editors
Gerhard GoosKarlsruhe Institute of Technology, Karlsruhe, Germany
Juris HartmanisCornell University, Ithaca, NY, USA
Editorial Board Members
Elisa BertinoPurdue University, West Lafayette, IN, USA
Wen GaoPeking University, Beijing, China
Bernhard SteffenTU Dortmund University, Dortmund, Germany
Gerhard WoegingerRWTH Aachen, Aachen, Germany
Moti YungColumbia University, New York, NY, USA
More information about this series at http://www.springer.com/series/7407
Igor V. Tetko • Věra Kůrková •
Pavel Karpov • Fabian Theis (Eds.)
Artificial Neural Networksand Machine Learning –
ICANN 2019
Deep Learning
28th International Conference on Artificial Neural NetworksMunich, Germany, September 17–19, 2019Proceedings, Part II
123
EditorsIgor V. TetkoHelmholtz Zentrum München - DeutschesForschungszentrum für Gesundheitund Umwelt (GmbH)Neuherberg, Germany
Věra KůrkováInstitute of Computer ScienceCzech Academy of SciencesPrague 8, Czech Republic
Pavel KarpovHelmholtz Zentrum München - DeutschesForschungszentrum für Gesundheitund Umwelt (GmbH)Neuherberg, Germany
Fabian TheisHelmholtz Zentrum München - DeutschesForschungszentrum für Gesundheitund Umwelt (GmbH)Neuherberg, Germany
ISSN 0302-9743 ISSN 1611-3349 (electronic)Lecture Notes in Computer ScienceISBN 978-3-030-30483-6 ISBN 978-3-030-30484-3 (eBook)https://doi.org/10.1007/978-3-030-30484-3
LNCS Sublibrary: SL1 – Theoretical Computer Science and General Issues
© Springer Nature Switzerland AG 2019This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of thematerial is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,broadcasting, reproduction on microfilms or in any other physical way, and transmission or informationstorage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology nowknown or hereafter developed.The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoes not imply, even in the absence of a specific statement, that such names are exempt from the relevantprotective laws and regulations and therefore free for general use.The publisher, the authors and the editors are safe to assume that the advice and information in this book arebelieved to be true and accurate at the date of publication. Neither the publisher nor the authors or the editorsgive a warranty, expressed or implied, with respect to the material contained herein or for any errors oromissions that may have been made. The publisher remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AGThe registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
The fast development of machine learning methods is influencing all aspects of our lifeand reaching new horizons of what we have previously considered being ArtificialIntelligence (AI). Examples include autonomous car driving, virtual assistants,automated customer support, clinical decision support, healthcare data analytics,financial forecast, and smart devices in the home, to name a few, which contribute tothe dramatic improvement in the quality of our lives. These developments, however,also bring risks for significant hazards, which were not imaginable previously,e.g., falsification of voice, videos, or even manipulation of people’s opinions duringelections. Many such developments become possible due to the appearance of largevolumes of data (“Big Data”). These proceedings include the theory and applications ofalgorithms behind these developments, many of which were inspired by the functioningof the brain.
The International Conference on Artificial Neural Networks (ICANN) is the annualflagship conference of the European Neural Network Society (ENNS). The 28thInternational Conference on Artificial Neural Networks (ICANN 2019) wasco-organized with the final conference of the Marie Skłodowska-Curie InnovativeTraining Network European Industrial Doctorate “Big Data in Chemistry” (http://bigchem.eu) project coordinated by Helmholtz Zentrum München (GmbH) to promotethe use of machine learning in Chemistry. The conference featured the main tracks“Brain-Inspired Computing” and “Machine Learning Research.” Within the conferencethe First International Workshop on Reservoir Computing as well as five specialsessions were organized, namely:
Artificial Intelligence in MedicineInformed and Explainable Methods for Machine LearningDeep Learning in Image ReconstructionMachine Learning with Graphs: Algorithms and ApplicationsBIGCHEM: Big Data and AI in chemistry
A Challenge for Automatic Dog Age Estimation (DogAge) also took place as partof the conference. The conference covered all main research fields dealing with neuralnetworks. ICANN 2019 was held during September 17–19, 2019, at Klinikum rechtsder Isar der Technische Universität München, Munich, Germany.
Following a long-standing tradition, the proceedings of the conference werepublished as Springer volumes belonging to the Lecture Notes in Computer Scienceseries. The conference had a historical record of 494 article submissions. The paperswent through a two-step peer-review process by at least two and in majority of cases bythree or four independent referees. In total, 503 Program Committee (PC) members andreviewers participated in this process. The majority of PC members had Doctoraldegrees (88%) and 52% of them were also Professors. These reviewers were assigned46 articles. The others were PhD students in the last years of their studies, who
reviewed one to two articles each. In total, for the 323 accepted articles, 975 and 985reports were submitted for the first and the second revision sessions. Thus, on average,each accepted article received 6.1 reports. A list of reviewers/PC Members, who agreedto publish their names, are included in these proceedings.
Based on the reviewers’ comments, 202 articles were accepted and more than 100articles were rejected after the first review. The remaining articles received anundecided status. The authors of the accepted articles as well as of those withundecided status were requested to address the reviewers’ comments within two weeks.On the basis of second reviewers’ feedback, another 121 articles were accepted and theauthors were requested to include reviewers’ remarks into the final upload. Based onthese evaluations, diversity of topics, as well as recommendations of reviewers, specialsession organizers, and PC Chairs, 120 articles were selected for oral presentations. Outof the total number of 323 accepted articles (65% of initially submitted), 46manuscripts were short articles with a length of five pages each, while the others werefull articles with an average length of 13 pages.
The accepted papers of the 28th ICANN conference were published as five volumes:
Volume I Theoretical Neural ComputationVolume II Deep LearningVolume III Image ProcessingVolume IV Text and Time series analysisVolume V Workshop and Special Sessions
The authors of accepted articles came from 50 different countries. While themajority of the articles were from academic researchers, the conference also attractedcontributions from manifold industries including automobile (Volkswagen, BMW,Honda, Toyota), multinational conglomerates (Hitachi, Mitsubishi), electronics(Philips), electrical systems (Thales), mobile (Samsung, Huawei, Nokia, Orange),software (Microsoft), multinational (Amazon) and global travel technology (Expedia),information (IBM), large (AstraZeneca, Boehringer Ingelheim) and medium (IdorsiaPharmaceuticals Ltd.) pharma companies, fragrance and flavor (Firmenich),architectural (Shimizu), weather forecast (Beijing Giant Weather Co.), robotics(UBTECH Robotics Corp., SoftBank Robotics Group Corp.), contract researchorganization (Lead Discovery Center GmbH), private credit bureau (Schufa), as well asmultiple startups. This wide involvement of companies reflects the increasing use ofartificial neural networks by the industry. Five keynote speakers were invited to givelectures on the timely aspects of intelligent robot design (gentle robots), nonlineardynamical analysis of brain activity, deep learning in biology and biomedicine,explainable AI, artificial curiosity, and meta-learning machines.
These proceedings provide a comprehensive and up-to-date coverage of thedynamically developing field of Artificial Neural Networks. They are of major interestboth for theoreticians as well as for applied scientists who are looking for new
vi Preface
innovative approaches to solve their practical problems. We sincerely thank theProgram and Steering Committee and the reviewers for their invaluable work.
September 2019 Igor V. TetkoFabian TheisPavel KarpovVěra Kůrková
Preface vii
Organization
General Chairs
Igor V. Tetko Helmholtz Zentrum München (GmbH), GermanyFabian Theis Helmholtz Zentrum München (GmbH), Germany
Honorary Chair
Věra Kůrková(ENNS President)
Czech Academy of Sciences, Czech Republic
Publication Chair
Pavel Karpov Helmholtz Zentrum München (GmbH), Germany
Local Organizing Committee Chairs
Monica Campillos Helmholtz Zentrum München (GmbH), GermanyAlessandra Lintas University of Lausanne, Switzerland
Communication Chair
Paolo Masulli Technical University of Denmark, Denmark
Steering Committee
Erkki Oja Aalto University, FinlandWlodzislaw Duch Nicolaus Copernicus University, PolandAlessandro Villa University of Lausanne, SwitzerlandCesare Alippi Politecnico di Milano, Italy, and Università della
Svizzera italiana, SwitzerlandJérémie Cabessa Université Paris 2 Panthéon-Assas, FranceMaxim Fedorov Skoltech, RussiaBarbara Hammer Bielefeld University, GermanyLazaros Iliadis Democritus University of Thrace, GreecePetia Koprinkova-Hristova Bulgarian Academy of Sciences, BulgariaAntonis Papaleonidas Democritus University of Thrace, GreeceJaakko Peltonen University of Tampere, FinlandAntonio Javier Pons Rivero Universitat Politècnica de Catalunya, SpainYifat Prut The Hebrew University Jerusalem, IsraelPaul F. M. J. Verschure Catalan Institute of Advanced Studies, SpainFrancisco Zamora-Martínez Veridas Digital Authentication Solutions SL, Spain
Program Committee
Nesreen Ahmed Intel Labs, USANarges Ahmidi Helmholtz Zentrum München (GmbH), GermanyTetiana Aksenova Commissariat à l’énergie atomique et aux énergies
alternatives, FranceElie Aljalbout Technical University Munich, GermanyPiotr Antonik CentraleSupélec, FranceJuan Manuel
Moreno-ArosteguiUniversitat Politècnica de Catalunya, Spain
Michael Aupetit Qatar Computing Research Institute, QatarCristian Axenie Huawei German Research Center Munich, GermanyDavide Bacciu University of Pisa, ItalyNoa Barbiro Booking.com, IsraelIgor Baskin Moscow State University, RussiaChristian Bauckhage Fraunhofer IAIS, GermanyCostas Bekas IBM Research, SwitzerlandBarry Bentley The Open University, UKDaniel Berrar Tokyo Institute of Technology, JapanSoma Bhattacharya Expedia, USAMonica Bianchini Università degli Studi di Siena, ItalyFrançois Blayo NeoInstinct, SwitzerlandSander Bohte Centrum Wiskunde & Informatica, The NetherlandsAndrás P. Borosy QualySense AG, SwitzerlandGiosuè Lo Bosco Universita’ di Palermo, ItalyFarah Bouakrif University of Jijel, AlgeriaLarbi Boubchir University Paris 8, FranceMaria Paula Brito University of Porto, PortugalEvgeny Burnaev Skoltech, RussiaMikhail Burtsev Moscow Institute of Physics and Technology, RussiaJérémie Cabessa Université Panthéon Assas (Paris II), FranceFrancisco de Assis Tenório
de CarvalhoUniversidade Federal de Pernambuco, Brazil
Wolfgang Graf zuCastell-Ruedenhausen
Helmholtz Zentrum München (GmbH), Germany
Stephan Chalup University of Newcastle, AustraliaHongming Chen AstraZeneca, SwedenArtem Cherkasov University of British Columbia, CanadaSylvain Chevallier Université de Versailles, FranceVladimir Chupakhin Janssen Pharmaceutical Companies, USADjork-Arné Clevert Bayer, GermanyPaulo Cortez University of Minho, PortugalGennady Cymbalyuk Georgia State University, USAMaximilien Danisch Pierre and Marie Curie University, FranceTirtharaj Dash Birla Institute of Technology and Science Pilani, IndiaTyler Derr Michigan State University, USA
x Organization
Sergey Dolenko Moscow State University, RussiaShirin Dora University of Amsterdam, The NetherlandsWerner Dubitzky Helmholtz Zentrum München (GmbH), GermanyWlodzislaw Duch Nicolaus Copernicus University, PolandUjjal Kr Dutta Indian Institute of Technology Madras, IndiaMohamed El-Sharkawy Purdue School of Engineering and Technology, USAMohamed Elati Université de Lille, FranceReda Elbasiony Tanta University, EgyptMark Embrechts Rensselaer Polytechnic Institute, USASebastian Engelke University of Geneva, SwitzerlandOla Engkvist AstraZeneca, SwedenManfred Eppe University of Hamburg, GermanyPeter Erdi Kalamazoo College, USAPeter Ertl Novartis Institutes for BioMedical Research,
SwitzerlandIgor Farkaš Comenius University in Bratislava, SlovakiaMaxim Fedorov Skoltech, RussiaMaurizio Fiasché F-engineering Consulting, ItalyMarco Frasca University of Milan, ItalyBenoît Frénay Université de Namur, BelgiumClaudio Gallicchio Università di Pisa, ItalyUdayan Ganguly Indian Institute of Technology at Bombay, IndiaTiantian Gao Stony Brook University, USAJuantomás García Sngular, SpainJosé García-Rodríguez University of Alicante, SpainErol Gelenbe Institute of Theoretical and Applied Informatics,
PolandPetia Georgieva University of Aveiro, PortugalSajjad Gharaghani University of Tehran, IranEvgin Goceri Akdeniz University, TurkeyAlexander Gorban University of Leicester, UKMarco Gori Università degli Studi di Siena, ItalyDenise Gorse University College London, UKLyudmila Grigoryeva University of Konstanz, GermanyXiaodong Gu Fudan University, ChinaMichael Guckert Technische Hochschule Mittelhessen, GermanyBenjamin Guedj Inria, France, and UCL, UKTatiana Valentine Guy Institute of Information Theory and Automation,
Czech RepublicFabian Hadiji Goedle.io, GermanyAbir Hadriche University of Sfax, TunisiaBarbara Hammer Bielefeld University, GermanyStefan Haufe ERC Research Group Leader at Charité, GermanyDominik Heider Philipps-University of Marburg, GermanyMatthias Heinig Helmholtz Zentrum München (GmbH), GermanyChristoph Henkelmann DIVISIO GmbH, Germany
Organization xi
Jean Benoit Héroux IBM Research, JapanChristian Hidber bSquare AG, SwitzerlandMartin Holeňa Institute of Computer Science, Czech RepublicAdrian Horzyk AGH University of Science and Technology, PolandJian Hou Bohai University, ChinaLynn Houthuys Thomas More, BelgiumBrian Hyland University of Otago, New ZealandNicolangelo Iannella University of Oslo, NorwayLazaros Iliadis Democritus University of Thrace, GreeceFrancesco Iorio Wellcome Trust Sanger Institute, UKOlexandr Isayev University of North Carolina at Chapel Hill, USAKeiichi Ito Helmholtz Zentrum München (GmbH), GermanyNils Jansen Radboud University Nijmegen, The NetherlandsNoman Javed Université d’Orléans, FranceWenbin Jiang Huazhong University of Science and Technology,
ChinaJan Kalina Institute of Computer Science, Czech RepublicArgyris Kalogeratos Université Paris-Saclay, FranceMichael Kamp Fraunhofer IAIS, GermanyDmitry Karlov Skoltech, RussiaPavel Karpov Helmholtz Zentrum München (GmbH), GermanyJohn Kelleher Technological University Dublin, IrelandAdil Mehmood Khan Innopolis, RussiaRainer Kiko GEOMAR Helmholtz-Zentrum für Ozeanforschung,
GermanyChristina Klüver Universität Duisburg-Essen, GermanyTaisuke Kobayashi Nara Institute of Science and Technology, JapanEkaterina Komendantskaya University of Dundee, UKPetia Koprinkova-Hristova Bulgarian Academy of Sciences, BulgariaIrena Koprinska University of Sydney, AustraliaConstantine Kotropoulos Aristotle University of Thessaloniki, GreeceIlias Kotsireas Wilfrid Laurier University, CanadaAthanasios Koutras University of Peloponnese, GreecePiotr Kowalski AGH University of Science and Technology, PolandValentin Kozlov Karlsruher Institut für Technologie, GermanyDean J. Krusienski Virginia Commonwealth University, USAAdam Krzyzak Concordia University, CanadaHanna Kujawska University of Bergen, NorwayVěra Kůrková Institute of Computer Science, Czech RepublicSumit Kushwaha Kamla Nehru Institute of Technology, IndiaAnna Ladi Fraunhofer IAIS, GermanyWard Van Laer Ixor, BelgiumOliver Lange Google Inc., USAJiyi Li University of Yamanashi, JapanLei Li Beijing University of Posts and Telecommunications,
China
xii Organization
Spiros Likothanassis University of Patras, GreeceChristian Limberg Universität Bielefeld, GermanyAlessandra Lintas University of Lausanne, SwitzerlandViktor Liviniuk MIT, USA, and Skoltech, RussiaDoina Logofatu Frankfurt University of Applied Sciences, GermanyVincenzo Lomonaco Università di Bologna, ItalySock Ching Low Institute for Bioengineering of Catalonia, SpainAbhijit Mahalunkar Technological University Dublin, IrelandMufti Mahmud Nottingham Trent University, UKAlexander Makarenko National Technical University of Ukraine - Kiev
Polytechnic Institute, UkraineKleanthis Malialis University of Cyprus, CyprusFragkiskos Malliaros University of Paris-Saclay, FranceGilles Marcou University of Strasbourg, FranceUrszula
Markowska-KaczmarWroclaw University of Technology, Poland
Carsten Marr Helmholtz Zentrum München (GmbH), GermanyGiuseppe Marra University of Firenze, ItalyPaolo Masulli Technical University of Denmark, DenmarkSiamak Mehrkanoon Maastricht University, The NetherlandsStefano Melacci Università degli Studi di Siena, ItalyMichael Menden Helmholtz Zentrum München (GmbH), GermanySebastian Mika Comtravo, GermanyNikolaos Mitianoudis Democritus University of Thrace, GreeceValeri Mladenov Technical University of Sofia, BulgariaHebatallah Mohamed Università degli Studi Roma, ItalyFiglu Mohanty International Institute of Information Technology
at Bhubaneswar, IndiaFrancesco Carlo Morabito University of Reggio Calabria, ItalyJerzy Mościński Silesian University of Technology, PolandHenning Müller University of Applied Sciences Western Switzerland,
SwitzerlandMaria-Viorela Muntean University of Alba-Iulia, RomaniaPhivos Mylonas Ionian University, GreeceShinichi Nakajima Technische Universität Berlin, GermanyKohei Nakajima University of Tokyo, JapanChi Nhan Nguyen Itemis, GermanyFlorian Nigsch Novartis Institutes for BioMedical Research,
SwitzerlandGiannis Nikolentzos École Polytechnique, FranceIkuko Nishikawa Ritsumeikan University, JapanHarri Niska University of Eastern FinlandHasna Njah ISIM-Sfax, TunisiaDimitri Nowicki Institute of Cybernetics of NASU, UkraineAlessandro Di Nuovo Sheffield Hallam University, UKStefan Oehmcke University of Copenhagen, Denmark
Organization xiii
Erkki Oja Aalto University, FinlandLuca Oneto Università di Pisa, ItalySilvia Ortin Institute of Neurosciences (IN) Alicante, SpainIvan Oseledets Skoltech, RussiaDmitry Osolodkin Chumakov FSC R&D IBP RAS, RussiaSebastian Otte University of Tübingen, GermanyLatifa Oukhellou The French Institute of Science and Technology
for Transport, FranceVladimir Palyulin Moscow State University, RussiaGeorge Panagopoulos École Polytechnique, FranceMassimo Panella Università degli Studi di Roma La Sapienza, ItalyAntonis Papaleonidas Democritus University of Thrace, GreeceEvangelos Papalexakis University of California Riverside, USADaniel Paurat Fraunhofer IAIS, GermanyJaakko Peltonen Tampere University, FinlandTingying Peng Technische Universität München, GermanyAlberto Guillén Perales Universidad de Granada, SpainCarlos Garcia Perez Helmholtz Zentrum München (GmbH), GermanyIsabelle Perseil INSERM, FranceVincenzo Piuri University of Milan, ItalyKathrin Plankensteiner Fachhochschule Vorarlberg, AustriaIsabella Pozzi Centrum Wiskunde & Informatica, The NetherlandsMike Preuss Leiden University, The NetherlandsYifat Prut The Hebrew University of Jerusalem, IsraelEugene Radchenko Moscow State University, RussiaRajkumar Ramamurthy Fraunhofer IAIS, GermanySrikanth Ramaswamy Swiss Federal Institute of Technology (EPFL),
SwitzerlandBeatriz Remeseiro Universidad de Oviedo, SpainXingzhang Ren Alibaba Group, ChinaJean-Louis Reymond University of Bern, SwitzerlandCristian Rodriguez Rivero University of California, USAAntonio Javier Pons Rivero Universitat Politècnica de Catalunya, SpainAndrea Emilio Rizzoli IDSIA, SUPSI, SwitzerlandFlorian Röhrbein Technical University Munich, GermanyRyan Rossi PARC - a Xerox Company, USAManuel Roveri Politecnico di Milano, ItalyVladimir Rybakov WaveAccess, RussiaMaryam Sabzevari Aalto University School of Science and Technology,
FinlandJulio Saez-Rodriguez Medizinische Fakultät Heidelberg, GermanyYulia Sandamirskaya NEUROTECH: Neuromorphic Computer Technology,
SwitzerlandCarlo Sansone University of Naples Federico II, ItalySreela Sasi Gannon University, USABurak Satar Uludag University, Turkey
xiv Organization
Axel Sauer Munich School of Robotics and Machine Intelligence,Germany
Konstantin Savenkov Intento, Inc., USAHanno Scharr Forschungszentrum Jülich, GermanyTjeerd olde Scheper Oxford Brookes University, UKRafal Scherer Czestochowa University of Technology, PolandMaria Secrier University College London, UKThomas Seidl Ludwig-Maximilians-Universität München, GermanyRafet Sifa Fraunhofer IAIS, GermanyPekka Siirtola University of Oulu, FinlandPrashant Singh Uppsala University, SwedenPatrick van der Smagt Volkswagen AG, GermanyMaximilian Soelch Volkswagen Machine Learning Research Lab,
GermanyMiguel Cornelles Soriano Campus Universitat de les Illes Balears, SpainMiguel Angelo Abreu Sousa Institute of Education Science and Technology, BrazilMichael Stiber University of Washington Bothell, USAAlessandro Sperduti Università degli Studi di Padova, ItalyRuxandra Stoean University of Craiova, RomaniaNicola Strisciuglio University of Groningen, The NetherlandsIrene Sturm Deutsche Bahn AG, GermanyJérémie Sublime ISEP, FranceMartin Swain Aberystwyth University, UKZoltan Szabo Ecole Polytechnique, FranceKazuhiko Takahashi Doshisha University, JapanFabian Theis Helmholtz Zentrum München (GmbH), GermanyPhilippe Thomas Universite de Lorraine, FranceMatteo Tiezzi University of Siena, ItalyRuben Tikidji-Hamburyan Louisiana State University, USAYancho Todorov VTT, FinlandAndrei Tolstikov Merck Group, GermanyMatthias Treder Cardiff University, UKAnton Tsitsulin Rheinische Friedrich-Wilhelms-Universität Bonn,
GermanyYury Tsoy Solidware Co. Ltd., South KoreaAntoni Valencia Independent Consultant, SpainCarlos Magno Valle Technical University Munich, GermanyMarley Vellasco Pontifícia Universidade Católica do Rio de Janeiro,
BrazilSagar Verma Université Paris-Saclay, FrancePaul Verschure Institute for Bioengineering of Catalonia, SpainVarvara Vetrova University of Canterbury, New ZealandRicardo Vigário University Nova’s School of Science and Technology,
PortugalAlessandro Villa University of Lausanne, SwitzerlandBruno Villoutreix Molecular informatics for Health, France
Organization xv
Paolo Viviani Università degli Studi di Torino, ItalyGeorge Vouros University of Piraeus, GreeceChristian Wallraven Korea University, South KoreaTinghuai Wang Nokia, FinlandYu Wang Leibniz Supercomputing Centre (LRZ), GermanyRoseli S. Wedemann Universidade do Estado do Rio de Janeiro, BrazilThomas Wennekers University of Plymouth, UKStefan Wermter University of Hamburg, GermanyHeiko Wersing Honda Research Institute and Bielefeld University,
GermanyTadeusz Wieczorek Silesian University of Technology, PolandChristoph Windheuser ThoughtWorks Inc., GermanyBorys Wróbel Adam Mickiewicz University in Poznan, PolandJianhong Wu York University, CanadaXia Xiao University of Connecticut, USATakaharu Yaguchi Kobe University, JapanSeul-Ki Yeom Technische Universität Berlin, GermanyHujun Yin University of Manchester, UKJunichiro Yoshimoto Nara Institute of Science and Technology, JapanQiang Yu Tianjin University, ChinaShigang Yue University of Lincoln, UKWlodek Zadrozny University of North Carolina Charlotte, USADanuta Zakrzewska Technical University of Lodz, PolandFrancisco Zamora-Martínez Veridas Digital Authentication Solutions SL, SpainGerson Zaverucha Federal University of Rio de Janeiro, BrazilJunge Zhang Institute of Automation, ChinaZhongnan Zhang Xiamen University, ChinaPengsheng Zheng Daimler AG, GermanySamson Zhou Indiana University, USARiccardo Zucca Institute for Bioengineering of Catalonia, SpainDietlind Zühlke Horn & Company Data Analytics GmbH, Germany
Exclusive Platinum Sponsor for the Automotive Branch
xvi Organization
Keynote Talks
Recurrent Patterns of Brain ActivityAssociated with Cognitive Tasks and AttractorDynamics (John Taylor Memorial Lecture)
Alessandro E. P. Villa
NeuroHeuristic Research Group, University of Lausanne,Quartier UNIL-Chamberonne, 1015 Lausanne, Switzerland
[email protected]://www.neuroheuristic.org
The simultaneous recording of the time series formed by the sequences of neuronaldischarges reveals important features of the dynamics of information processing in thebrain. Experimental evidence of firing sequences with a precision of a few millisecondshave been observed in the brain of behaving animals. We review some critical findingsshowing that this activity is likely to be associated with higher order neural (mental)processes, such as predictive guesses of a coming stimulus in a complex sensorimotordiscrimination task, in primates as well as in rats. We discuss some models of evolvableneural networks and their nonlinear deterministic dynamics and how such complexspatiotemporal patterns of firing may emerge. The attractors of such networks corre-spond precisely to the cycles in the graphs of their corresponding automata, and canthus be computed explicitly and exhaustively. We investigate further the effects ofnetwork topology on the dynamical activity of hierarchically organized networks ofsimulated spiking neurons. We describe how the activation and thebiologically-inspired processes of plasticity on the network shape its topology usinginvariants based on algebro-topological constructions. General features of a braintheory based on these results is presented for discussion.
Unsupervised Learning: Passive and Active
Jürgen Schmidhuber
Co-founder and Chief Scientist, NNAISENSE, Scientific Director,Swiss AI Lab IDSIA and Professor of AI, USI & SUPSI, Lugano, Switzerland
I’ll start with a concept of 1990 that has become popular: unsupervised learningwithout a teacher through two adversarial neural networks (NNs) that duel in amini-max game, where one NN minimizes the objective function maximized by theother. The first NN generates data through its output actions while the second NNpredicts the data. The second NN minimizes its error, thus becoming a better predictor.But it is a zero sum game: the first NN tries to find actions that maximize the errorof the second NN. The system exhibits what I called “artificial curiosity” because thefirst NN is motivated to invent actions that yield data that the second NN still findssurprising, until the data becomes familiar and eventually boring. A similar adversarialzero sum game was used for another unsupervised method called “predictabilityminimization,” where two NNs fight each other to discover a disentangled code of theincoming data (since 1991), remarkably similar to codes found in biological brains. I’llalso discuss passive unsupervised learning through predictive coding of an agent’sobservation stream (since 1991) to overcome the fundamental deep learning problemthrough data compression. I’ll offer thoughts as to why most current commercialapplications don’t use unsupervised learning, and whether that will change in thefuture.
Machine Learning and AI for the Sciences—Towards Understanding
Klaus-Robert Müller
Machine Learning Group, Technical University of Berlin, Germany
In recent years machine learning (ML) and Artificial Intelligence (AI) methods havebegun to play a more and more enabling role in the sciences and in industry. Inparticular, the advent of large and/or complex data corpora has given rise to newtechnological challenges and possibilities.
The talk will connect two topics (1) explainable AI (XAI) and (2) ML applicationsin sciences (e.g. Medicine and Quantum Chemistry) for gaining new insight. Specifi-cally I will first introduce XAI methods (such as LRP) that are now readily availableand allow for an understanding of the inner workings of nonlinear ML methods rangingfrom kernel methods to deep learning methods including LSTMs. In particular XAIallows unmasking clever Hans predictors. Then, ML for Quantum Chemistry is dis-cussed, showing that ML methods can lead to highly useful predictors of quantummechanical properties of molecules (and materials) reaching quantum chemical accu-racies both across chemical compound space and in molecular dynamics simulations.Notably, these ML models do not only speed up computation by several orders ofmagnitude but can give rise to novel chemical insight. Finally, I will analyze mor-phological and molecular data for cancer diagnosis, also here highly interesting novelinsights can be obtained.
Note that while XAI is used for gaining a better understanding in the sciences, theintroduced XAI techniques are readily useful in other application domains and industryas well.
Large-Scale Lineage and Latent-SpaceLearning in Single-Cell Genomic
Fabian Theis
Institute of Computational Biology, Helmholtz Zentrum München (GmbH),Germany
http://comp.bio
Accurately modeling single cell state changes e.g. during differentiation or in responseto perturbations is a central goal of computational biology. Single-cell technologiesnow give us easy and large-scale access to state observations on the transcriptomic andmore recently also epigenomic level, separately for each single cell. In particular theyallow resolving potential heterogeneities due to asynchronicity of differentiating orresponding cells, and profiles across multiple conditions such as time points andreplicates are being generated.
Typical questions asked to such data are how cells develop over time and afterperturbation such as disease. The statistical tools to address these questions are tech-niques from pseudo-temporal ordering and lineage estimation, or more broadly latentspace learning. In this talk I will give a short review of such approaches, in particularfocusing on recent extensions towards large-scale data integration using single-cellgraph mapping or neural networks, and finish with a perspective towards learningperturbations using variational autoencoders.
The Gentle Robot
Sami Haddadin
Technical University of Munich, Germany
Enabling robots for interaction with humans and unknown environments has been oneof the primary goals of robotics research over decades. I will outline howhuman-centered robot design, nonlinear soft-robotics control inspired by human neu-romechanics and physics grounded learning algorithms will let robots become acommodity in our near-future society. In particular, compliant and energy-controlledultra-lightweight systems capable of complex collision handling enablehigh-performance human assistance over a wide variety of application domains.Together with novel methods for dynamics and skill learning, flexible and easy-to-userobotic power tools and systems can be designed. Recently, our work has led to the firstnext generation robot Franka Emika that has recently become commercially available.The system is able to safely interact with humans, execute and even learn sensitivemanipulation skills, is affordable and designed as a distributed interconnected system.
Contents – Part II
Feature Selection
Adaptive Graph Fusion for Unsupervised Feature Selection . . . . . . . . . . . . . 3Sijia Niu, Pengfei Zhu, Qinghua Hu, and Hong Shi
Unsupervised Feature Selection via Local Total-Order Preservation . . . . . . . . 16Rui Ma, Yijie Wang, and Li Cheng
Discrete Stochastic Search and Its Application to Feature-Selectionfor Deep Relational Machines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Tirtharaj Dash, Ashwin Srinivasan, Ramprasad S. Joshi, and A. Baskar
Joint Dictionary Learning for Unsupervised Feature Selection. . . . . . . . . . . . 46Yang Fan, Jianhua Dai, Qilai Zhang, and Shuai Liu
Comparison Between Filter Criteria for Feature Selection in Regression. . . . . 59Alexandra Degeest, Michel Verleysen, and Benoît Frénay
CancelOut: A Layer for Feature Selection in Deep Neural Networks . . . . . . . 72Vadim Borisov, Johannes Haug, and Gjergji Kasneci
Adaptive-L2 Batch Neural Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Nicomedes L. Cavalcanti Jr., Marcelo Rodrigo Portela Ferreira,and Francisco de Assis Tenorio de Carvalho
Application of Self Organizing Map to Preprocessing Input Vectorsfor Convolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Hiroshi Dozono and Masafumi Tanaka
Augmentation Techniques
Hierarchical Reinforcement Learning with Unlimited RecursiveSubroutine Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Yuuji Ichisugi, Naoto Takahashi, Hidemoto Nakada, and Takashi Sano
Automatic Augmentation by Hill Climbing. . . . . . . . . . . . . . . . . . . . . . . . . 115Ricardo Cruz, Joaquim F. Pinto Costa, and Jaime S. Cardoso
Learning Camera-Invariant Representation for Person Re-identification . . . . . 125Shizheng Qin, Kangzheng Gu, Lecheng Wang, Lizhe Qi,and Wenqiang Zhang
PA-RetinaNet: Path Augmented RetinaNet for Dense Object Detection . . . . . 138Guanghua Tan, Zijun Guo, and Yi Xiao
Weights Initialization
Singular Value Decomposition and Neural Networks . . . . . . . . . . . . . . . . . . 153Bernhard Bermeitinger, Tomas Hrycej, and Siegfried Handschuh
PCI: Principal Component Initialization for Deep Autoencoders . . . . . . . . . . 165Aiga Suzuki and Hidenori Sakanashi
Improving Weight Initialization of ReLU and Output Layers . . . . . . . . . . . . 170Diego Aguirre and Olac Fuentes
Parameters Optimisation
Post-synaptic Potential Regularization Has Potential . . . . . . . . . . . . . . . . . . 187Enzo Tartaglione, Daniele Perlo, and Marco Grangetto
A Novel Modification on the Levenberg-Marquardt Algorithmfor Avoiding Overfitting in Neural Network Training . . . . . . . . . . . . . . . . . 201
Serdar Iplikci, Batuhan Bilgi, Ali Menemen, and Bedri Bahtiyar
Sign Based Derivative Filtering for Stochastic Gradient Descent . . . . . . . . . . 208Konstantin Berestizshevsky and Guy Even
Architecture-Aware Bayesian Optimization for Neural Network Tuning . . . . . 220Anders Sjöberg, Magnus Önnheim, Emil Gustavsson, and Mats Jirstrand
Non-convergence and Limit Cycles in the Adam Optimizer . . . . . . . . . . . . . 232Sebastian Bock and Martin Weiß
Pruning Networks
Learning Internal Dense But External Sparse Structures of DeepConvolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
Yiqun Duan and Chen Feng
Using Feature Entropy to Guide Filter Pruning for EfficientConvolutional Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Yun Li, Luyang Wang, Sifan Peng, Aakash Kumar, and Baoqun Yin
Simultaneously Learning Architectures and Features of DeepNeural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Tinghuai Wang, Lixin Fan, and Huiling Wang
xxvi Contents – Part II
Learning Sparse Hidden States in Long Short-Term Memory . . . . . . . . . . . . 288Niange Yu, Cornelius Weber, and Xiaolin Hu
Multi-objective Pruning for CNNs Using Genetic Algorithm . . . . . . . . . . . . 299Chuanguang Yang, Zhulin An, Chao Li, Boyu Diao, and Yongjun Xu
Dynamically Sacrificing Accuracy for Reduced Computation: CascadedInference Based on Softmax Confidence . . . . . . . . . . . . . . . . . . . . . . . . . . 306
Konstantin Berestizshevsky and Guy Even
Light-Weight Edge Enhanced Network for On-orbitSemantic Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
Junxing Hu, Ling Li, Yijun Lin, Fengge Wu, and Junsuo Zhao
Local Normalization Based BN Layer Pruning . . . . . . . . . . . . . . . . . . . . . . 334Yuan Liu, Xi Jia, Linlin Shen, Zhong Ming, and Jinming Duan
Search for an Optimal Architecture
On Practical Approach to Uniform Quantization of Non-redundantNeural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Alexander Goncharenko, Andrey Denisov, Sergey Alyamkin,and Evgeny Terentev
Residual Learning for FC Kernels of Convolutional Network . . . . . . . . . . . . 361Alexey Alexeev, Yuriy Matveev, Anton Matveev, and Dmitry Pavlenko
A Novel Neural Network-Based Symbolic Regression Method:Neuro-Encoded Expression Programming. . . . . . . . . . . . . . . . . . . . . . . . . . 373
Aftab Anjum, Fengyang Sun, Lin Wang, and Jeff Orchard
Compute-Efficient Neural Network Architecture Optimizationby a Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
Sebastian Litzinger, Andreas Klos, and Wolfram Schiffmann
Controlling Model Complexity in Probabilistic Model-Based DynamicOptimization of Neural Network Structures . . . . . . . . . . . . . . . . . . . . . . . . 393
Shota Saito and Shinichi Shirakawa
Confidence Estimation
Predictive Uncertainty Estimation with Temporal ConvolutionalNetworks for Dynamic Evolutionary Optimization. . . . . . . . . . . . . . . . . . . . 409
Almuth Meier and Oliver Kramer
Contents – Part II xxvii
Sparse Recurrent Mixture Density Networks for Forecasting HighVariability Time Series with Confidence Estimates . . . . . . . . . . . . . . . . . . . 422
Narendhar Gugulothu, Easwar Subramanian, and Sanjay P. Bhat
A Multitask Learning Neural Network for Short-Term Traffic SpeedPrediction and Confidence Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
Yanyun Tao, Xiang Wang, and Yuzhen Zhang
Continual Learning
Central-Diffused Instance Generation Method in ClassIncremental Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Mingyu Liu and Yijie Wang
Marginal Replay vs Conditional Replay for Continual Learning . . . . . . . . . . 466Timothée Lesort, Alexander Gepperth, Andrei Stoian, and David Filliat
Simplified Computation and Interpretation of Fisher Matricesin Incremental Learning with Deep Neural Networks . . . . . . . . . . . . . . . . . . 481
Alexander Gepperth and Florian Wiech
Active Learning for Image Recognition Using a Visualization-BasedUser Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
Christian Limberg, Kathrin Krieger, Heiko Wersing, and Helge Ritter
Basic Evaluation Scenarios for Incrementally Trained Classifiers. . . . . . . . . . 507Rudolf Szadkowski, Jan Drchal, and Jan Faigl
Embedding Complexity of Learned Representations in Neural Networks . . . . 518Tomáš Kuzma and Igor Farkaš
Metric Learning
Joint Metric Learning on Riemannian Manifold of GlobalGaussian Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
Qinqin Nie, Bin Zhou, Pengfei Zhu, Qinghua Hu, and Hao Cheng
Multi-task Sparse Regression Metric Learningfor Heterogeneous Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
Haotian Wu, Bin Zhou, Pengfei Zhu, Qinghua Hu, and Hong Shi
Fast Approximate Geodesics for Deep Generative Models . . . . . . . . . . . . . . 554Nutan Chen, Francesco Ferroni, Alexej Klushyn, Alexandros Paraschos,Justin Bayer, and Patrick van der Smagt
Spatial Attention Network for Few-Shot Learning . . . . . . . . . . . . . . . . . . . . 567Xianhao He, Peng Qiao, Yong Dou, and Xin Niu
xxviii Contents – Part II
Routine Modeling with Time Series Metric Learning . . . . . . . . . . . . . . . . . . 579Paul Compagnon, Grégoire Lefebvre, Stefan Duffner,and Christophe Garcia
Domain Knowledge Incorporation
Leveraging Domain Knowledge for Reinforcement LearningUsing MMC Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
Rajkumar Ramamurthy, Christian Bauckhage, Rafet Sifa,Jannis Schücker, and Stefan Wrobel
Conditions for Unnecessary Logical Constraints in Kernel Machines . . . . . . . 608Francesco Giannini and Marco Maggini
HiSeqGAN: Hierarchical Sequence Synthesis and Prediction . . . . . . . . . . . . 621Yun-Chieh Tien, Chen-Min Hsu, and Fang Yu
DeepEX: Bridging the Gap Between Knowledge and Data DrivenTechniques for Time Series Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . 639
Muhammad Ali Chattha, Shoaib Ahmed Siddiqui, Mohsin Munir,Muhammad Imran Malik, Ludger van Elst, Andreas Dengel,and Sheraz Ahmed
Domain Adaptation Approaches
Transferable Adversarial Cycle Alignment for Domain Adaption. . . . . . . . . . 655Yingcan Wei
Evaluation of Domain Adaptation Approaches for Robust Classificationof Heterogeneous Biological Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 673
Michael Schneider, Lichao Wang, and Carsten Marr
Named Entity Recognition for Chinese Social Media with DomainAdversarial Training and Language Modeling. . . . . . . . . . . . . . . . . . . . . . . 687
Yong Xu, Qi Lu, and Muhua Zhu
Deep Domain Knowledge Distillation for Person Re-identification . . . . . . . . 700Junjie Yan
A Study on Catastrophic Forgetting in Deep LSTM Networks . . . . . . . . . . . 714Monika Schak and Alexander Gepperth
Multiclass
A Label-Specific Attention-Based Network with Regularized Lossfor Multi-label Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 731
Xiangyang Luo, Xiangying Ran, Wei Sun, Yunlai Xu,and Chongjun Wang
Contents – Part II xxix
An Empirical Study of Multi-domain and Multi-task Learningin Chinese Named Entity Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 743
Yun Hu, Mingxue Liao, Pin Lv, and Changwen Zheng
Filter Method Ensemble with Neural Networks . . . . . . . . . . . . . . . . . . . . . . 755Anuran Chakraborty, Rajonya De, Agneet Chatterjee,Friedhelm Schwenker, and Ram Sarkar
Dynamic Centroid Insertion and Adjustment for Data Setswith Multiple Imbalanced Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766
Evandro J. R. Silva and Cleber Zanchettin
Increasing the Generalisaton Capacity of Conditional VAEs . . . . . . . . . . . . . 779Alexej Klushyn, Nutan Chen, Botond Cseke, Justin Bayer,and Patrick van der Smagt
Playing the Large Margin Preference Game . . . . . . . . . . . . . . . . . . . . . . . . 792Mirko Polato, Guglielmo Faggioli, Ivano Lauriola, and Fabio Aiolli
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805
xxx Contents – Part II