Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | kathleen-simpson |
View: | 223 times |
Download: | 2 times |
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Humanoids, Brain and Cognitive
SciencesGiulio SandiniItalian Institute of TechnologyRobotics, Brain and Cognitive Sciences Dept.andLIRA-Lab, University of Genoa
Information and Communication Technology Research Forum 2009Khalifa University of Science, Technology and ResearchApril 5, 2009 – Abu Dhabi
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Humanoids may have to do with “automation”, service robotics, rescue and intervention but what do they have to do with “communication technologies”?
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
I am going to argue that communication is at the very core of humanoid robotics and that research on humanoids can help advancing communication technologies
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Communication not in the sense of “transmission” but in the sense of “mutual understanding” and team work.
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Today’s humans are operating artificial systems by pushing, pulling, sliding, turning, dragging, and, to a limited extent, speaking and gesturing
Artificial systems do not understand (anticipate) the intention of the human operator.
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Conversely communication in humans is a process of mutual understanding.
At both ends of a communication channel there is always a human being
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Why there is still no communication channel involving at least one artificial system at one terminal?
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Because to understand each other two agents need to share not only the communication channel but also knowledge and experience (and to some extent also the body shape)
The body is human’s communication channel
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
…humanoid robots could be the equivalent of today’s visual displays in the sense of “conveying information” but with “mutual understanding”
…in the future…
humanoids as communication terminals (with the added value that humans and humanoids will be able to act together)
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
..future of Humanoid Robots
Copyright: Toyota Motor Corporation – Robot Technologies, 2007
Close interactions with humans supported by natural multimodal communication channels such as gesture, vision, body-language, haptics, speech and natural language
…and its value chain
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Ubiquitous Robot Companion
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
For this we have to build systems that are able to communicate with humans in a natural way
We have to build systems that “understand” what a human is doing and, to some extent, are able to predict what a human will be doing in the immediate future
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
This brings about the issue of “Brain Sciences”: to implement systems that understand what humans are doing requires a lot of “brain research” to go along with “humanoid research”
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
How does a human being understands what another human being is doing?
How does a humanoid understands what a human being is doing?
Brain Science
Robotics Science
Cognitive
Science
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Humanoids Research
Humanoid robotics research today can be seen as a human centered discipline advancing science and developing new technologies along three main streams
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Human Centered Technologies
1. Build state-of-the-art humanoids2. Study humans (i.e. perceptual,
motor, learning, communication abilities)
3. Exploit human-machine interaction
Build Humanoids
Study Humans Interaction
Italian Institute of Technology
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Why a body?
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
The chess player…
Up to a few years ago the prevalent opinion was that it was possible to study the “human system” by investigating its “sub-systems” separately (visual, auditory, motor etc.) and that it was possible to separate perception and reasoning from the physics and mechanics of the body the chess player
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
The if-then-else approach
The “mind” as a logical machine:
If there is an obstacle in front then turn else go ahead.
Is it possible to arrive always of a logical decision?Is it worth to look always for a logical decision?
In the real world the possible alternatives may be so numerous that it is not possible to find a “logical” solution in time…
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Obstacle
Cow-catcher
What is an obstacle? Depends on the bodyWhat does it means “avoid”? Depends on the body
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Our understanding and our decisions depend in many different ways on the body and the actions that our body is able to perform
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
A point in space is represented by:
• Where it is• In the visual space• In the somatosensory (body) space• In the auditory space
• How it can be reached– With hands– With eyes– …
• …
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Object properties can be coded in terms of motor
primitives• Position of objects can be coded in terms of the
action required to reach it• Trajectory of objects can be coded in terms of
“collision trajectory” (which body part is going to hit)• Size can be coded in terms of “grasp type” (small is
whatever can be grasped with a pinch grasp)
One can express size with gestures: how big it is?
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Recognize an object without the object
• How relevant is “action” in recognizing objects?
• Can the visual system exploit the view of haptic exploration to extract shape information of the explored object?
Study done with Francesco Campanella
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Vision as an embodied process
Understanding as a perceptual process
Understanding with your eyes
Understanding as a motor process (goal-driven)
Understanding with your bottom
A chair is whatever gives a goal to my sitting action
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
F5 Mirror NeuronsActive when another individual is seen performing manipulative gestures
From: Fadiga, L., L. Fogassi, V. Gallese, and G. Rizzolatti, Visuomotor Neurons: ambiguity of the discharge or "motor“ Perception? Internation Journal of Psychophysiology, 2000. 35: p. 165-177.
SII7b
ITSTs
F1
F4AIPVIP
Reaching
Grasping
Object/hand recognition
Visualcortex
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Understanding actions performed by others means to activate the areas of the brain that are activated when we execute the same actions
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Being able (learning) to act is essential to understand actions
done by others.
Understanding mirror neurons: a bio-robotic approach. G. Metta, G. Sandini, L. Natale, L. Craighero, L. Fadiga. Interaction Studies. Volume 7 Issue 2. 2006
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Perception provides the essential information about
the context of the action
Which are the actions compatible with the current context?
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Through the context it is possible to predict the future course of events…
Object’s representation also contains “actions”
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
These multimodal motor representations need to be
learned and adapted continuously…
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
…by active exploration and interaction
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Our Guiding Philosophy
It is essential to study not only how the system works but also how it is constructed.
Cognition is necessarily the product of a process of embodied development (cannot be hand-coded )
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Developmental approach
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
The “process of building” a human being is based on learning thought interaction with the environment and with other
human beingsShaping the self
Interacting with others
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Learning to control gaze
1998
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
RobotCub
U. Genoa IIT
HerthfordshireIST - Lisbon U. Salford
Telerobot U. Ferrara
EPFL - Lausanne U. UppsalaU. Zurich
Scuola S. Anna
Robotcub is a collaborative 5-years project in the area of “cognitive systems”
We are now in the middle of the 4th year
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
People• Giorgio Metta, David Vernon• Lorenzo Natale, Francesco Nori: Software, testing, calibration• Marco Maggiali, Marco Randazzo: firmware, DSP libraries, tactile sensing• Francesco Becchi, Paolo Pino, Giulio Maggiolo, Gabriele Careddu: design
and integration• Roberto Puddu, Gabriele Tabbita, Walter Fancellu: assembly• Darwin Caldwell Nikos Tsagarakis, William Hinojosa: legs and spine,
force/torque sensors• Bruno Bonino, Fabrizio Larosa, Claudio Lorini: electronics• Davide Dellepiane: wiring• Mattia Salvi: CAD maintenance• Alberto Zolezzi: managing quotes, orders and spare parts• Ricardo Beira, Luis Vargas, Miguel Praca: design of the head and face• Paul Fitzpatrick & Alessandro Scalzo: software middleware• Alberto Parmiggiani: joint level sensing• Alexander Schmitz: fingertips• Ravinder Dahiya: FET-PVDF tactile senors• Lorenzo Jamone: fingertips• Auke Ijspeert and Ludovic Righetti: simulation and initial torque
specification
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
The RobotCub Approach
• Scientific Stance:Cognition emerges through interactions (a body is essential).
• Research Methodology:Cognition is best studied through a programme of progressive development (gradual acquisition of a prospective capability)
• Research Strategy:Global scientific community is best served by creating an open platform and exploiting synergies
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
….RobotCub two goals…
Advance our understanding of several key issues in cognition
Realize a physical platform for embodied cognitive research
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
The iCub platform
•The iCub is a full humanoid robot sized as a three and half year-old child.•The total height is 104cm.•It has 53 degrees of freedom, including articulated hands to be used for manipulation and gesturing.•The robot will be able to crawl and sit and autonomously transition from crawling to sitting and vice-versa.•The robot is GPL/FDL: software, hardware, drawings, documentation, etc.
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Degrees of freedom
• Head: vergence required + 3 dof neck• Arms: 7 dof each
– 3 shoulder, elbow, 3 wrist
• Hands: 9 dof each ► 17 joints– 5 fingers ► underactuated
• Legs: 6 dof each– 3 hip, knee, 2 ankle
• Waist: 3 dof
Σ = 53 dof
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Body cover: concept
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
It is small…
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Sensors…
• Cameras
• Skin/tactile
• Gyros/inertial
• Microphones
640x480 color - Remote headVery small but otherwise standard
Small (0.67mm)Designing force/torque sensorFingertip 3-axis sensorJoint torque measurementConductive paint…QTC: rubber-like materialTension sensors?Organic FETs?
Standard, condenser electret
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Artificial sensorized skinPrinciple of operation
Multiple elements
Skin structure
By Marco Maggiali: IITand Giorgio Cannata: UNIGE
Patented
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Miniaturized electronics
Hand sensor sampling PIC-based card
Force/torque sensor fitted into the sensor
Wired with 25micron coated wires
Design and documentation
Control cards 4 times smaller than the commercial equivalent
Embedded electronics in a 6-axis sensor
With: Alexander Schmitz, Lorenzo NataleElectronics: Bruno Bonino, Fabrizio Larosa
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Fingertip
Outline
3D CAD Electrode fabrication
Silicon cover
Complete prototype
Electronics
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Hands are key!
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
WikiCVS
Part lists
Drawings
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
iCubs in Europe
13 iCubs are on the make
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
.....and we are planning for more....
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Yoga…
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
…ICub Drummer
Ludovic Righetti, Auke Jan Ijspeert: Programmable Central Pattern Generators: an Application to Biped Locomotion Control. ICRA 2006: 1585-1590
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
ICub helper
Peter Ford Dominey Laboratoire d'Etude des Mécanismes Cognitifs (EMC) Equipe Neuroscience Cognitive et Representations Multimodales (NCRM)Université Lumière Lyon 2
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
ICub Observer
Work by: Jonas Ruesch, Manuel Lopes, Alexandre Bernardino, Jonas Hornstein, José Santos Victor, Rolf Pfeifer – IST Lisbon and University of Zurich AI Lab.
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
…still a long way to go…
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
The challenges• Mindware:
– Machine learning with little human intervention (irrespective of the application but tailored to the AI application domain)
– To tackle problems like perception and control (and possibly the two together) …vision…(still not solved..)
– Robust sensorimotor coordination, manipulation, full-body control
• Bodyware:– Materials: stronger and lighter bones– Actuators: flexible, compliant, impact absorbing– Sensors: e.g. skin, but also more resolution, range, etc.– Computation: the brain of our robot is still bulky, wiring is
also a complex problem– Energy: for autonomy– Safety
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Conclusions
• Humanoids and Human research may be mutually supportive
• Perception is an “action” process• Critical mass and international collaboration
(exploit body similarity)• Communication is a key aspect of humanoid
research• Body materials will be a key aspect of humanoid
research
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Thanks!
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Form and function are correlated in biological systems
Physical growth and morphological change is an essential ingredient of nature’s smart solutions
…are we missing something?
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
“Natural” minds are shaped by interactions
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
In nature also the bodies are shaped by interactions
Lucia Galli, TRENDS in Neuroscience Vol. 25 No. 12 – December 2002
Micromechanical hypothesis of mosaic assembly
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Development and learning are associated to morphological changes at the musculoskeletal and at the neuronal level
In biological systems body and mind co-
develop
Is this unique of biological systems?
Can we think about “artificial growth”?
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Solutions in biological systems are always multi-technological
Intelligent manipulation depends as much on how the controller works (mind) as on the physical properties of the muscles
Rizzolatti discovery of Mirror Neurons: exploit motor representation for visual recognition
Bizzi e Mussa-Ivaldi theory of force fieldsExploit visco-elastic properties of muscles
SII
7b
ITSTs
F1
F4AIP
VIP
Reaching
Grasping
Object/hand recognition
Visualcortex
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
Exploit co-development of function and physical growth
Living materials: living in the sense that their physical characteristics are changed by functions (e.g. they become harder or softer, they change shape, they “grow” differently).
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini
…the missing dimension…?
Life science “Robotics”
Material Science
…so far we have attempted to study the adaptation of the mind but the body is fixed…
Robotics, Brain and Cognitive Sciences DepartmentGiulio Sandini