Lec2 Human Capabilities

7/29/2019 Lec2 Human Capabilities

1/56

1

Chapter 2INS Interfaces

Announcement

Login name and password needed toaccess lecture notes on the coursewebsite User name: sysc5303

Password: BIOM5402


2/56

2

Note Completeness

The information given in this chapter shouldnot be regarded ascomplete or comprehensive, but rather as providing arepresentative sampling of some available results.

AccuracyThe field is active and fast-moving, therefore, the informationcontained in this chapter might have a short half-life. So,although the materials lectured are accurate for the time beingfrom our current knowledge, but some of them subsequently mayhave to be updated.

Consistency:While attempts will be made to preserve consistency whereverpossible, in some case the desired information was unavailableor available materials are inconsistent by themselves.

Definitions

Interfaces: In INS systems, they are thosedevices acting as a mediator (e.g., data gloves,exoskeleton, stereo glasses) through which thehuman operator can observe, sense, manipulateand control a remote physical dynamic system over a distance a physical system at micro- or nano- scale

a virtual simulated dynamic system.

Human operator: a person doing the observingand acting.

What kind of interfaces is ideal? The human operator does not feel its existence.


3/56

3

Part 1- Human I/O Channels

Visual, auditory and haptic

Introduction

Peoples research onhuman I/O has a longhistory

In Ancient China Human body was thought

to be made from gold,wood, water, fire and earth.

It was long believed thathuman thinks using heartrather than brain.

Aristotles concept Sense organs are made of

earth, air, water and fire. Five senses project to the

heart either directly orindirectly


4/56

4

Overview Input/sensory channels

Visual Auditory Haptic

Smell Taste The sixth channel/sense ?

Output channels

Verbal Haptic

hand, arm, body, leg, foot

Others: head, eye

Human Information Processing

Human three-stage processing of input information Reception: the process of accepting an energy stimulus from

the external world and translating the stimulus into a form whichcan then excite perception.

Perception: the encoding of the received energy into a neuralmessage which is then transferred to the appropriate braincenter which can achieve cognition.

Cognition: the assimilation of the information contained in theneural message with respect to data extracted from the memory,thus achieving identification based upon some aspect of pastexperience.

Man-machine: reception and perception

Reception Perception Cognition


5/56

5

Characterization of Sense Modalities

Each of human sense modalities can bemainly characterized by Type of accepted data

Sensitivity (Absolute and JND)

Temporal resolution

Spatial resolution

Information processing rate (bandwidth) Adaptation

Some other factors

1. Visual Channel


6/56

6

Visual Channel The most important single sense modality in

human gathering of information concerning hisrelation to the real world

Absorption of light energy by the eye and thesuccessive conversion of this energy into neuralmessages which are mediated by the brain intoperceptual patterns.

Wavelength range 0.3 - 0.7 microns (1micron=10-6meter)

2 eyeballs for binocular vision

Eyeball Approximately spherical, about 1 inch

(24~25 mm) in diameter

Black-looking aperture, the pupil, thatallows light to enter the eye (it appearsdark because of the absorbing pigmentsin the retina).

A colored circular muscle, the iris, which

is giving us our eye's color. This circularmuscle controls the size of the pupil sothat more or less light, depending onconditions, is allowed to enter the eye.

A transparent external surface, thecornea, that covers both the pupil and theiris. This is the first and most powerfullens of the optical system of the eye

The "white of the eye", the sclera, whichforms part of the supporting wall of theeyeball.


7/56

7

Eyeball Light rays are focused

and passed through thetransparent cornea andlens upon the retina.

The details of the imageare formed at the retinaand transmitted directlyto the brain for the higheroperations needed forperception and cognition.

The central point forimage focus (the visualaxis) in the human retinais the fovea.

Here image has thefinest detail

Eyeball


8/56

8

Eye Movement Capability of rotation:

To the left = to the right

Vertical upward movement:


9/56

9

Resolution of Human Visual

Systems Fact

When we watch a high-resolution digital movie, we donot perceive a series of still pictures that arepresented in sequence, nor do we apprehend anarray of colored pixels.

Instead, we perceive a visual scene that is close toeveryday visual experiences.

Reason The (temporal resolution) of the human visual system

is not sufficient to detect the fast presentation of themovie frames

Human visual system can not resolve individual pixels(spatial resolution is limited).

Spatial Resolution The capacity of the eye to see fine detail.

In practice, various ways are employed to measure andspecify visual spatial resolution, depending on the typeof acuity tasks used. Targetdetection: requires only the perception of the presence

or absence of an aspect of the stimuli, not the discrimination oftarget detail

Target recognition: are most commonly used in clinical visualacuity measurements, require the recognition or naming of atarget

Target localization: involves discriminating differences in thespatial position of segments of a test object


10/56

10

Spatial Resolution Determined by:

Density and type of photoreceptors in the retina

Limiting factors

Diffraction

Aberration

Refractive errors:

such as myopia (short-sightedness) and hyperopia (long-sightedness)

Size of the pupil

Illumination: background luminance

Time of exposure of the target

State of adaptation of the eye

Eye movement

Area of the retina stimulated

Foveation: Non-uniform Resolution Photoreceptors (cones and rods) are non-uniformly distributed in

the retina. Fovea : all cones (6 million) Periphery: mostly rods (125 million) interleaved

The density ofcone receptors determines the ability of our eyesto resolve what we see. Cones are more one-to-one to activate neurons

lower temporal resolution, but higher spatial resolution

Rods activate neurons in groups higher temporal resolution less spatial resolution

Foveation Resolution has the highest value at the point of the fovea (point of

gaze) and drops rapidly away from that point as a function of thedistance to the central point since there is the highest concentrationof cone photoreceptors at the point of gaze.

The region around the point of fixation (or foveation point) is projectedinto the fovea, sampled and perceived with the highest density.

The sampling density decrease dramatically with increasing distance tothe fovea.


11/56

11

Human Visual Foveation ModelFovea

The spatialresolution ofhuman vision iscut in half atabout 2.3degrees fromthe point of

fixation, fovea.

Human Visual Foveation ModelComputervision

Humanvision

Computervision

Humanvision


12/56

12

Temporal Resolution The eye constantly samples information ( i.e. images) projected onto

the retina in a periodic intermittent manner since there is a finiteamount of time required to collect and process information.

Information is then integrated so objects around us appear to bestable or move smoothly.

When intermittent stimuli are presented to the eye at a very low rate

they are perceived as separate

When the presentation rate is high, but lower than a curtain rate

they appear to stay on but with changes in intensity, producing thesensation called flicker.

Above a certain critical rate, the flicker stops. This point is called thecritical flicker frequency (CFF) that is influenced by a number offactors.

Temporal Resolution Critical flicker frequency (CFF)

Transition frequency point of anintermittent light source where theflickering light stops and appears as acontinuous light.

Fovea CFF is around 60 Hz; PeripheralCFF is around75 Hz;

Basis of film and TV Film: 60Hz Northern American TV: 75Hz

CFF =a logL +b, where a and b areconstants and L is the luminance offlickering stimulus in normal conditions.

Q: from a practical point of view, if acomputer monitor is flickering, what wecan do?

Increase refreshment rate Decrease the intensity.


13/56

13

Depth Perception Ten well-defined cues in depth

perception

Binocular cue is the most important

Each eye captures its own view

The two separate images are sent on tothe brain for processing.

When the two images arrivesimultaneously in the back of the brain,they are united into one picture.

The brain combines the two images bymatching up the similarities and addingin the small differences.

The small differences between the twoimages add up to a big difference in thefinal picture, to create a stereo picture

Depth Perception

Other cues: Relative size

Overlapping

Paralleled line convergence

Color contrast or difference

from the known contrast Relation of lights and

shadows

Texture gradients

Accommodation within theeye

Experience plays an importantrole

Most stereo-vision systems usehuman binocular cues to renderdepth information


14/56

14

Depth Perception

Limitation of Human In-depthPerception

x0: the arranged line by the subject is truly straightxx0: the points trace convex curves

x0

Distinct difference between the visual space that is perceived by thesubject and corresponding physical space of the real world underobservation: The sensed distance of distant objects is not proportional to its real

physical distance. The greater the distance, the more obvious this distortion becomes


15/56

15

Color Perception Three principal color receptors

(cones) (Three primaries) Any color to be matched by a

mixture of three colors: blue,green and red

Any color can be fully specified interms of their hue, lightness andsaturation.

Note: wavelength doesntnecessarily directly determines

color appearance? Can perceive as many as

30,000 different colors Does the so-called true-color

monitor make sense? 1million different colors

More colors, bigger data size

Other Remarks

Adult can see 3-6mm of movement per secondwhen an object is 1 meteraway

10300,000 different visual configurations might

conceivably be seen Retina is reflective

Eye blink does not affect perception

Attention and gaze direction are correlated

Lots of illusions to play with size and distance


16/56

16

Limitation of Human Visual

Perception

Limitation of Human VisualPerception


17/56

17


Perception



18/56

18


Perception



19/56

19


Perception



20/56

20


Perception



21/56

21

Relevance to INS Systems Input (Feedback)

Necessary Primary and fundamental

Issues Lack of depth information (how to incorporate depth information?)

Stereo vision by using specific interfaces based on human binocular cures Proper arrangement of the environment Can color cues be used for this purpose?

Costly: need a significant amount of network bandwidth possible solution: image foveation (few results)

Data rate? ( Is 100 frames/second good?)

Image size: does it make sense to have 24bits per pixel ?

Output (commanding) Eye movement ? (few results)

Advantage: fast (high-bandwidth), no extra payload Limitations: #of DOFs, range of movement, effect of flick, etc.

Traditional Computer Vision vs.Image Foveation

Traditional computer vision systems representimages on rectangular uniformly sampled lattices.

Equal spatial resolution

However, information in the image is not equally useful

Large size

Image foveation by mimicking human visual system

Foveated images:

Highest resolution at the point of gaze; spatial resolution dropswith the increase in the distance to the point of gaze.

Image size: can be significantly reduced

Less network bandwidth

Less time delay


22/56

22

Image Foveation in INS Systems

Any Discussion?


23/56

23

2. Auditory Channel

Is It Relevant?

Auditory stimulus does increase the realism (degree offidelity/telepresence) of a INS system in many instances: In the research conducted at NASA Ames Research Center, it

was found that pilots had difficulty knowing whether they hadpositively engaged a touch-screen virtual button without

auditory feedback. It was demonstrated by Massimino and Sheridan that auditory

cues could be used to substitute for force feedback in varioustelemanipulation tasks.

In an experiment at the J PL Advanced Teleoperation Laboratory,it was found that auditory feedback speeded the completion ofmanipulation tasks, given in addition to visual and hapticfeedback.

It was shown that, the addition of specialized sound significantlyincreased the reported sense of presence in a VE


24/56

24

The Ear

Human Auditory System

The human auditory sensing system hasthree basic functions:

to transmit sound through the outer, middle,

and inner ears, to transduce sound waves into neural energy

in the inner ear,

and to perform neural processing within andtransmit through the neural levels to theauditory cortex.


25/56

25

Human Auditory System The outer ear consists of the pinna and the ear canal.

When a sound reaches the outer ear: The pinna concentrates it, increases its amplitude, and reflects

the sound at the entrance of the ear canal.

This results in the intensity of the sound being changed by asmuch as 10 dB.

The effects caused by the pinna provide cues to the location of asound source and help to give the impression that a soundsource is external to the listener.

It should be noted: When auditory stimulus is transmitted via headphones, the

sound bypasses the pinna and arrives directly at the ear canal,so that most of the effect of the pinna disappears.

The difference in propagation delay to the two ears is also a veryimportant source of localization, especially at low frequencies.

Human Auditory System

After the sound passes through the ear canal, it reachesthe middle ear. to match the impedance of the air in the outer ear with that of the

fluid in the inner ear,

to preventing sound loss due to reflection,

and to allow sound to reach the inner ear with little attenuation.

Finally, sound vibrations are transmitted to the inner ear causing the membrane to vibrate.

The vibration lead to bending and activation of the hair-cellreceptors, which are connected to the fibers of the auditorynerve.

These fibers transfer these vibration messages to the centralnervous system, causing the sound waves to be perceived asaudible sound.


26/56

26

Audibility Audible frequency

Audible frequency range: 16Hz to 20,000Hz Most efficient: 1000Hz and 4000 Hz A drop in efficiency as the sound frequency becomes

higher or lower

Audible power level Threshold: 0db

Persons with very good hearing can detect Brownianmovement in a soundproofed chamber.

If the ear were more sensitive than this, random Brownianmovements would produce a constant sound and would tendto mask auditory stimulus.

Perceived loudness is approximately a log function ofactual loudness

Range ofAuditorySoundLevels


27/56

27

Physical-Space Description A single or pure tone sound:

Frequency

Magnitude

Phase

A sound consists of harmonics of lower frequency Harmonics are integral multiples of the fundamental, the lowest

frequency sinusoidal.

A sound can be described in terms of Frequency (spectrum components and predominant frequency)

Intensity

Direction (relative position to the listener)

Duration

Perception-Space Representation

Pitch (predominant frequency, not the highest frequency) Roughly corresponds to the frequency of the predominant

sinusoidal components A single-valued subjective summary of the sensed spectral

properties of the sound stimulus When the spectral property of a sound is made more diffuse over

a band of frequencies, it becomes more difficult for the listener todistinguish pitch.

White noise: NO PITCH AT ALL

Frequency just-noticeable difference (jnd)Dependent upon the loudness level and frequency of the sound

Smaller (higher perception precision) for low frequency tones Below 20db, the average human loses the ability to perceive change of

frequency Above 20db, 3/1000 of the tones frequency.


28/56

28

Perception-Space Representation Loudness (intensity)

Subjective judgment of the intensity of the observedsound stimulus

Audible loudness: 0db ~ 160db Discomfort: 120db+; Pain: 140db+

Loudness just-noticeable difference (jnd)

Dependent upon the sensation loudness level andfrequency

Below 20db, 2 to 6 db dependent upon frequency

Above 20db, to 1db is sufficient

At extremely high frequencies, jnd is large

Most sensitive (jnd smallest): 500Hz ~10K Hz

Perception-Space Representation

Duration For very short tones, the perceived intensity is

inversely proportional to duration

For very long tones, it is possible to have anauditory after-image

Physical tones less than 0.01s are insufficientto yield a pitch

A maximum loudness is reached at about 0.5seconds followed by a decrease in theintensity


29/56

29

Perception-Space Representation Spatial localization

Each ear has a non-uniform directional sensitivity

However, localization information is primarily a resultof the comparison of stimuli (intensity difference andtime interval) separately sensed by the ears.

Below 1K Hz, time difference is the predominant source ofdirection information

Above 1K Hz, loudness difference becomes significant forthe discrimination of direction

Least sensitive on median plane (due to symmetry)

Easiest to locate tones in 500 ~ 700Hz

Difficult to locate tones around 2K Hz

Speech

The most important class of auditory stimuli. The quality of a speech is a subjective description of the

particular waveform. Frequency range

100Hz to 10K Hz

The fundamental frequency of male voice is 125 Hz while thisfrequency for the female voice is 250 Hz.

Various sounds within speech differ in the way energy isdistributed. Vowels are predominantly sinusoidal with most of the energy

concentrated in the lower frequency portion of the spectrum.

Power typical conversational speech 10~15 microwatts


30/56

30

Relevance in INS Systems

Input

What environmental sounds are necessary or helpful

in a specific INS system or task ?

Sensory substitution

Artificial sound cues (loudness, pitch, spatial location andduration) are used to substitute for depth information

Question: what frequency band to be used if location cues areused for depth information? Which band should be avoided?

Which location should be avoided in placing the interface?

Output

Voice-based control

loudness, pitch, spatial location and duration ?


31/56


32/56

2

Terminology Haptic

originated from Greek haptesthai meaning to touch

relating to or based on the sense of touch

Haptic feedback (Haptic Interaction to be exact)

Refers to various forms of sensation evoked when theskin is subject to haptic stimuli (mechanical, thermal,etc).

It usually refers to force sensing, but more precisely torefer to the sense of force and differential forces (or,equivalently, displacements) on the skin in time and in

space.

Importance of Haptic Feedback

Haptic sensation is very important in our daily life. We manipulate an object based on the sensation of its shape, stiffness,

texture, and slippage. We feel the weight when lifting an object (otherwise, we may break our

arms) We rely on force information to maintain the position and orientation of

ourselves

Lack of haptic sensation in our daily life, May crush the food jar when we are attempting to screw the lid onto it. Lose the capability to balance our body and maintain our position and

orientation Even may not be able to hold a book

Lack of haptic sensation in INS systems The remote manipulator may crush the object that it is supposed to

grasp The surgeon may not be able to feel the presence of tumors in

underlying body tissue in minimally invasive surgery Even worse when the surgeon performs cutting tasks


33/56

3

Difference from other I/O Channels Closed-loop, bi-directional channel

Sensing Acting (motoring)

Relies on action/exploration to stimulate perception Usually active

Truly interactive Informatic, but more importantly energetic

Need direct contact Not well understood compared to visual and auditory

channels Haptic feedback (I would rather say haptic interaction)

can be categorized into: Tactile (cutaneous) Kinesthetic (force)

Haptic Exploration

Suppose the hand comes up to an object freelysuspended in space. The initial sense of contact is provided by the touch

receptors (nerve endings) in the skin, which providesinformation on the geometry, texture, slippage, etc. of the

object surface. This information is tactile1. When the hand applies more force, trying to hold this

object, kinesthetic force2 comes into play providingdetails about the position and motion of the hand andarm relative to the object. In the mean time, the forcefeedback now also gives a sense of total contact force,compliance (stiffness), and the weightof the object (if thehand is supporting the object in some way) . Thisperception is with the muscles and tendons beneath theskin.


34/56

4

Haptic Exploration In order for the hand to manipulate the object, say

move it horizontally, rotate it, or pinch it, the hapticsystem must issue stronger motor action3 that appliesforces on the object.

That response (feedback) will, in turn, guide furthermanipulation.

In summary, haptic interaction involves Tactile feedback Kinesthetic feedback

Motoring action Note that in general tactile and kinestheticsensations occur simultaneously.

Haptic Sensation Processing

Stimulus: Force Vibration Heat

Three types of receptors For mechanical action (force, vibration, slip) For temperature For pain

Information processing a response is triggered and an electrical discharge is generated

into the never fiber; Second-order neurons transmit the signal further up to the spine

and into the thalamus region of the brain; Here third-order neurons complete the path to the cortex where

the corresponding sensations of pressure, temperature, or painare registered.


35/56

5

Important Factors in Describing Haptic

Sensation

Adaptation Temporal variation in the responses of a receptor in response to

a constant stimulus

Sensitivity Threshold or JND

Temporal resolution The minimum time difference that can be detected by the

receptors

Spatial resolution The minimum spatial difference that can be detected by the

receptors

Saturation Maximum force exertion

1 Tactile Sensing

Important in object discrimination and manipulation In general, tactile sensations include:

Tactual Pressure Texture Softness

Wetness Friction-induced phenomena such as slip, adhesion etc. Local features of objects such as shape, edges, embossing and recessed

features Electrical conductivity. Vibrotactile sensations

Thermal Cold or warmth

Pain (both tactile and kinesthetic)


36/56

6

The Skin Very heavy and largest organ

roughly 2 m2; 5 kilograms

Two layers Epidermis: the outer protective layer of the skin, covering the

dermis. Dermis: the sensitive connective tissue layer of the skin

located below the epidermis, containing nerve endings,glands and blood vessels

Hairy skin vs. hairless skin Functions

Prevents body fluids from escaping Protects us Provides tactile information about stimuli

up to hundreds of receptors per cm2

the biophysical attributes of the skin vary tremendously with theparts of the body it covers

Tactile Sensation Hairless skin

Palm/fingertip

up to 135 receptors per square centimeter at the finger tip

The highest sensorial density of specialized receptors

Mapping the hand receptors to nearly a quarter of the total cortex

surface (of the brain). The sensorial mapping is dynamic (why?)

Five major types of receptors

Hairy skin

An additional type of receptors, i.e., the hair-root plexus that detectsmovement on the surface of the skin

Hairy regions are more sensitive

since the hairs act as levers, providing a considerable amplification of theapplied force.


37/56

7

Five Main Tactile Receptors Free receptor:

close to surface responds to distributed pain

Meissners Corpuscles surface curvature, local shape, slippage poor spatial resolution 43%

Pacinian Corpuscles vibration, slippage, acceleration 70-1000Hz response frequency range 13%

Merkels Disks

skin curvature, local shape, pressure 25%

Ruffini Endings skin stretch, local force 19%

Locations of Tactile Receptors


38/56

8

Sensorial Adaptation Temporal variation of responses of a receptor in

response to a constant stimulus.

Slowly adapting (SA) receptors:

The stimulus can be detected for a long time without muchdecay

Example

weight

Rapidly adapting (RA) receptors:

Stimulus becomes undetected in a very short time.

Example

wearing gloves or glasses.

Adaptation Rates of TactileReceptors

Merkel disks SA receptors, as they produce a long discharge rate in response to

force applied skin curvature, local shape, pressure

Ruffini corpuscles SA receptors as they produce a regular discharge rate for a steady load.

skin stretch, local force Meissner corpuscles

RA as they discharge mostly at the onset of the stimulus surface curvature, velocity, local shape, slip

Pacinian corpuscles RA type receptors as they discharge once for each stimuli application,

not sensitive to constant pressure. vibration, slip, acceleration


39/56

9

Sensitivity of Tactile Sensation Absolute threshold: the minimal touch force/weight felt by the hand

80mg on fingertips (on average)

150 mg on the palm (on average)

The threshold of the vibrotactile stimulus is 5~10times of the absolutethreshold

Pressure just-noticeable-difference (J ND): The value is a function of area over which the pressure is applied

When the contact area is increased, is it more or less sensitive?

(N/cm2)

Sensitivity: Frequency Dependant


40/56

10

Resolution of Tactile Receptors Temporal resolution

Varying for different

receptors

In general, increasesin tactile stimulusduration can lowerdetection thresholds.

Spatial resolution Human fingerpad

1.5mm

Summary of Tactile Sensing


41/56

11

Pain Sensing Excess mechanical, chemical, thermal or

electrical stimulus can excite the painsensation.

Negative adaptation

The subject perceive an increase in pain afteran injury.

Thermal Sensing

Touching or no touching Separate types of receptors. Cold sensitivity extends to greater depth than does warm in the skin. Some areas are sensitive only to cold

e.g. cornea.

Many points on the skin surface respond only to cold, or only towarmth, or even neither. E.g. on the forearm, cold spots average 13 to 14 per mm2, but warm

spots average 1 or 2 per mm2

Cold spots may be excited by a warm stimulus (over 45C) with theresulting sensation of paradoxical cold the hot stimulus actuallyfeels cold.

Cold receptors are most sensitive at 1C while warmth receptorshave a maximum sensitivity around 37 C.

There is a physiological zero temperature region around which notemperature is sensed.


42/56

12

Other Remarks on Tactile Sensing Partially understood

A great deal of remaining uncertainty concerning the exactfunction of the individual receptor cell types.

The encoding mechanism is very subtle.

Skin is not uniformly sensitive to excitation over itsentire region

The delay time of these receptors ranges from about50 to 500 msec.

The thresholds of different receptors overlap, and the

perceptual qualities of touch are determined by thecombined inputs from different types of receptors.

The operating frequency range: from at least 0.04 togreater than 1K Hz.

Other Remarks on Tactile Sensing

In general, the thresholds for tactile sensations arelowered with increases in duration and contact areas.

Skin surface temperature can also affect the sensitivityof tactile sensations.

In general, the sensation received from a particularstimulus is not the result of single excitation of areceptor, or even a single type of receptors.

No single receptor has a private line into the centralnervous system. The entire tactile sensing system is aninterlace of complex receptor networks.


43/56

13

2. Kinesthetic Sensing To recognize the object

Overall shape

Stiffness

Weight

Overall force

Awareness of human body parts relative to the

object Position and orientation

Movement (velocity and acceleration)

Force (resistance, weight, )

Kinesthetic Sensing Receptors

Locations Skin: skin stretch and cutaneous deformation J oints:

Endings in joint ligaments Endings in joint capsules Slowly adapting

Muscles Tendon organs: monitoring muscle tension Spindle organs: measuring stretch and rate of change

Together, these various receptors provide information about joint angles, muscle length and tension rates of change

None of the skin, joint, or muscle receptors provide awareness ofweight; instead, this sense arises mainly from signals derivedentirely within the central nervous system (negative adaptation)


44/56

14

Sensitivity and Resolution of

Kinesthetic Sensation The threshold and resolution depends on locations of the

joints No actual number is available.

Pressure J ND: roughly 0.06 - 0.09 N/cm2

The greater the contact area, the more sensitive the human armis to pressure

Position J ND (just-noticeable-difference) finger: 2.5 degrees wrist: 2.0 degrees elbow: 2.0 degrees shoulder: 0.8 degrees

Force J ND: 7%

Sensing bandwidth: up to 30 ~ 50 Hz

Summary of Haptic Feedback Haptic feedback is categorized into

Tactile (cutaneous) feedback Related to the skin Initial contact with the environment High bandwidth Tiny and distributed force

Sensing: shape, texture, slip, hardness, temperature, pain. Informatic

Kinesthetic (force) feedback Receptors placed deeper (muscle tendon, bones, and joints ). Stimulated by bodily movements Relatively low frequency It usually generates a single relatively big force Sensing: resistance, compliance, weight, position, orientation and

movement relative to the environment The only haptic sensing source during free motion Energetic


45/56

15

Motoring Aspects Maximum and sustained force exertion

Finger mechanical impedance

Force control bandwidth

Maximum and Sustained ForceExertion

Finger manipulation forces depend on The way objects are grasped (geometry) Individual gender Age

Motor skill and handicaps. Grasping geometry has been classified by

Power grasps: high stability and force, because the whole hand and palm

are used lack dexterity (fingers are locked on the grasped object)

Precision grasps: less force higher dexterity (only the fingertips are used)


46/56

16

Classification of Hand Grasps

Remarks on Hand Grasps

Tactile signals also play a significant role inalmost all manipulation tasks

May serve as preconditions for triggering some of themotor commands associated with these actions;

Directly reflect the accomplishments of manymanipulation actions

Provide information about an objects physicalproperties that are used in guiding the use ofmanipulation forces.

Together, tactile and kinesthetic sensations arefundamental to manipulation and locomotion.


47/56

17

Maximum Controllable ForceExertion for Different Grasps

Sustained Force and ForceResolution

Forces can be sustained comfortably for longdurations

Depend on the task and configuration of the humanhand.

Less than about 15% of the maximum exertableforce, i.e.

Index finger: safely exert about 7 N without encounteredfatigue and discomfort.

Middle finger: 6 N

Ring finger: 4.5 N

Force output resolution: 0.36 N


48/56

18

Finger Mechanical Impedance Describing the relationship between the applied force

and the motion in frequency domain

Playing a key role in the interface

Fidelity

Stability

Hogan at MIT found that the impedance of human handis indistinguishable from that of a passive system eventhough human hand is clearly an active system.

Fundamental assumption for the stability of a number of controlalgorithms

)(/)()( sVsFsZ

Asymmetry in Sensing andManipulation Bandwidth

Sensing bandwidth Maximum frequency with which tactile and force stimuli can be

sensed.

Manipulation bandwidth The speed (rapidity) with which humans can respond/act.

The sensing and manipulation loops are asymmetric Sensing band/bandwidth is much higher and larger than the

manipulation ones, i.e, we sense haptic stimuli much faster thanwe can respond to them.

The ability of the hand and fingers to exert forces: 5 ~10 Hz Force sensing : up to 30 Hz (or some people say 50 Hz) Tactile sensing: 0.4Hz up to 10KHz ( it is reported that it is up to

10K Hz for very fine feature recognition. In most literature, thisnumber is 1 K Hz)

Also, the finger is sensitive to up to 10,000 Hz vibrations whensensing textures, and is most sensitive at approximately 230 Hz.


49/56

19

Human finger sensing and manipulation bandwidth

Implications in INS Systems

Haptic feedback is critical for the success or efficiency ofmanipulation tasks

Haptic interface design Maximum and sustained force exertion Tactile and force kinesthetic sensitivity (absolute threshold, J ND) Temporal and spatial resolution

Sensing and manipulation bandwidth

System Design Input/output asymmetry Stability vs. Fidelity

For fidelity, high-frequency feedback is required For stability, high frequency should be filtered out

Data Transmission Transmission rate (depending on types of haptic signals) Data volume (depending on spatial resolution)


50/56

20

Final Remarks on I/O Channels Chemical channels: smell and taste

extremely complex and poorly understood

We also can sense: time (protensity)

Probability

Intensity

Purpose of perception

Not actual values But a mental image

Interactive effects of the sensory channels.

Sensory Substitution


51/56

21

Stimulus vs. Sensation Magnitudes A common experience for everyone

For a specific sensory modality, whenever the stimulusincreases, the intensity of the sensation grows accordingly.

A common basic principle governing the correspondencebetween stimulus magnitude and sensation magnitude. the sensation magnitude grows as a power function of the

stimulus magnitude

where:

k: constant depending on the unit of measurement

: differing from one sensory modality to another.

Perceptual magnitude can be scaled by quantity.

k

Cross-modality Mapping

Can a scaling relation be created between twodistinct sensory modalities?

The modality 1 stimulus is applied to the subjects.The subjects are asked what would be the sensation

magnitude in modality 2 by matching numbers andmaking direct comparison between the two different

sensory modalities.

Yes, there is a relation between any two distinctsensory modalities [Stevens 1975].

The cross-modality matching is common in nature.

[Stevens 1959 1966 1975; Stevens et al. 1963; Marks1986; and Hubbard 1993]


52/56

22

Sensory Substitution Sensory substitution is the provision to the

brain of information that is usually in onesensory domain (for example hapticinformation) by means of the stimuli,receptors, pathways and brain areas ofanother sensory system (for exampleauditory sensory system). [Bach et al. 1987].

The "sensory substitution" systems transform

stimuli characteristic of one sensory modality (forexample, haptic) into stimuli of another sensorymodality (for example, auditory).

Sensory Substitution

Originally for aid for handicapped persons

Vision through touch

Braille for the blind: information acquired visually

(reading) is, instead, acquired through thefingertips

Sound through vision

Sign language for deaf

In this course, we will see how andwhether it can be used in INS systems


53/56

23

Projects Flying airplanes as early as 1936 [deFlorez, 1936]

Apparently a pilot was dependent on external visual referencesto maintain flight under clear weather conditions.

In fog, even the most experienced pilot could not maintain aproper orientation without suitable instruments.

Research was conducted to establish aural reference axes thatcould be substituted for visual ones during instrument flyingconditions.

providing a turn indication consisting of an increase in soundintensity in one ear and a decrease in the other

having changes in the sounds pitch represent changes in airspeed.

It was demonstrated that blindfolded pilots could fly airplaneswhen two of their instrument indications were presented aurally.

Projects

Further research in this area was conducted to determine theaccuracy and speed of pilot response to a variety of auditory cues atHarvard University in the early to mid 1940s [Forbes 1946] . what types of auditory signals could be followed with greatest ease, with what accuracy such signals could be utilized, how many simultaneous auditory signals could be followed

successfully. It was found that if the signals were properly designed, up to four auditory indications

could be followed without hurting overall flying performance. three characteristics of a single signal could be combined to indicate

turn, bank, and airspeed. repetitive or sweeping type of motion of the signal from left to right to

indicate a change in directional heading a change in the pitch of the signal to represent a certain tilt or orientation of

the airplane, a putt sound that would change its rate of occurrence in association with

the sound of the airplane motor to indicate a change in airspeed.


54/56

24

Projects The tactile vision substitution system (TVSS) for the

blind [since 1969; Bach-y-Rita ] A head-mounted video camera captures image of the

environment. The image is then converted into a "tactile image". The tactile

image is produced by a matrix of 400 activators ( 20 rows and 20columns of solenoids of one millimeter diameter). The matrix isplaced either on the back, or on the chest.

Equipped with the TVSS, blind (or blindfolded) subjects arealmost immediately able to detect simple targets and to orientthemselves. They are also rapidly able to discriminate vertical

and horizontal lines, and to indicate the direction of movement ofmobile targets. The recognition of simple geometric shapes requires some

learning (around 50 trials to achieve 100% correct recognition). More extensive learning is required in order to identify ordinary

objects in different orientations. The latter task requires 10 hoursof learning in order to achieve recognition within 5 seconds.

Projects Auditory and tactile senses were studied to substitute

kinesthetic feedback in time-delayed teleoperation[Massimino, MIT 1993, 1995] The force feedback is substituted by the auditory sensations

A task of inserting a rectangular peg into a rectangular hole To indicate force from contact at the left or right side of the hole, a

medium pitch (1000Hz) tone sounded in the left or right ear (thesubject wore earphones).

To indicate contact at the top or bottom, the tone was at high(3500Hz) or low (350Hz) pitch in the both ears, which made thetone appear to the subjects as from the middle of the head.

The loudness of the tone was to indicate the magnitude of the force.


55/56

25

Projects M. Kitagawa, D. Dokko, A. M. Okamura, and D. D. Yuh,

"Effect of Sensory Substitution on Suture ManipulationForces for Robotic Surgical Systems," J ournal of

Thoracic and Cardiovascular Surgery, Vol. 129, No. 1,pp. 151-158, 2005. (The J ohns Hopkins University )

Work was done to substitute direct haptic feedback withvisual and auditory cues in robotic surgical systems.

Substitution for Force Feedback inINS Systems

Sound cues: Pitch, loudness, direction and duration

Visual cues An active role in our daily life. For examples:

Traffic signals, which use colorful lights, warn people forattention to create correct reactions to different trafficconditions.

Animals have severe reactions to some aggressive colors,such as red and yellow.

We can estimate the rough temperature of flame byobserving its color.

What visual cues we can use Color, brightness ..


56/56

Pros and Cons of Substitution for

Kinesthetic Feedback Advantages:

Avoiding instability due to transmission time delay Bidirectional =>unidirectional

Energy-exchange =>information exchange

No need for expensive and bulky haptic devices

Disadvantage: Accuracy

Opposite to the human-centric principle Not intuitive.

May not work when the user is tired or in fatigue.

Mismatching is possible For example, if used for tactile feedback: dynamic bandwidth of up

to 1 kHz is required for realistic force reflection, yet visual cues runvery much slower.

One Last Question?

Are blind persons using tactile-visionsensory substitution actually seeing?

Depends

However, if blind subjects were given similar information to

that which causes the sighted to see, and

if the blind subjects were capable of giving similarresponses;

we can say blind subjects do use the information inthe same way that sighted people do.

Date post:	04-Apr-2018
Category:	Documents
Upload:	mjdcan563730775
View:	219 times
Download:	0 times

Lec2 Human Capabilities

Documents