On the parameterization of clapping
Herwin van WelbergenZsófia Ruttkay
Human Media Interaction, University of Twente
Content
Context and goals Related work Experiment setup Results Conclusion Questions
Context: Reactive Virtual Trainer An ECA acting out exercises a user is
supposed to do Perceives the movement of the user Reactive
Gives feedback Using speech, gestures and motion
(Re)schedules and adapts exercises Tempo changes
Subtle timing and lifelikeness of motion is important
Goal The generation of believable,
adaptable exercise motion in real time How can we parameterize motion?
What parameters? Tempo, amplitude (for accentuation?), …
How do the parameters relate? How do they affect movement?
How is speech synchronized with exercise motion?
Related work: biomechanics Typical biomechanical research setup
Very obtrusive Measuring one movement characteristic Gaining ‘deep’ knowledge
Our setup Unobtrusive Measuring a wider range of characteristics Less depth, measure on an abstraction level
that gains us parameters for movement generation
Related work: finding animation parameters Statistical methods (Egges et al), machine
learning (Brand et al) Finds independed parameters Not intuitive Highly depended on analyzed data set
Laban Movement Analysis Effort can automatically be found from
movement data (Zhao et al) Shape? Our parameters (tempo, amplitude) can be
mapped to LMA parameters
Related work: parameterized animation Rule based (EMOTE, Neff et al, Hartmann et
al, ..) Uses movement models Typically does not deal with dependence
between parameters Lack of detail
Example based (Wiley et al, Kovar et al) By blending examples Nr of examples needed grows exponentially
with nr of parameters
Related work: model based gesture synthesis Kopp et al: Uses biomechanical rules of
thumb to generate movement Real time Domain: speech accompanying gestures
We plan to extend on this work Use in rhythmic movement domain Providing parameterization Providing whole body movement Introducing movement variability
Focus Analysis of a clapping exercise Analyzed aspects:
Synchronization of speech and motion How does a change of tempo affect
movement? Time distribution Movement path Amplitude Left-right hand symmetry Whole body involvement
Clapping experiment: setup
Mocap analysis of two subjects
Instructions: ‘Free clap’:
Clap and count from 21 to 31
‘Metronome driven clap’:
Clap and count to the metronome
Synchronization of clap and speech
Phases from movement in gestures The phonological synchrony rule
holds for clapping
Time distribution in phases Free clap was executed consistently
at ≈ 60 bpm One subject made use of a pre-
stroke hold at 30 bpm The relative duration of the phases
does not change with tempo The standard deviation of the
relative duration decreased with one subject
Movement path of the hands
Amplitude: how to measure
Maximum distance between hands Path is curved
Max distance between hands alone does not display the amount of motion
Distance along path
Amplitude: observations Path distance and max. hand distance
decrease with tempo Average speed is constant at different
tempos There is a linear relation between period
and path distance Pathdistance = a + b ▪ period
Amplitude of free clap is higher Average speed of free clap is higher
Amplitude
Period vs path length
Left-right hand symmetry:How to measure Model: self oscillating
systems Closed orbit between position
(x) and speed (v) x is the normalized angle x^ θ is the phase angle Relative phase angle:
Φ = θleft-θright
Negative Φ means right hand leads
Left-right hand symmetry at 90 bpm
Left-right hand symmetry:Theory
Right handed subjects lead a rhythmic task with their right hand (Treffner et al) But such asymmetry can disappear when the task
is metronome driven Stability of Φ depends on the tempo and
mass imbalance (Treffner et al, Fitzpatrick et al) Higher tempo => higher |Φ| Higher tempo => higher variability in Φ
Left-right hand symmetry:Findings Mean Φ is consistently negative for our
right-handed subjects No difference between metronome
driven and free clap in mean Φ The standard deviation of Φ
increases with tempo No significant relation between mean Φ
and tempo was found
Whole body involvement
By annotating if markers move in the same tempo as the hands
Movement was found on the head and torso for all tempos
For low tempos movement was even observed up to the thighs and knees
Conclusions The phonological synchrony rule was validated
for clapping Clapping can be sped up by making the path
distance smaller A pre-stroke hold can be used to slow down Clapping is clearly a whole body motion At a faster tempo, fewer body parts are
perceivably involved Left-right hand movement variability increases
with tempo For both right-handed subjects, the right hand
was leading The metronome did not diminish this lead
Further work Ultimately: generate clapping motion given
tempo + personal characteristics More recordings
Free clapping without counting Tempo transitions How do personal characteristics affect movement?
Deeper analysis How does variability affect the movement path?
Generation Can movement on the rest of the body be generated
given movement on the arms (as in Egges, Pullen)? Blending clap animation at different tempos to gain
animation at a new tempo (as in Kovar)?
Questions
Easter eggs
Why use gesture-like phases for clapping? The stroke of a speech accompanying gestures (SAG) is
at an energy peak in the movement and expresses meaning (McNeill)
Claps have such a clear peak But this peak does not express meaning
Why compare SAG and clapping? The form of clap movement and SAG is similar
Excursions: start in rest, end in rest Peak structure Well bounded But not symmetric
May find information on the nature of the phonological synchrony rule
Does it depend on form or meaning?
Precision
Precision No significant correlation between
metronome period and avg clap ‘error’ or variability of clap tempo was found
Measured both absolute and relative to the metronome period
Left-right hand position at 90 bpm
3D hand & elbow positions
3D hand positions at different tempos
Free clap amplitude vs metronome drive clap amplitude