International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
Music Mind Machine in Sheffield Research Centre
Department of Music, The University of Sheffield
International Conference on the Multimodal Experience of Music
PROGRAM
ICMEM Organisers:
Renee Timmers, Nikki Dibben. Zohar Eitan, Roni Granot, Victoria Willamson. & Andrea Schiavio
Conference Assistants:
Stephanie Bramley (Editor of Abstract Booklet)
Julian Cespédes-Guevara (Editor of Program)
Tim Metcalfe (Editor of Proceedings)
ICMEM was realised with financial support of the British Academy, International Partnership and
Mobility Scheme PM120092 to Renee Timmers and Zohar Eitan.
Student stipends were made available by SEMPRE and ESCOM.
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
2
Welcome
On behalf of the entire Organising Committee, we are truly delighted to welcome you to
Sheffield for the first International Conference on the Multimodal Experience of Music
(ICMEM). After having hosted a broad range of events, activities, summer schools and
conferences in the last few years, we are once again excited by the possibility to provide the
growing community of scholars and researchers in music psychology and cognition with a
unique occasion to share knowledge and disseminate their own research findings.
The ICMEM 2015 program features keynotes, symposia, papers, posters, concerts and
demonstrations, which will engender an exploration of different aspects of human musical
experience, examined though the lenses of multimodality and interactivity. In addition to the
plenary sessions by key figures in the field – Prof. Amir Amedi and Prof. Nicolas Cook - the
schedule contains 13 paper sessions, 2 poster sessions, a symposium on music and language
and a symposium on cross-modality and emotion, a concert with talks, and a final roundtable
presented by two eminent scholars in the field: Prof. Eric Clarke and Prof. Peter Walker.
We would like to thank our speakers, and also you for coming to Sheffield and contributing to
what we hope will be a very successful event that will provide great scientific impact. A great
thanks to the conference assistants Stephanie Bramley, Julian Cespédes-Guevara, and Tim
Metcalfe who edited the booklets and proceedings of the conference, and to Isha Nandra for
organisations at the conference venue. Special thanks as well to our psychology of music
student volunteers helping out during the conference and before: Yuko Arthurs, William
Chew, Caroline Curwen, Ioanna Filippidi, Alexander Keegan, Shen Li, Henrique Meissner,
Martha Nye, Landon Peck, Rosemary Pollock, David Quiroga Martinez, Dimitri Trouka,
Harriet Wells. Finally, thanks to all for bringing your expertise into this gathering and
welcome to ICMEM 2015!
ICMEM Committee:
Renee Timmers, Nicola Dibben, Zohar Eitan, Roni Granot, Victoria Williamson & Andrea
Schiavio
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
3
CONTENTS Papers Day 1 4 Papers Day 2 5 Papers Day 3 7 Posters Day 1 9 Posters and Demonstrations Day 2 10 Practical information and maps 11 Monday 23 March 2015 10.00 Welcome & Plenary 1 11.00 Break 11.30 Paper sessions 1 & 2 13.00 Lunch 14.00 Poster session 1 15.00 Break 15.30 Paper sessions 3 & 4 18.30 Conference Dinner (Halifax Hotel) Tuesday 24 March 2015 9.00 Paper sessions 5 & 6 10.30 Break 11.00 Symposium, Paper session 7 13.00 Lunch 14.00 Poster session 2 & demonstrations 15.00 Break 15.30 Paper sessions 8 & 9 19.30 Concert with 2 talks (Firth Hall)
Wednesday 25 March 2015 9.00 Symposium, Paper session 10 11.00 Break 11.30 Paper sessions 11 & 12 13.00 Lunch with piano performance 14.00 Plenary 2 14.50 Break 15.20 Round Table & Closure
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
4
Day 1: Monday 23 March 2015 10.00 High
Tor 2 Welcome: Renee Timmers
Plenary 1: Musical representation of depth, 2D visual images and full 3D: technologies, psychophysics and neuronal correlates1
Amir Amedi Chair: Renee Timmers
11.30
High Tor 2
Paper session 1: Music & Motor Responses Chair: Justin London
High Tor 5
Paper session 2: Acousmatic music and new technology Chair: Karen Burland
Eaves, Burridge, Griffiths, McBain, & Butcher Action observation modulates both the desire to move and the perception of ‘groove’ while listening to percussive music2
Moore Cross-modality in acousmatic music: the physical and virtual in music where there’s ‘nothing to see’5
12.00 Küssner & Caramiaux Motor invariants in gestural responses to music3
Dack Multimodal references in translating Schaeffer’s ‘Traité’6
12.30 Su How an observed human movement shapes auditory rhythm perception4
Penalba, Valles, Partesotti, Costañón & Angeles Sevillano Types of interaction in the use of MotionComposer, a device that turns movement into sound7
13.00 The Edge Lunch 14.00 High Tor 2 & Meeting Room Poster Session 1 15.00 The Edge Break 15.30
High Tor 2
Paper session 3: Performance (1) Chair: Jane Ginsborg
High Tor 5
Paper session 4: Melody and Naarrative Chair: Renee Timmers
Moelants, Van Dyck, VanSteenkiste & Leman Visual and Auditory Cues in Music Performance28
Costa-Giomi The effects of mode of presentation on infants’ recognition of melodies and voices26
16.00 Rozé, Aramaki, Bourdin, Chadefaux, Dufrenne, Kronland-Martinet, Voinier & Ystad Exploring the influence of cellists’ postural movements on musical expressivity29
Granot Musical improvisations on cross-modal prompts27
16.30 Bamford & Davidson Moving in Character: a Pilot Study into Music, Movement, and Empathy30
Eyes ‘Song for Aberfan’ – Investigating the interplay of musical performance, film and narrative8
17.00 Fulford, Ginsborg, Hopkins, Seiffert & Mate-Cid Multimodal sensory interaction effects resulting from D/deafness and artificial attenuation of auditory stimuli in music performance31
Rudenko Imagery in Piano Pedagogy: Visualisation of musical language in children’s cycle. Musical Toys, Sofia Gubaidulina9
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
5
Day 2: Tuesday 24 March 2015 9.00 High
Tor 2 Paper session 5: Creativity; Tactile experience of music Chair: Andrea Schiavio
High Tor 5
Paper session 6: Music Analysis Chair: Nicola Dibben
Nagy The Multimodality of Creative Experience: Towards the Embodiment of Musical Creativity32
Atkinson Without Narrative: Meaning in Einstein on the Beach from Sonic and Visual Domains35
9.30 Merchel & Altinsoy Designing Auditory-Tactile Perception of Music33
Piilonen Messiaen, Composer of 'Awe': The Cross-Modal Pursuit of Divine Fear in Apparition de l'Église éternelle36
10.00 McKenzie, Lennox & Wiggins Inside-Outside: 3-D music through Tissue conduction34
Kozak Listeners’ Bodies in Music Analysis37
10.30 The Edge Break
11.00
High Tor 2
Symposium on Music & Language: Multimodal roots of communication in infancy38i
Chair: Fabia Franco
High Tor 5
Paper session 7: Performance (2) Chair: Mine Doğantan-Dack
François, Teixido, Agut, Bosch & Rodriguez-Fornells Songs facilitate speech segmentation in 2 days-old neonates38 Van Puyvelde & Franco The interaction of music and language in the ontogenesis of human communication: A multimodal parent-infant co-regulation system39 Costa-Giomi The role of words and the human voice in infants’ attraction to song40 Franco, Sobolewska, Stewart & Waller Infant-directed song and speech: Attention deployment to sound and articulatory gesture in infancy41
Meals, Morrison & Confredo Effects of Temporal Congruence on Evaluations of Conductor Efficacy42
11.30 Fabian Analysing classical music performance: how limiting are audio-only approaches? 43
12.00 Vuoskoski, Thompson, Spence & Clarke A novel method to investigate the interaction of sight and sound in the perception of musical performance44
12.30 McKeigue A Compositional Perspective on the Exploitation of the Visual in Performance in "Letting the Tail Wag the Dog" for string octet and “When I am at my best" for solo Bb clarinet45
13.00 The Edge Lunch
14.00 High Tor 2, 3 & 4, Meeting Room Poster Session 2 and demonstrations 15.00 The
Edge Break
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
6
Day 2: Tuesday 24 March 2015 (continued…)
15.30
High Tor 2
Paper session 8: Multimedia Chair: Jonna Vuoskoski
High Tor 5
Paper session 9: Imagery and shape Chair: Victoria Williamson
Herbert Designing Experience: Everyday Multimodal Listening, Sound Art and Multisensory Experiential Marketing62
Bretherton, Ng, Wilson & Watt Music, Doodles and the Autonomic Nervous System65
16.00 Hammerschmidt & Wöllner The influence of image compression rate on perceived audio quality in music video-clips63
Prior Multi-modal understandings of musical shape66
16.30 Ireland & Windsor Measuring and modelling perceived emotion and audiovisual incongruence between film and music64
Filippidi Musical Imagery as the Unconditioned Response of Music in Everyday Life67
19.30 Firth Hall
Concert with talks: Tactile experience in performance Chair: Zohar Eitan Davidovic Reading the Style: the use of auditory and visual texts in historically-informed performance practice68 Doğantan-Dack Hearing through touch: correspondences between tactile and aural sensations in artistic pianism69
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
7
Day 3: Wednesday 25 March 2015
9.00 High Tor 2
Symposium on Cross-modality and Musical Emotions70i Chair: Julian Céspedes-Guevara
High Tor 5
Paper session 10: Musical Learning Chair: Victoria Williamson
Isbilen & Krumhansl The Color of Music: Synaesthesia or emotion-mediated cross-modal associations?70 Eitan & Timmers Cross-modality and Emotion in Schubert’s Die Stadt71
Bravo Human emotion processing investigated through musical information within controlled audiovisual paradigms72 Bhattacharya Crossmodal transfer of musical emotion: An implicit process? 73
Hawes & Gee Silent score reading with and without audio: an eye-tracking and self-report study74
9.30 Glasser The impact of idiopathic synaesthesia on musical abilities75
10.00 Schiavio & Cummins An inter(en)active approach to musical agency and learning76
10.30 Wöllner & Halpern Auditory, visual, and verbal working memory in different musical professions77
11.00 The Edge Break 11.30
High Tor 2
Paper session 11: Body knowledge and Indian music Chair: Zohar Eitan
High Tor 5
Paper session 12: Tempo and Movement Chair: Clemens Wöllner
Pearson Hand gestures in South Indian vocal lessons: The exploitation of cross-modality as a pedagogic tool78
Jakubowski, Farrugia & Stewart Probing imagined tempo for music: Effects of motor engagement and musical background81
12.30 Paschalidou & Clayton Towards a sound-gesture analysis in Hindustani dhrupad vocal music: effort and rag space79
London, Burger, Thompson & Toiviainen Speed on the Dance Floor: Interactions Between Visual and Auditory Cues for Tempo82
12.30 Bremmer What the body knows about teaching music. Pedagogical content knowledge viewed form an embodied cognition perspective80
Metcalfe Assessing the contribution of different musical variables to the effect of background music on motor behaviour83
13.00 The Edge Lunch
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
8
Day 3: Wednesday 25 March 2015 (continued…)
14.00 High Tor 2
Plenary 2: Seeing Sound 84 Nicholas Cook Chair: Nicola Dibben
14.50 The Edge Break
15.20 High Tor 2
Round Table: Perspectives on cross-modality Chair: Roni Granot
Walker Cross-sensory correspondences and simple sounds 85 Clarke Cross-modality or plenisentience? Ecological encounters of a realist kind86 Timmers Cross-modal perception of music 87
17.00 Closure: Renee Timmers
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
9
Poster session 1 and Performance: Emotion, Neuroscience, Performance, New Technology, Social Cognition (Monday 23 March, 14.00-15.00)
Location: Meeting Room
Emotion Christensen Comparing the emotional responses of multimodal haptic audio sources with similar unimodal audio sources10 Ancelle, Tietze & Hunter Differences in psychophisiological responses to self-selected music11 Kwok Emotion perception of multi-modal interaction between music and image12 Weisgerber, Gilson & Vermeulen The influence of music on the processing of emotional faces13 Performance Ricketson Listen with your eyes19 Waddell & Williamon The effects of visual performance cues on continuous ratings of musical performance quality20 Wilson, Bertolin & Stafford Musicians make more music mistakes, non-musicians make more motor mistakes21 Zhou Hearing and Visual Complementation: Discussion on Three Findings of Accent22
Location: High Tor 2
Social Cognition Loaiza Music as a social other? The experience of music via an Enactive theory of social cognition24 Neuroscience Ata, Kutluk, Calli, Gonul, Oguz & Aksoy The Musician's Brain: Increased Grey Matter Volume and White Matter Integrity. A Voxel Based Morphometry and Diffusion Tensor Imaging Study14 Pérez-Acosta & Ramos-Amézquita A study on the effect of musical imagery on spontaneous otoacoustic emissions in musicians and non-musicians15 New Technology Mycroft, Reiss & Stockman The effect of differing user interface presentation styles on audio mixing16 Brophy, Neff, Fitzpatrick, Pitt & Noonan Evaluating the cueing efficiency of sound icon designs in a multimodal system during acoustically chaotic events17 Stirnat Perceived Spaciousness in Music Presented through Headphones, Loudspeakers and Wave Field Synthesis18
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
10
Poster session 2 and Demonstrations: Analysis and composition, Creativity, Education, Evolution, Movement, Perception and Cognition, Special Population (Tuesday 24 March, 14.00-15.00)
Location: Meeting Room
Perception & Cognition Antović, Stamenković, & Figar Is distinction between “inherent” and “ad-hoc” musical meanings psychologically real? An experimental study55 Gomez, Rodriguez-Cuadrado, Puigcerver, & Navarra Spatial recording of acoustic loudness56 Menouti, Akiva-Kabiri, Banissy, & Stewart Characterizing timbre-colour synaesthesia57 Li, & Timmers Spatial representations of pitch: a comparison between pianists and flautists58 Schiavio & Timmers Cross-modal facilitation in learning atonal melodies59 Evolution Miani Where are the icons, indexes, and symbols in musical meaning? An evolutionary account53 Movement London, Burger, Thompson & Toiviainen Changing speeds: Dancing faster changes the character of one's movement54
Location: High Tor 2
Education Godinho Miming to recorded music: multimodality and education49 Larrouy-Maestri Evaluation of pitch accuracy in ecological contexts: What about non-musical variables? 50 Nijs, & Leman The Music Paint Machine: A multimodal approach to instrumental music learning and teaching51 Olivera Testing the multitasking ability of musicians and conference interpreters52 Special Population Fernandez, Caprile, Gómez-Berrocal, López-Sala, Póo-Argüelles, Pons, & Navarra Visuospatial and auditory deficits in preadolescents and adolescents with nonverbal learning disorder60 Jack, McPherson & Stockman Hearing Impairment and the Vibrotactile Experience of music61 Analysis and Composition Deng, Wang & Kang Soundscape Composition as a New Music Genre48
Location: High Tor 3 and 4
Demonstrations Brereton & Kearney Multi-modal experience of a virtual concert hall: A tool for exploring the influence of the visual modality in interactive real-time room acoustic simulations46 McKenzie, Lennox & Wiggins Inside-Outside: 3-D music through Tissue conduction47
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
11
How to get to the conference venue The Edge, The Endcliffe Village Endcliffe Crescent, Sheffield, S10 3ED Tel: +44 (0)114 222 8800 From Sheffield Train Station and the City Centre, go to the Sheffield bus interchange, which is a short walk away from the train station. Take bus First 120 in the direction of Fulwood. Exit at Endcliffe Crescent (this takes about 15 minutes). Walk downhill following the signs to Endcliffe Village, and The Edge.
How to get to the conference dinner at Halifax Hotel (Monday) – available only if pre-booked Endcliffe Vale Road, Sheffield, S10 3ER Tel: +44 (0) 0114 222 8810 From the Edge, head south on Endcliffe Crescent toward Endcliffe Avenue, turn right on Red Lane. The Hotel will be on the left.
How to get to the concert venue: Firth Hall (Tuesday) Firth Court Western Bank Sheffield, S10 2TN Walking: From The Edge (Conference venue) walk uphill Encliffe Crescent and turn right on
Fulwood Road. Walk downhill for about 15 minutes, (on the way Fulwood road becomes Whitham Road and then Western Bank). Firth Court will be on the left.
By Bus: From The Edge (Conference venue) walk uphill Encliffe Crescent and on the bus stop take the bus 120 towards Waterthorpe. Alight at the Clarkson Street bus stop. Walk downhill for about 5 minutes to Firth Court.
Firth Hall is located in Firth Court which is an old red brick building on the northern side of Western Bank. To get to Firth Hall, enter the main entrance which faces Western Bank, walk up the main staircase and when you get to the first floor you will be facing Firth Hall.
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
12
Conference venue, Halifax Hotel and Firth Hall:
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
13
If you need assistance during the conference, please contact any of these members from the organizing committee: Renee Timmers, Nicola Dibben, Victoria Williamson, Andrea Schiavio, Julian Cespedes-Guevara, Ioanna Filippidi, Henrique Meissner, Tim Metcalfe. Emergency number For immediate emergency assistance call: 999 (Police, Medical Emergencies and Fire Department) WIFI at The Edge Wifi is available for conference visitors. Information is available onsite. Restaurants near Endcliffe Village All of these restaurants serve vegetarian dishes. Please keep in mind that in Broomhill and Hunter’s Bar you can also find low-priced cafes and “take-aways”. In the Broomhill area (about 12 minutes away, walking)
Name, type Address Average price for main meal
The York (Pub) 243-247 Fulwood Road (S10 3BA) £10
Lokanta (Turkish) 478-480 Glossop Road (S10 2QA) £14
El Toro (Spanish) 129 Newbould Lane (S10 2PL) £10
Thai Thai Ka (Thai) 198 Whitham Road (S10 2SS) £8
Zanzibar (African) 257 Fullwood Road (S10 3BD) £8.40
In the Hunter’s Bar area (about 10 minutes away, walking)
Name, type Address Average price for main meal
All Siam Thai (Thai) 639 Ecclesall road (S11 8PT) £9
La Terrazza (Italian) 352 Sharrow Vale Road (S11 8ZP) £9
Otto’s Restaurant (Moroccan) 344 Sharrow Vale Road (S11 8ZP) £14
In the Ranmoor area (about 18 minutes away, walking)
Name, type Address Average price for main meal
Canton Orchard (Chinese) 377 Fulwood Road (S10 3GA) £10
Ranmoor Tandoori (Indian) 384-386 Fulwood Road (S10 3GD) £8
West 10 (British) 376 Fulwood Road (S10 3GD) £14.50
International Conference on the Multimodal Experience of Music, 2015 MMM in Sheffield
14
Areas with restaurants near Endcliffe Village:
