Date post: | 13-Aug-2015 |
Category: |
Technology |
Upload: | philip-desenne |
View: | 72 times |
Download: | 1 times |
Decoding Digital Audio Visualizing and Annotating Linear Time-Based Media
Phil Desenne Center for Hellenic Studies,
Harvard University
May 8, 2015
To decode or interpret audio is to explain the meaning
or understanding
of something about it
Relevant for Research, Teaching and Learning across all disciplines
Decode –> Interpret
Curiosity - Discovery - Interpretation - Research
Amateurs - Learners - Educators - Scholars
Each one decodes audio in their own realm
the process transcends realms and roles
}}
Songs
Music
Voice recordings
Field recordings
Lyrics / Transcription / Translation
Notes / Musicology / Ethnomusicology
Oral History / Languages / Speech Therapy
Bioacoustics Research / Anthropology
Audio Visualize and Annotate
{}Listen Decode
In our day-to-day we are constantly, filtering, decoding
and attaching meaning in our brain to daily sound
bites that hit our ears
!
digital audio opens a broader spectrum of
decoding possibilities for
research, teaching and learning
Visualization and Annotation of Audio
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam in augue sodales nisl pharetra efficitur in in felis. Ut malesuada justo nec libero finibus placerat. Donec vitae enim risus. Nunc eget purus eget nunc bibendum tempus hendrerit vel eros. Praesent mollis diam augue, vel convallis quam interdum eu.
Transcript
we now have digital tools that facilitate and enhance the process of decoding, attaching
meaning and understanding
Audio Annotation & Visualizations in
Research
Acoustics / Bioacoustics / Hydroacoustics
Anthropology / Musicology / Ethnomusicology
Speech Therapy
Education / Teaching
Importance of Visualizing Sound:
Human ear does not hear or discern all sounds
www.cochlea.org739 × 252Search by image Frequency hearing range in man and some common animal
very high-frequency
sounds
very low-frequency
sounds
}visualization –> accessibility
Recording Natural Sounds www.leaps.ms900 × 396Search by image
You can also use Raven Lite to slow down natural recordings so that the full complexity of a song may be heard. Listen to the complexity in the ending trill ...
Bioacoustics: detection and interpretation of sounds in animals
Annotating the visual wave form of audio: Amplitude and Frequency
BIRD SONGS AND CALLS WITH SPECTROGRAMS ( SONOGRAMS ) OF SOUTHERN ... www.birdsongs.it489 × 396Search by image
Fig . 2 shows a 20 second fragment over 1-minute song sequence of a Cirl Bunting (Emberiza cirlus). Fig. 3 blows up one phrase of the same song and shows ...
Sonograms and Spectrograms of Bird Songs
The Mind's Machine - Chapter 15 A Step Further www.mindsmachine.com800 × 619Search by image
(a) These sonograms show the typical adult song patterns of two sparrow species. The songs illustrated in part (b) were produced by males reared ...
Species identification and animal behavior
WarblerWatch: Warbler Guy, where do I learn about "reading ... warblerwatch.blogspot.com650 × 578Search by image
Ergo, you'll quickly have no problems identifying a song sparrow classic song via its sonogram in comparison to a common yellowthroat's, and
so on.
Prof. Tom Kelly, First Nights course, Harvard College
Music Courses: Annotated Interactive Play-throughs
Learners explore music
Foreign Culture Courses: Listening Guides
Prof. Richard Wolfe, Foreign Cultures Course, Harvard College
Foreign Culture Courses: Listening Guides
Prof. Richard Wolfe, Foreign Cultures Course, Harvard College
Research Teachingdrives
Prof. Richard Wolfe
Ethnomusicology research in South, Central and West Asia
Courses in Ethnomusicology at Harvard College
Audio annotations
richardkwolf.com www.music.fas.harvard.edu/faculty/rwolf.html
Annotation Meta layer
Visualization layer
Audio layer (URI)
Annotation DB
Media Repositories
ideally under same repository entities
Media Search
Open Annotation Data Model
Client tool !
Persistent Annotation Meta-layer Open API Access
Stable Digital Repositories URNs resolving to URLs
Ephemeral Tools / Content / Learning Management Systems
Open Annotation Model
Public Archives
Other Institutions Archives
Open Source
Museums
HUL DRS
Research Database
Catalyst, LibraryLab
HILT ...
Course & Student Content
Peer Researchers
Personal Research & Archives
Subject Experts
Incubator Projects
Persistent Annotation Repositories
Individual Repositories
External Repositories
Internal RepositoriesAnn
Ann
Open Annotation Federated Systems
across all media
Future of audio annotation
• Searching: faceted, specific range target searches
• Semantic tagging: machine learning
• Automatic Annotations:
• transcription / translation • acoustic detection, • individual voice recognition • bioacoustics species id • AI -> Pairing crowdsourced data and automatic
annotation using semantic annotated data (OA)
• High definition audio and detailed audio analysis
• Collaboration and crowdsourcing tools
• Cross-referencing media annotations