A Time Based Approach to Musical Pattern Discovery

in Polyphonic Music

Tamar Berman

Graduate School of Library and Information ScienceUniversity of Illinois at Urbana-Champaign

ICMPC 9, Bologna 2006

Musical Pattern Retrieval

• Method and system for musical pattern discovery and retrieval

• Designed as a tool for music researchers, scholars and students. Not designed as a model of human perception

• Yet, analysis of the system’s outputs through evaluation by humans yields interesting data for music theory/perception/cognition research

Musical Pattern Retrieval

• Question: Can we create a search engine that receives a sung or played melody as input, and searches for matches in a music database?

• Answer: Yes– String matching: McNab et al. (1996)– N-grams: Downie and Nelson (2000)– Markov models: Birmingham et al. (2001)

Musical Pattern Retrieval

• Question: Can we create a search engine that receives a description of a musical structure as input, and searches for matches in a music database?

Musical Schemas / Style Structures

• Leonard Meyer describes archetypical patterns and traditional schemata that are the “classes” in terms of which particular musical events are perceived and comprehended.

• “No melody, however original and inventive, is an exception to this principle” (Meyer 1973)

Musical Schemas / Style Structures

• Eugene Narmour (1977) discusses style forms and style structures, upon which a “stylistic language” is constructed.

• Style forms are “parametric entities” which achieve enough closure so we can understand their functional coherence without reference to the specific contexts from which they come.

• Style structures can be created from style forms by arranging them in various contexts “according to their statistically most common occurrences”

Example: The 1-7…4-3 schema

• Prevalent in 18th century music• First noted by Meyer (1973) and studied further

by Gjerdingen (1988)• Consists of two event pairs (*):

– 1-7: The melody descends from the 1st degree to the 7th. The harmony shifts from I to V

– 4-3: The melody descends from the 4th degree to the 3rd. The harmony shifts from V7 to I

• Examples:– KV543.sib– KV200.sib (*) Simplified definition

System for Musical Pattern Retrieval

• Distinguishing features:– Support for the description and retrieval of

complex, polyphonic patterns– Noise resilience: instances will be retrieved

even if embedded within other patterns or interspersed with other events

– User-friendly interface for pattern specification– Retrieved instances can be ranked according

to their likelihood of fit to the desired pattern

Retrieving the 1-7…4-3

Retrieving the 1-7…4-3

Sequence Retrieval Example

kv268-1.sib

Mozart, Violin Concerto No. 6 in Eb K268, Allegro moderato, measures 24-29 (51.5’)

Test Data

• 505 Midi files of music by W.A. Mozart, taken from http://www.classicalarchives.com

• Includes symphonies, piano sonatas, piano concertos, other concertos and piano trios

• Truncated to first 50 measures

• Normalized

• Converted into note objects and then into time series

Time Series Representation

• A time series is a set of observations on the value of one or more variables, taken at successive points in time

• In the musical time series:– Variables: 12 pitch classes– Values: role played by pitch class at the time

of observation (top/bass/middle/absent)

• Result: a series of “harmonic windows” representing each musical piece

Musical Time Series Parameters

• Window length: size (in seconds) of the time interval described by each harmonic window

• Onset interval: time (in seconds) between window onsets (“sampling rate”)

Use of Absolute Time Units

• Motivation:– Readily and reliably available in midi data– Potential application to audio data

• Justification:– For events that are close to each other in time,

seconds – rather than beats – are likely more relevant

– For fast music, schema events could be further apart (in beats/measures) than for slow music

System Evaluation

• A selection of 115 retrieved candidate instances were evaluated by 3 human judges and by 12 queries

• The queries differed from each other in parameters such as window length, onset interval and role specifications for pitch classes within each event

• Instances that were rated as correct by a majority of the human judges were rated as correct by a majority of the queries

=> 100% precision is attainable!

System PerformanceWindow Length Onset Interval Query Type Precision

1.000 0.500 TV 0.632

1.000 0.500 CB 0.282

0.500 0.500 TV 0.875

0.500 0.500 CB 0.500

0.500 0.250 TV 0.733

0.500 0.250 CB 0.317

0.250 0.250 TV 0.778

0.250 0.250 CB 0.538

0.250 0.125 TV 0.538

0.250 0.125 CB 0.333

0.125 0.125 TV 0.857

0.125 0.125 CB 0.600

N/A N/A  Majority vote 1.000

Optimal at 0.5 second windows - Observed by Wundt (1874)

Question

• Do these excerpts sound similar?

– Mozart Clarinet Concerto in A, K622, beginning of Allegro

– Mozart Piano Concerto No. 6 in Bb, K238, beginning of Rondo

Similarity

They both contain sequences which satisfy the following constrains:

1. The first event includes pitches C, E, G with G on top 2. The second event includes pitches C, E with E on top3. The third event includes pitches F, A, C4. The fourth event includes pitches C, E 5. The fifth event includes pitches D, F, A 6. The sixth event includes pitches D, F, A with F on top7. The seventh event includes pitches C, G8. The eighth event includes pitches G, B, D, F9. The maximum duration of the sequence is 15 seconds

Conclusions

• Applying simple pitch constraints at multiple time resolutions yields successful retrieval

• The “top voice” requirement for melody is effective– Observed by Meek and Birmingham (2001)

• Creating a search tool for musical structures is feasible!

• The technology could be used for similarity retrieval or theme variations retrieval

Future Work

• Support for constraints on rhythm, contour and metric placement

• Enabling multiple roles per pitch class

• Describing distance in beats and measures

• Integration with alternative representations

• Application to audio data

Thank You!