FUZZ-IEEE 2009, Korea, August 20-24,2009

Potential Applications of Fuzzy Logic in Music

A, Egemen Yilmaz, Member, IEEE and Ziya Telatar, Member, IEEE

Abstract-Even though the application spectrum of the fuzzy

logic is quite wide, fuzzy based implementations in music are

rarely encountered. In this study, we try to give the definitions

of the problems in music theory; and we try to adapt fuzzy

based reasoning particularly for the counterpoint problem.

Despite the fact that this study is currently limited to note­

against-note two-voice counterpoint technique, the approach

can extended to more complicated problems of harmonization

(either vocal or orchestral), improvization, and even

composition.

I. INTRODUCTION

FUZZy logic has proved to be successful not only in

engineering applications but also in diverse areas such as

finance, sociology, medicine, defense and military systems,

etc. On the other hand, the number of research studies about

the application of fuzzy logic to artistic development

activities is not high.

So far, there have been a limited number of attempts and

discussions about the application of fuzzy logic in music. In an unpublished manuscript prepared as a lecture note [I],

Tsang identified how fuzzy logic can be used for jazz

improvisation. In this work, Tsang used the mathematical

infrastructure defined by Elsea [2], whose mathematical

representation seems to be feasible due to its power in

modeling various concepts in music (such as a scale or a

chord in addition to a stand-alone note) simultaneously.

Independent from Tsang and Elsea, in [3] Landy mentioned

the potential use of fuzzy logic in composition, even though

he did not put any solid mathematical proposal to support his

argument. With a different aspect in [4] and [5], Cadiz

discussed the applicability of fuzzy logic in audiovisual

mapping and sound synthesis. In these works, Cadiz did not

get into any topic related to composition or harmonization

theory; instead he tried to construct synchronous and

correlated aural and visual structures (i.e. similar to those of

provided by Microsoft Media Player© or Winamp©) by

means of the fuzzy logic. Among all these works regarding

the application of fuzzy logic to music, the approach of

Tokumaru et at. [6] is the one which resembles ours at most.

In this work, the authors applied fuzzy logic coupled with

genetic algorithm in order to determine the accompaniment

chord progression for a given melody.

A. E. Yilmaz is with the Electronics Engineering Department, Ankara University, 06100, Tandogan, Ankara/Turkey (phone: +90 312 203 35 00; fax: +90 312 212 54 80; e-mail: aeyilmaz@eng.ankara.edu.tr).

Z. Telatar is with the Electronics Engineering Department, Ankara University, 06100, Tandogan, Ankara/Turkey (e-mail: ztelatar@eng.ankara.edu.tr).

978-1-4244-3597-5/09/$25.00 ©2009 IEEE 670

In this study, we try to identify the potential areas in music

where the fuzzy logic seems to be applicable. For this

purpose, in Section II we start with identifying the main

research areas in computer aided music. In Section III, we

present brief mathematical background of fuzzy theory.

Section IV follows with a more solid discussion and a

possible application of the fuzzy logic in High Renaissance

and Baroque style two-voice counterpoint technique. Finally,

in Section V we put more ideas about probable near and

mid-future innovation that can be performed in the scope of

this study.

II. PROBLEM STATEMENT

In our point of view, the main research areas in computer

aided music can be classified into three main categories : (i)

harmonization and orchestration; (ii) improvization; and (iii)

composition. For the solution of all problems in these lanes,

the harmony theory (in fact multiple theories due to its

evolution throughout the centuries) provide a complete and a

consistent set of tools to the researchers. Once the problem

in hand is clearly identified with the inputs, the constraints,

and the expected outputs, the solution can be achieved by the

application of the rules imposed by these theories. Moreover,

some variety and flavor can also be added by means of

defining and applying some stylistic rules and techniques.

Regarding the categories listed above, harmonization (or

orchestration) is nothing but the determination of the

accompanying synchronous set of voicing for a given

melody. Certainly, this operation should be performed

subject to constraints, where these constraints are imposed

by the selected theory and practical situations. As a matter of

fact, this problem is sometimes referred to as 'harmonization

with constraints'. Since the pioneering works of Ebcioglu in

1980s [7]-[ 1 0], several researchers have been putting

considerable effort for the solution of this problem by means

of various techniques. A 2001 dated survey regarding the

publications in this topic is given by Pachet and Roy [11].

Throughout the development of the music theory [12],

rules of harmony have evolved; and especially at the

beginning of the 20th century, it went to extremes

particularly by the so-called 12-tone music and jazz. While

these rules used to be very strict during the 17th century,

later more flexibility has been added by the innovative music

theorists. In addition to the rules imposed by the theory,

other constraints arise due to the fact that the final melody

shall be appropriate for live performances. For example a

vocal partiture beyond the normal human vocal capability

would be meaningless, even if it satisfies all harmonic and

melodic rules imposed by the theory. Similar constraints are

also valid for the other categories (i.e. improvization and

composition). Table I and Table 2 describe the problem

statement together with the inputs and constraints for the

harmonization (and orchestration) and improvization

respectively. Due to space considerations, the problem

statement for composition (which is much more complicated)

is not given for the moment. The content of these tables are

self-descriptive, so that we find it redundant and unnecessary

to discuss them once more. A couple of points to be

mentioned here are as follows: The terminology of

Renaissance & Baroque era is slightly different than the

modem music terminology (e.g. mode instead of scale,

Cantus Firmus instead of the main melody, etc.). As

identified in Table I, the chord progression might not be

available all the time. Determining the chord progression for

a given melody is a preliminary step for harmonization. In

[6], Tokumaru et al. 's proposal was nothing but the solution

to this particular problem. On the other hand, in [1], Tsang's

discussions and ideas were about the solution of the

improvization problem seen in Table 2, particularly for jazz.

III. Fuzzy LOGIC

A. Theory

Generally in engineering and scientific world, classical

definition of an object behavior, either logically existing or

not, might be belonging to a certain valued set. In most

cases, an operator might not tell linguistically what kind of

action he takes in a particular situation. In this respect, it is

quite useful to model his actions using different numerical

data under uncertainty. In such a case, the concept of a fuzzy

set [ 13], which is a simple extension of the notion of a

classical set, shall be preferred. For a fuzzy set of objects, on

the other hand, every object has a grade of membership in

the set. This grade arbitrarily runs from zero to one, and is

represented by a membership function m(x), where x is the

domain, or 'universe of the discourse' of the set of all

objects. Every object x has its own membership function

value m(x).

Fuzzy sets in the same domain can be formed using logical

connectives such as 'and' and 'or'. The mathematical

relations for forming a set C from the sets A and Bare:

C = A and B: mJx ) = min(mA (x ), mB (x )) (I)

C = A or B: mc (x ) = max(mA (x ), mB (x )) (2)

Further, 'if.. . then . . . ' rules can be evaluated to give an

output fuzzy set in one domain from an input fuzzy set in

another domain. The rules are also known as fuzzy relations.

Mathematically, the process for the rule is given by :

If A is B:

B = Ao R: mR (Y) = max(min(mAu), mR (u, y ))) (3) 11

671

FUZZ-IEEE 2009, Korea, August 20-24,2009

where B is the fuzzy output set in y; A is the fuzzy input set

in u; and R is the fuzzy relation in the (u, y) domain.

B. Application in Music

According to the fuzzy theory given in the previous

subsection, a fuzzy feedback decision system might be

implemented in order to perform the accompanying tone

generation dynamically. In such a case, the first step here

would be to produce a relational fuzzy model of the general

harmony rules by separating them into proper sets. Here,

inputs can be classified into fuzzy sets (fuzzification

process), and output produces a result according to the input

by making de-classification from the sets to scalar sets

(defuzzification process). Then, the fuzzy system estimates a

function between defined sets without a mathematical model

of how the outputs depend on the inputs. For this purpose, a

number of fuzzy reference sets can be used to fit each of the

parameter values. The model can be trained by the rules

introduced, and produce a proper accompaniment voicing for

the current value of inputs.

In the following section, a more solid example will be

discussed in order to visualize the application of fuzzy logic

m music, particularly the constrained harmonization

problem.

IV. A PARTICULAR EXAMPLE

A. Music Theoretical Background

The current western music theory depends on the 12 tones

C, C# (or Db), D, D# (or Eb), E, F, F# (or Gb), G, G# (or Ab),

A, A# (or Bb), and B; where there is a semi-tone difference

between two succeeding tones. The modes/scales, which

determine the overall atmosphere and the tonic mood of

musical pieces, are nothing but 7-element subsets of this 12-

element set constructed by means of certain rules. Melody of

a musical piece includes the elements of this subset wholly

(or dominantly in some cases). Similarly, accompaniment to

the main melody would include the elements of the same

subset. These are imposed by the rules of music theory,

which primarily promotes the simultaneous usage of

consonant tones, and demotes the dissonant ones.

For computer aided music applications, generally the

Musical Instrument Digital Interface (MIDI) protocol [14] is

used. The protocol not only supplies the representation of all

probable musical events appearing in a musical piece, but

also provides a 16-track environment for simulation of

orchestral performances. Among numerous attributes defined

in the MIDI standard, for the time being the most important

one for us is the 'pitch'. As seen in Table 3, the standard

represents notes of almost 10.5 octaves (more than the range

of a piano), where these values are appropriate for usage in

arithmetic operations.

FUZZ-IEEE 2009, Korea, August 20-24,2009

TABLE I PROBLEM DEFINITION FOR VOCAL HARMONIZATION AND ORCHESTRATION

Vocal Harmonization Orchestration

Renaissance & Baroque Era Classical & Post-Classical Era

Given • Mode • Scale • Scale • Cantus Firmus • Single Partiture Melody • Single Partiture Melody

• Chords (*) • Chords (*) Constraints • Vocal Pitch Ranges and Capabilities • Vocal Pitch Ranges and Capabilities • Instrumental Pitch Ranges and

• Counterpoint Rules • Harmony Rules Capabilities (Timbre, Agility, etc.) • Stylistic Issues • Stylistic Issues • Harmony Rules

0 Melodic 0 Melodic • Stylistic Issues 0 Contrapuntal 0 Harmonic 0 Melodic

0 Harmonic 0 Rhythmic

Task Construction of Other Partiture Construction of Other Partitures Construction of Other Partitures (�): May not be aVailable all the time.

TABLE II PROBLEM DEFINITION FOR IMPROVIZA TlON

Western Music (Classical & Contemporary)

Given • Mode/Scale • Main Theme

0 Chord Progression 0 Rhythm

Constraints • Instrumental Pitch Ranges and Capabilities (Timbre, Agility, etc.)

• Harmony Rules • Stylistic Issues

0 Melodic 0 Harmonic 0 Rhythmic

Task Improvized Melodic Progression (e.g. Instrumental Solo)

Another important concept Tn musIc theory IS the

interval. As stated by the jazz theorist Levine in [15], "as

the atoms are building blocks of matter, intervals are the

building blocks of melody and harmony". Given two

notes, interval is mainly the difference of their positions

on the stave. As seen in Fig. 1, the interval between two

notes at the same position is not 0 but 1 by definition.

TABLE III MIDI PITCH VALUES

Octave

C C# D D# E F F# G G# A A# B

0 0 2 3 4 5 6 7 8 9 10 11

1 12 13 14 15 16 17 18 19 20 21 22 23

2 24 25 26 27 28 29 30 31 32 33 34 35

3 36 37 38 39 40 41 42 43 44 45 46 47

4 48 49 50 51 52 53 54 55 56 57 58 59

5 60 61 62 63 64 65 66 67 68 69 70 71

6 72 73 74 75 76 77 78 79 80 81 82 83

7 84 85 86 87 88 89 90 91 92 93 94 95

8 96 97 98 99 100 101 102 103 104 105 106 107

9 108 109 110 111 112 113 114 115 116 117 118 119

10 120 121 122 123 124 125 126 127

672

Classical Turkish Music

• Maqam • Main Theme

0 Oem 0 Rhythm (Usfil)

• Instrumental Pitch Ranges and Capabilities (Timbre, Agility, etc.)

• Rules • Stylistic Issues

0 Melodic 0 Rhythmic (Usfil)

Improvized Melodic Progression (e.g. Instrumental Taksim)

J J J J J J r 13

Fig. 1. The definition of intervals. It should be noted that the interval between two notes at the same position is not 0 but I .

It should b e noted that the interval between two notes is

neither the difference of the MIDI pitch values of them,

nor the 1/2 of it. The relation is somewhat complicated as

seen in Table 3. For example, at a fixed octave, the

interval and the pitch difference between C and E are 2

and 4, respectively. On the other hand, the pitch difference

between E and G is 3, even though the interval between

these is 2. Thus, the interval-pitch relationship shall be

handled carefully during the implementation of any music

software.

B. Note-against-Note Two-Voice Counterpoint

In music theory, counterpoint is the relationship of two

or more voices which are harmonically dependent.

Derived from the Latin term punctus contra punctum

(meaning 'point against point') and being developed in the

High Renaissance, it dominated the common practices of

Baroque period. It is principally based on construction of

an accompanying melody with respect to a main melody

called Cantus Firmus. The most elementary technique of

counterpoint is the note-against-note, where each note in

the accompaniment is added against one particular note of

Cantus Firmus.

Based on the modes depending on the church music, the

counterpoint rules generally strictly dictate what not to do,

and provide some possibilities about the things that can be

done. Due to such behavior, we fmd fuzzy logic

appropriate for applying to the technique. Previously,

Adiloglu and Alpaslan [16] tried to solve the same

problem with an artificial intelligence approach. Even

though the problem handled was not exactly the same, in

[17], Geis and Middendorf tried to solve the Baroque style

harmonization by means of the ant colony optimization;

and in [18], Mcintyre handled the problem by means of

the genetic algorithm.

The counterpoint rules very basically define the

consonant and dissonant intervals. Harmonically, the

consonant and dissonant intervals are as seen in Fig. 2.

For example, the note C is in perfect consonance with G,

where as it is dissonant to D or B. This means that given a

note C in the Cantus Firmus, adding the notes D and B to

the accompaniment is not appropriate. These sorts of rules

can be grouped as the Harmony Rules.

141; ; J J J J J J 8 r ! 1 I 5 1 Pe�ectConsonances

31 6 Impe�ect Consonances

2 4 1 7 1 Dissonances

Fig. 2. The consonant and dissonant harmonic intervals. Even though traditional harmony rules prohibits dissonance. contemporary music theory (e.g. 12-tone music of Arnold Schonberg, or jazz) promotes intentional usage of dissonances.

Another aspect of the counterpoint is the melody

construction. While adding a new note to the

accompaniment, its relation to the preceding note should

also be considered. This relation is again identified by the

interval between the notes. The definition of consonance

and dissonance is similar to the case of Harmony Rules,

but slightly different. For example, an interval of 4 is

harmonically dissonant; whereas it is melodically

considered as consonant. We group such rules as the

Melody Rules. During the construction of the

accompanying melody, small steps shall be preferred as

much as possible. We call this rule as the Step-Size Rule.

Other sets of rules define the behavioral relationships of

different partitures. The general tendency is to create

counter movements between the partitures (i.e. if the

Cantus Firm us ascends, the accompaniment should

descend). Moreover, the partitures shall not cross. Such

rules can be named as the Counter Movement Rule.

With all these constraints, the problem of adding a note

with the 'note-against-note two-voice counterpoint

673

FUZZ-IEEE 2009, Korea, August 20-24,2009

technique' can be pictorially defined as in Fig. 3. For the

evaluation of probable P4'S, the operations illustrated in

Fig. 4 shall be performed.

The next step is to construct the actual membership

functions. The membership functions for the harmony

rules can be defined by means of the consonance and

dissonance relationships given in Fig. 2. By considering

the differences between the melodic and harmonic

consonance definitions, the membership functions for the

melody rules can also be created. Similarly, the counter

movement rules can be transformed into membership

functions. All membership functions constructed can be

seen in Fig. 5.

The selection of the note to be assigned can be posed as

a testing in which the null hypothesis, Ho, corresponds to

the case of no proper match in the set against the

alternative Hj:

Ho : lrl l <Tl or lr2 1 < T2 HI : lrl l ::::Tl and Ir2 1 ::::T2

(4)

(5)

where Yl and Y2 are input values; whereas Tl and T2 are

experimentally determined thresholds.

Cantus Finnus

Q 1'2 !2

P2 Accompaniment

I' 2 Q I Fig. 3. The problem statement of the 'note-against-note two-voice counterpoint technique'. The task is to find all possible P4'S for PI, P2, and P3 (subject to the constraints of counterpoint rules).

An example, which is generated by means of the

proposed approach, is given in Fig. 6. Here, a soprano

partiture is taken as an input (Cantus Firmus), and the

corresponding alto accompaniment is constructed.

V. CONCLUSION

In this study, which can be considered as a conference

provocative, we tried to identify the potential usages of

fuzzy logic in music. Due to the nature of the harmony

rules (full restrictions and multiple allowances) in music,

fuzzy logic seems to be a promising tool for the

assignment of new notes to the new partitures. The

selection of the proper threshold values mentioned in (4) and (5) is a subject of investigation in order to obtain best

results.

For simplicity and due to space considerations, we have

focused on note-against-note two voice counterpoint

technique.

FUZZ-IEEE 2009, Korea, August 20-24,2009

� ,+. __________ --L-___ --"-____ -"-_----'-__ +. interval .... \ mmelopit: (i nt erval)

" ," \ ..... , .. fir> /

'\ I \ \ ,' , A

" " \ , \ \ , , "\ , ' " \ ' \ , , " \

, \ , .... ... \ , \ , " .... ... \ \ +-L-----�------�--L-�------�----�L------+interval

/' .-.-'

/ /

/' /

_.-

• interval mrounter_movement (interval)

+-------------------L--n-----------------------+ interval

frounteryoint (pitdl)

.Pz

Fig. 4. Pictorial description of the fuzzificationldefuzzification process for the solution of the 'note-on-note two-voice counterpoint technique'. Once the values of membership functions (w.r.t. intervals) for each rule are identified, the solution can be performed via combination and rescaling (w.r.t. pitches) of them appropriately. Here, the membership functions are notional in order to illustrate the methodology.

! mharmonic(interval) 1.0

1 f, Itep size (interval)

-=F==r==CP==r='��----'-------II-t-· -1-L-+-+-------'-----'-���r==r===f===i,_. interval -10 -9 .g -7 -6 -5 -4

(-0) -3 -2 -I 0 6 7 9 10

(0) ,,[ mm�'''_""wm''"lintervan

.... -+1 -+I-If--+I -------t-I -+I--'---+I---jl---------+ I -+I-f--+I _. interval -10 -9 .g -7 -6 -5 -4 -3 -2 -I 2 3 5 6 7 9 10

Fig. 5. Exact membership functions for the set of counterpoint rules. It should be noticed that the functions regarding harmony and melody rules are slightly different.

674

I' ia��us rr�us �so:.raT)

9

i Accompaniment (Alto) I· 2 Q I 9 I 9 I 9

Q 9

E2 9

t:i! 9 t:i!

9 ...,. ...,.

FUZZ-IEEE 2009, Korea, August 20-24,2009

9 E::i! II t:i! Q tI 9

II Q tI Q ...,. ...,. ...,.

Fig. 6. An example generated by means of the proposed approach. The Cantus Firmus is for soprano, and the accompaniment is for alto.

But it is clear that our approach can be extended to

other species of counterpoint (i.e. not necessarily note­

against note). Moreover, the approach can be extended for

a typical 4-voice modern choral harmonization.

Not limited to Baroque counterpoint rules, fuzzy logic

can also be applied to harmonization with respect to other

artistic movements (e.g. Classical, Romantic, 12-tone,

Gospel, Barbershop, Lady's Barbershop, etc.).

As stated before, the infrastructure provided by fuzzy

logic makes the potential applications open ended.

Orchestration related issues, improvization, even

composition can be handled by means of this approach.

Another interesting issue to be mentioned here is

nothing but the applicability of the technique to Classical

Turkish music, in which an octave is divided into 53 equal

intervals instead of the 12 as in the western music system.

REFERENCES

[I] M. Tsang, "Picking up lines in trendy bars: Musical trend representation", The University of British Columbia, Vancouver BC, 2000, Online: http://courses.ece.ubc.ca/518/previous/hit2000/pdf/l l_tsang.pdf

[2] P. Elsea, "Fuzzy Logic and Musical Decisions", University of California, Santa Cruz, 1995, Online: http://arts.ucsc.edu/EMS/Music/researchiFuzzyLogicTutorlFuzzyT ut.html.

[3] L. Landy, "From algorithmic jukeboxes to zero-time synthesis: a potential a-z of music in tomorrow's world (a conference provocation)", Organised Sound, vol. 6, no. 2, pp. 91-96, 2001.

[4] R. F. Cadiz, "Fuzzy logic in the arts: applications in audiovisual composition and sound synthesis", in Froc. Annual Meeting of the

North American Fuzzy Information Processing Society (NAFIPS

2005),2005, pp. 551-556 . [5] R. F. Cadiz, "A Fuzzy-Logic Mapper for Audiovisual Media,"

Computer Music Journal, vol. 30, no. I, pp. 67-82, Spring 2006. [6] M. Tokumaru, K. Yamashita, N. Muranaka, and S. Imanishi,

"Membership Functions in Automatic Harmonization System", in Proc. IEEE International Symposium on 28th Multiple-Valued Logic, 1998, pp. 350-355.

[7] K. Ebcioglu, "Computer Counterpoint", in Proc. international

Computer Music Conference, 1981, pp. 534-543. [8] K. Ebcioglu, "Report on the CHORAL project: An Expert System

for Harmonizing Four-part Chorales", Research report RC 12628, IBM Thomas 1. Watson Research Center, March 1987.

[9] K. Ebcioglu, "An Expert System for Harmonizing Four-Part Chorales", Computer Music Journal, vol. 12, no. 3, pp. 43-51, Fall 1988.

[10] K. Ebcioglu, "An Expert System for Harmonizing Chorales in the Style of J.S. Bach ", Journal of Logic Programming, vol 8, pp. 145-185,1990.

[ I I ] F. Pachet, and P. Roy, "Musical harmonization with constraints: A survey", Constraints Journal, vol. 6, no. I, pp. 7-19,2001.

[12] G. Bumcke, Harmonielehre I-II, Saturn-Verlag, Berlin, 1921. [13] L. A. Zadeh, "Fuzzy sets", information and Control, vol. 8, no. 3,

pp. 338-353, 1965 .

675

[14] MIDI Protocol, Online: http://www.midi.org. [15] M. Levine, The Jazz Theory Book, Sher Music Co., 1995. [16] K. Adiloglu, and F. N. Alpaslan, "A machine learning approach to

two-voice counterpoint composition", Knowledge-Based Systems,

vol. 20, pp. 300-309, 2007. [17] M. Geis, and M. Middendorf, "An ant colony optimizer for melody

creation with baroque harmony", in Froc. iEEE Congress on

Evolutionary Computation (CEC 2007),2007, pp. 461-468. [18] R. A. McIntyre, "Bach in a Box: The Evolution of Four Part

Baroque Harmony Using the Genetic Algorithm", in Proc. IEEE

World Congress on Computational intelligence & First iEEE

Conference on Evolutionary Computation, 1994, vol. 2, pp. 852-857.