Exercise: Spectrogram Cross-Correlation

Summary: This exercise demonstrates the process of calculating the similarity between sounds using a process called spectrogram cross-correlation. We will walk through the process using a folder of sounds produced by a Red-eyed Vireo. Learning objectives:

• Master the mechanics of applying spectrogram cross-correlation • Be able to predict situations where spectrogram cross-correlation will be informative

and when it is poorly suited and unlikely to perform well Prerequisite knowledge:

• Basic familiarity with spectrogram parameters and terminology Instructions:

1. Open Raven.

2. Under the menu options, find Tools -> Batch Correlator. Note: There is also a Tools -> Correlator option, but this can only compare two individual sound files.

3. You will now see an options window that looks like the image below.

4. To choose the first folder of clips to correlate, click the “Browse” button and navigate to the folder of Red-eyed Vireo selections. These selections were created using the Band Limited Energy Detector in a previous exercise. Here, we will start with this folder of 91 selections, each containing a Red-eyed Vireo vocalization.

5. Next, use the browse button next to Folder 2 to select the second set of clips to correlate. In this case, we will use exactly the same set of red-eyed vireo recordings. It is common to correlate all clips against all other clips from the same recording session in order to find clips that are similar to each other. Alternatively, you may have a set of pre-defined call types that you wish to correlate with new recordings. In this case, the two folders would be different, with one folder including reference clips and one folder containing segments from new recordings.

6. By default, the output will be written to a file called “BatchCorrOutput.txt.” If you wish, you can provide a more informative title for your output matrix (e.g. Red-eyed vireo cross-correlation).

7. Under the correlate options, select Spectrogram. For spectrogram parameters, be aware that the options that you choose will affect the correlation values. Generally, the spectrogram parameters that you use to view the sounds are also appropriate for correlating the sounds. Note: If you wanted to correlate the time signature and ignore frequency differences, you can choose the Waveform option.

8. Under the remaining options, use the default settings. Most of these options are meant for relatively specialized applications and are rarely used.

9. After correlations have been calculated, click ‘close’ to complete the dialog window.

10. You will now see a window that provides correlation values among detections. Click the

“Colors” radio button box in the center of the top panel to color cells by the correlation value.

11. Note that there is a diagonal line of cells with a correlation value of 1. This reflects the perfect correlation when a clip is correlated with itself. Note the presence of other high correlation values that reflects similarity between different clips.

12. Double click on a cell in order to see the two clips that were correlated and the alignment between them. The top panel shows the correlation that is achieved at different time offsets between the clip. If there is one good alignment, there is typically one strong peak, but particularly for calls with repeated pulses, there may be multiple alignments that would produce relatively high correlations between the two sounds.

13. These correlation matrices can now be used with hierarchical cluster analysis or other analytical approaches to reveal groups of similar sounds. These approaches can be applied through a variety of software such as R, Matlab, and Python. By correlating a library of labeled sounds against a folder of unlabeled sounds, it is also possible to assign sounds to existing categories using an objective criterion.

Synthesis 1. What are some of the factors that may cause spectrogram cross-correlation to be

artificially high? 2. What are some factors that may lead to low correlation values, even when sounds

appear similar? Answers

1. If two sounds have similar continuous background noise, spectrogram cross-correlation may be high simply because the backgrounds match and produce high correlation values.

2. Because spectrogram cross-correlation relies on pixel-by-pixel comparison, if sounds are shifted slightly in frequency or stretched/compressed slightly in time, it can lead to low cross-correlation values beween seemingly similar sounds.