in C. Woods, G.Luck, R. Brochard, F. Seddon, & J. A. Sloboda (Eds.)  Proceedings of the Sixth International Conference on Music Perception and Cognition. (2000)  Keele, UK: Keele University Department of Psychology.

Discontinuity  Amplification in melodic perception©

Fabio Cifariello Ciardi
Conservatorio di Musica di Perugia, Econa
f.cifariellociardi@edisonstudio.it

1)	Introduction
Our ability to parse the acoustic array into its many meaningful sound objects rely on our capacity to process sensorial information.Variables that influence perception of sonic entities in every day life can be broadly organized into three sets. The former refers to subject-dependent factors as attention, memory, knowledge as well as gestures and postures. The second group is environment-dependent factors as masking, interference that can arise from interactions among different sonic events. The latter includes the cognitive correlates of the physical parameters that define each sonic event.

The possibility to interact with these variables facilitates our ability to answer to specific tasks in cognitive processing.  We can modify our attention or posture, and we can move our body with the purpose of optimizing hearing conditions. In the same way, we can change cognitive strategies in order to fulfill different needs and improve our adaptability to different situations. Under certain circumstances we can improve perception by partially modifying the acoustic environment around the sonic event inorder to attenuate the interference from other events. Still, we can directly alter the sonic event itself by changing some of its parameters.
By means of these actions we can affect the segmentation of the sonic surface into cognitive units useful for further cognitive processing regardless any cultural difference (Dowilng, Jay & Harwood, 1986).  Such units interact in terms of structuring forces in the processes underlying the definition of cognitive correlates of a sonic event.  
A typical examples of sound parameters partial modification can be found in musical practice when musicians gradually modify the result of their sight-reading by adding nuances and micro-variations to the their performance. The role of such variations has been studied in several computer models of musical performance (Frieberg, A. & Sundberg J., 1995).  Generally, these models allow more convincing results for tonal than for non-tonal music. Weaker results for several compositions of twenty century may be linked with the lack of coherence between the structuring rules defined by the composer and strategies the listeners apply during the performance of the composer's work (Imberty, 1996).
In order to face the issue it might be useful to rely on concepts that are likely to be applied in any acoustic context rather than notions linked with specific musical grammars (e.g. the link between harmony and tonal music).
Within this framework the aim of this study is to investigate the concepts of continuity and discontinuity. The interest in these categories is twofold. 
1)	Continuity and discontinuity may be considered as perceptual qualities that derive from  variations on many of the sonic event dimensions
2) The same categories are likely to be applied over different time spans.
The second part of the paper will describe the basic principles underlying a model for discontinuity amplification of short monophonic fragments. We suppose that listener segmentation of the sonic event may be affected by means of frequency and duration micro-variations that modify discontinuity perception. In the last part experimental results from research on melodic fragments modified according to this model will be briefly outlined. A deeper analysis of the results is included elsewhere in these proceedings (Olivetti Belardinelli et al., 2000).

1. A Continuity/Discontinuity framework
Several studies stress the relation between the variation on certain dimensions of a sonic event  (frequency, amplitude, duration) and listener ability to organize the acoustic surface into meaningful "chunks" or segments.
Segmentation of a sonic event have been linked with amplitude (Fraisse, 1974; Vos, 1977; Gabrielsson, 1973) frequency (Tekman, H.G., 1998), duration (Vos, 1977) and timbre variations (Dowling, 1973).
As well as pitch and loudness are perceptual correlates of frequency and amplitude, discontinuity and continuity may be perceptually correlated with the presence/absence of perceptible variations on these or others dimensions. 
Therefore, from this point of view, subjects' ability to extract cognitive units during the listening experience may be related to their ability in the detection of discontinuities and continuities on multiple acoustic dimensions.
As a consequence of their cognitive nature, continuities and discontinuities may also emerge in absence of any notable acoustic variation. While istening to a regular sequence of pulses, listeners often add discontinuities that lead to the perception of "strong" and "weak" pulses. Moreover, discontinuities on one dimension may emerge from the variation other than the first one. Frequency, amplitude and duration strongly interact in simple acoustic stimuli (Tekman, H.G., 1997). 

The perception of continuities within a stream of sensorial information leads towards the compression and, consequently, to the reduction of the information.
In this case the result of cognitive processing is influenced by subject skillfulness in shadowing non-relevant information.  An example of such selective listening is the classical Cherry's "Cocktail party effect" (reviewed in Wood & Cowan, 1995).  Selective listening is also crucial in auditory scene analysis when we face the problem of  separatating sounds reaching hears simultanously (Bregman, 1990). In other words: the more information reduction lacks to be selective, the more continuity perception leads to a generalized deterioration of the information. On the other hand, selective information reduction facilitate cognitive processing for those entities that exhibit temporal coherence: attenuation of meaningless variations permits the perception of the event as a "whole".  Moreover, unitary perception of one dimension may ease other dimensions processing (e.g. continuity in pulse may facilitate melodic or harmonic processing). For instance, because of possible interference and distortions, amplitude variations of speaking voices are often compressed in radio transmission in order to guarantee a better compression of the spoken text.
Subjects tend to preserve perceived continuities as long as a new variation in the information stream is detected.  This aptitude is coherent with listener tendency to produce expectations that are influenced by past information. (Dowilng, Jay & Harwood, 1986)

On the other hand, discontinuity perception determines an increase of information. In this case the result of cognitive processing rely on the effectiveness of subject's strategies used for information organization. As an example, the more different pitches the composer uses, the strong correlation among pitches should be perceived by the listener in order to process the acoustic information.

Correlation is more evident when discontinuities on two different dimensions take place at the same time (e.g. a simultaneous change in pitch and duration). When that happen information is amplified by means of distinctiveness: stressed variations will lead to the perception of the sonic object in term of multiple units clearly marked by simultaneous discontinuities.
On the contrary, when several discontinuities are perceived within a short time-span, ambiguity may arise from mutual masking, interactions or distortions.

Complexity of twenty-century music often induces listener to focus on continuities. When this is the case continuity perception may sometimes emerge as a cognitive reaction to discontinuities redundancy. As an example, the very complex superimposition of different rhythmic structure in Xenakis, Ligeti, Penderecki music forces listeners to process acoustic information in terms of texture more than in terms of polyphonic contrapoint. On the opposite side, continuity abundance may force listeners to concentrate on few detected discontinuities.  As an example, an expressively call for a sharper discontinuity perception seems to arise from the over-repeated patterns of Reich and Riley "minimal" music.  An even more extreme case might be J. Cage 4'33'': during the silent performance of the pianist listeners are forced to switch their attention to the aleatory small discontinuities produced by the public itself.

Musical practice show several examples in which specific segmentation may be emphasized by means of an amplification of the continuities and discontinuities underlining the musical surface. 
In music performance interpretation of a given score may be suggested through the acoustic realization of a segmentation hypothesis influenced by instrumental peculiarities and esthetical/historical reasons. In a broad sense, also the variation strategies defined by the composer during the different stages of the creative plan may be interpreted as continuity /discontinuity amplification of an initial idea.

Amplification may arise from the increment of the difference between two values on the same dimension (e.g. accentuation of dynamic contrasts). More complex amplifications may be obtained when a discontinuity on one dimension is perceived by means of other dimension variations. For instance, professional baroque music players know very well how to amplify the discontinuity produced by a dynamic accent just with a slight prolongation of tone duration (Harnoncourt, 1984).

Since in most occidental music duration are explicit and dynamic indication are much more vague, discontinuities added by performers mainly arise from variation on amplitudes (accentuation, dynamics) and, to a slighter extend, on duration (articulation). These variations often determine complex timbre alterations specifically related with the acoustic of musical instruments. Other discontinuities may be acquired through temporal deviations from a regular pulse (accelerando / rallentando, rubato).
Commonplace knowledge assumes that the structuring of patterns is affected by the quick amplitude discontinuity (i.e. accentuation) of one or of several elements. These discontinuities are situated most spontaneously at the beginning or at the end of the pattern (Fraisse, 1974, 1982). 
Rhythmic grouping is also influenced by a slight increase in duration. According to Vos (1977) tones that are separated by short intervals are perceptually grouped together. Other studies have shown that easier-to-detect segments have short duration at the beginning of the pattern and long duration at the end. Generally, experimental data confirm most of the assumptions reported by Leopold Mozart, Carl Philip Emanuel Bach and others eighteen and nineteen centuries music theorists. 

2. Principles for discontinuity amplification in melodic processing
Assuming these results within the discontinuity/continuity framework we may hypothesize that strong discontinuities on a certain point of the sonic surface will lead listeners to perceive that point as the beginning or the end of a segment.
Starting from this supposition the model we present in this article is aimed at amplifying discontinuities in short melodic by means of amplitude and temporal micro-variations.
The input of the model is a melodic fragment described only in terms of frequencies and duration (i.e. a MIDI file with timbre and dynamic being constant). The model procedures has been divided into an analysis phase and an amplification phase. The analysis phase include 1) detection and evaluation of a limited number of continuities and discontinuities, 2) production of a "well-formed" set of segmentations (i.e. segmentations that are not inconsistent with the previously detected discontinuities), 3) hierarchical organization of the segments on two layers. In the amplification phase dynamic and temporal micro-variation are added to the notes of the original stimulus in order to emphasize a specific segmentation. The output of the model (Fig.1) is a score where each melodic fragment is presented in two versions. The former presents the results of the first two steps of the analysis phase. The latter present the final two-layer segmentation and the dynamic (accents) and temporal (articulations) micro-variations to be perceived in term of discontinuity amplification.

Continuity/Discontinuity detection and evaluation
Considered discontinuities are linked to the detection of pitch, duration and contour variations while reviewed continuities are linked to the perception of regular pulses.  According to literature on pattern organization (Lerdahl & Jakendoff, 1983 among others) detection criteria are based on Gestalt grouping principles (i.e. Proximity, Similarity, and Good Continuation). Evaluation is mainly based on constraints empirically acquired from musical practice.
Possible differences in continuity/discontinuity strength have been reduced on a strong/weak scale. Strong and weak discontinuity are marked respectively with 'x' and 'o' below the tone that indicate the discontinuity. Vice versa, strong continuities are marked with 'o' above the note where continuity has been detected whereas weak continuities are marked with 'x' (that is continuity/discontinuity notation is discontinuity-oriented).
Continuities and discontinuities may be detected on different temporal spans (i.e. segmentation layers) that are perceived as hierarchically organized in binary or ternary temporal structures (Lerdahl & Jakendoff, 1983). Our analysis phase takes into account the discontinuities only on the lowest temporal level  (i.e. discontinuities between contiguous tones). On the other hand, continuities are considered on a two-layer basis.

Pitch discontinuities detection is based on the following frequency proximity rule. Being two pitches n1, n2, a discontinuity will be marked on n2 if the interval |n1-n2| is above a given threshold value.
On an empirical basis the model take the major second as threshold value (i.e. pitch variation below the minor third threshold do not determine relevant discontinuities) and assign a weak discontinuity (o) if  |n1-n2| > major second or a strong discontinuity (x) if  |n1-n2| > major third.

Duration discontinuities are detected by means of duration similarities rule and onset proximity rule. According to duration similarities rule being two tones with duration d1 and d2, a discontinuity will be marked on d2 if the difference |d1-d2| is above a given threshold value. In our model duration threshold is 1/3 of the shorter tone. A weak discontinuity (o) is assigned if d2<d1; a strong discontinuity (x) is assigned if d2>d1.
Onset proximity rule state that being three tones with onset time t1, t2, t3, a discontinuity will be marked on t2 if the difference between delta times t2-t1 and t3-t2 is above a given threshold value. For the present purposes onset threshold has been empirically set to 1/3 of the shorter delta time. Since a pause physically produce an evident discontinuity pauses above the threshold value always determine a strong discontinuity marking (x).

Contour discontinuities detection is based on Good Continuation principle. Being three pitches n1, n2, n3 a discontinuity will be marked on n3 if n3 show a change in the melodic contour defined by n1 and n2. Within this context a contour change has been marked with a weak discontinuity (o) except for n2≠ n1 and n3=n2 (i.e. the repetition of the same tone). In this case a strong discontinuity (x) has been assigned on n3. 

Regular pulses convey continuity over several temporal spans.
The model takes into account only two regular pulses. The first one (p1) is binary or ternary multiple of the shortest perceptible duration above a defined threshold. Threshold is necessary in order to separate short duration that operates as basic element of a temporal hierarchy from short duration that is inconsistent with rhythmic structure of higher level (e.g. grace notes and embellishments within a defined metrical context).  Because of such lack of coherence the latter is more likely to be perceived as a discontinuity.
Binary or ternary preference is assigned by researching the best-fitting correlation between pulses and onsets of each tone. 
The second pulse (p2) that has been considered in the model is the binary or ternary multiple of p1.  In this case, an evaluation of the correlation between pulses and previously defined discontinuities is invoked in order to assign the preference: the more coincidences between the pulses and the tones marked during the discontinuity detection phase, the more probable the p2 pulse will be.
Pulses are marked above notes. Detected changes in pulse subdivision have been marked as 'x' otherwise a 'o' has been annotated. A change in pulse that occurs just on the last note of the excerpt due to an evaluation of its length has not been considered.

Segment definition
Early research (reviewed in Fraisse, 1982) has shown that extreme temporal limit for the perception of a single group can not be more than 5 sec. According to these results our model allows continuity/discontinuity amplification only on three temporal layers. The longest (L3) coincides with the beginning and the end of 7-9 sec. melodic fragment, and has not been marked on the score.  The second (L2) divide the excerpt in two or three segments and is marked. The latter (L1) divide each L2 segment in 2-3 sub-segments. A third sub-level (L0) is not considered in the amplification phase but is used to generate segments that will be subsequently grouped on the L1 layer.
The segment production step is threefold. Firstly, all possible L0 segments are produced by means a small set of rules. Secondly, produced segments are reduced by means of constrains. Thirdly, segments are grouped on L1 and L2 layer through a set of other constrains.
Rules used for L0 segments are the followings. 
1) Each segment must contain at least two distinct sounds.
2) Each segment must begin on a marked tone and should end on marked or unmarked tone.
Constrains used to reduce the number of L0 segments may be biased either by discontinuity or continuity. In our hypotesis continuity and discontinuity behave as powerful structuring forces in subjective auditory organization during music composition, performance and listening. Emphasis on one or the other relies upon individual tendencies but has its roots in historical and esthetical traditions. For instance, today baroque music performances emphasize discontinuities whereas continuities were much more stressed in early twenty-century interpretations (Harnoncourt, 1984).
Constrains based on discontinuity stress the contrast among inner differences thus induce further subdivisions of the excerpt. On the other hand, more continuity-oriented constrains often lead to the perception of a unified auditory image.
In the present version our model is more influenced by discontinuity constrains. That is preferred segments to be grouped in level L1 are those that begin and end with tones marked with a discontinuity. 
Each group of two or three L0 "well-formed" segments will produce a segment on L1 layer. Segments of similar duration have been preferred among others. That means that segments of different length are likely to be assigned to different layers more than grouped on the same temporal span. L1 segments are combined into L2 segments according to the same principles.

Discontinuity amplification
In the discontinuity amplification phase amplitudes and duration discontinuities are added on each well-formed segment of L1 and L2 layers. Discontinuity positioning could not be defined by strict rules but depends, once again, on a fairly variable combination of discontinuity and continuity constrains.
Nevertheless, once defined, introduced discontinuities must cohere over the different layers. In other words, the positioning of the discontinuity on a higher layer segment must coincide with one of the discontinuities introduced on the sub-divisions of that segment.
Discontinuity strength varies according to the layer of the segment: the higher is the segmentation layer the stronger the discontinuity will be.
In our model, in order to make subjects fell metrical continuity within each melodic excerpt, amplitude discontinuities are added on notes that are marked both with discontinuities and pulse continuities. Amplitude discontinuities are marked with an accent (>) on the output score.  Duration discontinuity is added by shortening the last note of each segment and is indicated with a staccato (.). The dimension of the sign varies according to the strength of the discontinuity.

3. Experimental results
An indirect validation of the model can be found in the experimental results from researches on recognition memory for unknown musical fragments  (Olivetti Belardinelli et al., 2000).  Experimental materials for these researches derive from 48 melodic fragments devised by the author for experiment on recognition memory (Olivetti Belardinelli, Cifariello Ciardi, Rossi-Arnaud, 1998). The original MIDI files used to record the original set of stimuli have been modified according to the described model and recorded using a Digidesign-Sample Cell piano tone.  Introduced amplitude and duration discontinuities were realized with a 70% increase in velocity (i.e. a MIDI correlated for loudness) and a 40% decrease in duration on L2 segments. Discontinuities on L1 segments were realized with a 50% increase in velocity (i.e. a MIDI correlated for loudness) and a 20% decrease in duration.  Experimental data  show that subjects are more likely to remember stimuli affected by discontinuity amplification.

4. Conclusions
We are cautious about conclusions from the above mentioned experiments because not directly designed to test the model. This issues aside, results seem to suggest that a discontinuity amplification may improve melodic processing regardless of any musical language (e.g. tonal, atonal, serial). Moreover, being acoustic sequential events organized according to criteria of spectral continuity (Mc Adams, 1984), a continuity/discontinuity framework would be of interest because it provides hints for a generalized analysis of sonic surfaces.
Clearly, future research must provide experimental evidences for many of the empirical constrains included in the model. Moreover, since analysis phase is based on relatively few sonic variations further studies has to determine to which extend discontinuities emerging from more complex parameters interactions (e.g. timbre and dissonance variations, symmetry detection) affect listener auditory organization.

5. References
Bregman, A.S. (1990).  Auditory scene analysis. Cambridge, MA, MIT Press.
Deutsch, D. (1982). Grouping Mechanisms in Music in D.Deutsch (Eds.). The Psychology of Music. New York, NY, Academic Press.
Dowilng, W.J. (1973) The perception of Interlaved melodies. Cognitive Psychology, 5, 322 - 337.
Dowilng, W.J. Harwood, D. L. (1986), Music Cognition, New York, NY, Academic Press.
Fraisse, P. (1974). Psychologie du ritme. Paris: Presses Univeritaries de France.
Fraisse, P. (1982). Rhythm and Tempo in D.Deutsch (Eds.)The Psychology of Music. New York, NY, Academic Press.
Frieberg, A. & Sundberg J. (Eds.) (1995). Grammars for music performance. Proceedings of the KTH Symposium. Stockholm: KTH
Harnoncourt, N. (1984). Der musikalische Dialog. Salzburg: Residenz Verlag.
Imberty, M. (1996). Ordine e disordine. Un punto di vista psico-cognitivo sulla creazione musicale, in Di Matteo R. (a cura di) Psicologia Cognitiva e Composizione Musicale. Roma: edizioni Kappa.
Lerdahl, F., Jakendoff, R. (1983).  A Generative Theory of Tonal Music. Cambridge, MA, MIT Press.
Olivetti Belardinelli, M., Cifariello Ciardi F., Rossi-Arnaud, C. (1998). Recognition memory for previously novel musical themes in children, in Proceedings of XV Congress of the International Association of Empirical Aesthetics, Roma: Edizioni Universitarie Romane, Roma.
Olivetti Belardinelli, M., Rossi-Arnaud, Pitti G.,  Vecchio S. (2000). Looking for the anchore points for musical memory. ICMPC2000 proceedings.
Tekman, H.G. (1997). Interaction of Percieved of Intensity, Duration, and Pitch in Pure Tone Sequences. Music Perception, 14, 281-294.
Tekman, H.G. (1998).  Effects of Melodic Accents on Perception of Intensity. Music Perception, 15, 391-401.
Wood, N.L., Cowan, N. (1995) The Cocktail Party Phenomenon Revised: Attentio and Memory in the Classic selective Listening Procedure of Cherry (1953). Journal of Experimental Psychology: General, 124, 243-262.