Differences

This shows you the differences between the selected revision and the current version of the page.

software:om-pursuit 2012/06/14 23:59 software:om-pursuit 2014/09/15 14:02 current
Line 1: Line 1:
====== OM-Pursuit ====== ====== OM-Pursuit ======
-Dictionary-Based Sound Analysis/Synthesis in Computer-Aided Composition.+Dictionary-Based Sound Modeling in Computer-Aided Composition.
{{template>:software_summary {{template>:software_summary
-|participants=[[people: Marlon Schumacher]]\\ [[people:Marcelo M. Wanderley]] (supervisor+|participants=[[people: Marlon Schumacher]]\\ Graham Boyes (collaborator
-|funding= CIRMMT student award 2011/2012+|funding= CIRMMT student award 2011/2012 \\ FQRNT DED scholarship \\ CIRMMT Director's Interdisciplinary Excellence Prize
|license= GPL |license= GPL
-|period= 2011 - 2012+|period= 2011 - 2014
|status= ongoing |status= ongoing
|image=software:om-pursuit:logo3.png |image=software:om-pursuit:logo3.png
Line 34: Line 34:
-===== Overview ===== 
-[[https://github.com/marleynoe/OM-Pursuit|OM-Pursuit]] is a library for dictionary-based sound modelling in computer-aided composition.  
-Parametric sound representations provide interesting conceptual models for music composition (see e.g. the French spectralist school). A number of software tools are available allowing composers to integrate signal and symbolic data within a compositional framework, such as extracting frequency structures from a sound file via partial tracking. Most of these tools, however, are based on sinusoidal sound models and hence rely on certain assumptions which are less adequate for non-stationary sounds containing noise and transients, for example. They typically require large numbers of individual components to describe these types of signals and poorly represent their temporal fine-structure. Dictionary-based methods, instead, represent sound as a linear combination of atoms stored in a dictionary. Mostly used for sparse audio coding purposes (compression, transmission), we propose creative applications for this model within the context of computer-aided composition, orchestration and transcription.  
-Using sound samples as atoms for the dictionary we create the 'timbral' basis of our analysis/synthesis system. Dictionaries can be built from arbitrary collections of sounds, such as instrumental samples, synthesized sounds, field recordings, etc. Dictionaries can be freely combined and pruned on via sound descriptors. 
-Due to the specifics of working with sound samples (as opposed to parametric basis functions), we developed an adapted matching pursuit algorithm (implemented in [[http://www.github.com/gboyes/pydbm|pydbm]] by G.Boyes) for sparse decomposition, which allows approximating a given target sound via combination of the samples in the dictionary. This can in a sense be compared to techniques for creating photo mosaics in the visual domain.  
-The resulting model is an abstract description, which can be visualized and edited in OpenMusic using different representations (spatial, tabulated, temporal). The model can then be used to control sound synthesis processes, or for orchestrations and instrumental transcriptions. 
----- 
-{{software:om-pursuit:dict2.png?320|}}{{software:om-pursuit:om-pursuit-ov2.png?320|}} 
----- 
-Atomic Decomposition offers interesting possibilities for musical applications:  
-  * Iterative Approximation. Control over degree or resolution of the model 
-  * Approximation is signal-based rather than descriptor-based  
-  * Dictionaries can be arbitrarily designed and combined (sample libraries, field-recordings, etc.) 
-  * Model is naturally polyphonic (combination of superposed atoms) 
-  * Constrained Matching Pursuit (control of horizontal/vertical distribution of atoms) 
-  * Process leaves a "residual" - Multiple iterative decompositions (serial, parallel) of the same target are possible 
-Interesting applications are possible in combination with [[software:OMPrisma|OMPrisma]] (dictionary-based spatialization), and [[software:om-sox|OM-SoX]] (batch-processing on the dictionaries, e.g. transpositions, filterings, etc.). 
-OM-Pursuit was awarded the [[http://www.cirmmt.mcgill.ca/activities/support/student/past/awards11-12|CIRMMT director's interdisciplinary excellence price 2011]]. 
----- 
Line 90: Line 71:
-===== Sound Examples ===== 
 +===== Overview =====
-=== Target ===+[[https://github.com/marleynoe/OM-Pursuit|OM-Pursuit]] is a library for dictionary-based sound modelling in OpenMusic.
-  * Bird's Call +Parametric sound representations have since long served as conceptual models in composition contexts (see e.g. the French spectralist school). Today there are a number of software tools allowing composers to derive symbolic data from continuous sound phenomena, such as extracting frequency structures via sinusoidal partial tracking. Most of these tools, however, are based on the Fourier transform -which decomposes (static) frames of a time-domain signal into sinusoidal (frequency) components. This method can be less adequate to faithfully represent non-stationary sounds such as noise and transients, for example. Dictionary-based methods offer a different model, representing sound as a linear combination of individual atoms stored in a dictionary. Mostly used for sparse representation of signals for audio coding purposes (compression, transmission, etc), this atomic model offers interesting new possibilities for computer-aided composition applications, such as algorithmic transcription, orchestration, and sound synthesis.
-<html> +
-<object height="81" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47482262"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47482262" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-short-16k">Bird's Call short 16k</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  +
-</html>+
-=== Models ===+Using sound samples as atoms for the dictionary we can design the 'timbral vocabulary' of an analysis/synthesis system. The dictionaries used to analyze a given audio signal can be built from arbitrary collections of sounds, such as instrumental samples, synthesized sounds, recordings, etc. OM-Pursuit uses an adapted matching pursuit algorithm (see [[http://www.github.com/gboyes/pydbm|pydbm]] by G.Boyes), to iteratively approximate a given sound using a combination of the samples in the dictionary -in a way comparable to photo-mosaicing techniques in the visual domain. 
 + 
 +Communication between OpenMusic and the dsp-kernel (pydbm-executable) is handled via an SDIF interface and scripts which are generated and manipulated using the visual programming tools in the computer-aided composition environment. The decomposition of a target sound results in a sound file (the model), the residual (difference between target and model), and a parametric description of the model. We developed a number of tools for visualization and editing of this model using different representations (spatial, tabulated, temporal). The model-data can be used and manipulated within OpenMusic in a variety of contexts, from the control sound spatialization and synthesis processes, for computer-aided orchestration and -transcription, to graphical notation of musical audio.
-  * Piano (10000 iterations) 
-<html> 
-<object height="41" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483314"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483314" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-piano-10000">Bird's Call Piano 10000</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  
-</html> 
-  * Piano (1000 iterations) 
-<html> 
-<object height="81" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483353"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483353" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-piano-1000">Bird's Call Piano 1000</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  
-</html> 
-  * Piano (100 iterations) 
-<html> 
-<object height="81" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483991"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483991" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-piano-100-1">Bird's Call Piano 100</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  
-</html> 
-  * Piano (10 iterations) 
-<html> 
-<object height="81" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483427"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483427" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-piano-10">Bird's Call Piano 10</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  
-</html> 
-  * Clavichord 
-<html> 
-<object height="81" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483514"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483514" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-clavichord">Bird's Call Clavichord</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  
-</html> 
-  * Lautenwerck 
-<html> 
-<object height="81" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47485043"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47485043" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-lautenwerck">Bird's Call Lautenwerck</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  
-</html> 
-  * Knocks on Grand Piano 
-<html> 
-<object height="60" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483828&amp;show_comments=true&amp;auto_play=false&amp;color=ff7700"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483828&amp;show_comments=true&amp;auto_play=false&amp;color=ff7700" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-piano-knocks">Bird's Call Piano Knocks</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span> 
-</html> 
-  * Piano Sustain (no attack) 
-<html> 
-<object height="81" width="100%"> <param name="movie" value="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483784"></param> <param name="allowscriptaccess" value="always"></param> <embed allowscriptaccess="always" height="81" src="https://player.soundcloud.com/player.swf?url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F47483784" type="application/x-shockwave-flash" width="100%"></embed> </object>  <span><a href="http://soundcloud.com/marleynoe/birds-call-piano-sustain">Bird's Call Piano Sustain</a> by <a href="http://soundcloud.com/marleynoe">marleynoe</a></span>  
-</html> 
---- ----
 +{{software:om-pursuit:dict2.png?320|}}{{software:om-pursuit:om-pursuit-ov2.png?320|}}
 +----
 +Atomic Decomposition offers interesting possibilities for musical applications:
 +
 +  * Iterative Approximation. Control over degree/resolution of the model
 +  * Approximation is based directly on the signal rather than signal descriptors
 +  * Dictionaries can be arbitrarily designed and combined (sample libraries, field-recordings, etc.)
 +  * Model is naturally polyphonic (juxtaposition and superposition of sound grains)
 +  * Constrained Matching Pursuit (constraint-based control over distribution of sound grains)
 +  * Process leaves a "residual" - Multiple iterative decompositions (serial, parallel) of the same target are possible, e.g. for different orchestrations
 +
 +Interesting applications are possible in combination with [[software:OMPrisma|OMPrisma]] (dictionary-based spatialization), and [[software:om-sox|OM-SoX]] (batch-processing for designing dictionaries, e.g. transpositions, filterings, etc.).
 +
 +
 +
 +OM-Pursuit was awarded the [[http://www.cirmmt.mcgill.ca/activities/support/student/past/awards11-12|CIRMMT Director's Interdisciplinary Excellence Prize]].
 +----
 +
 +In a recent project we have employed OM-Pursuit for the notation of musical audio:
 +
 +The notation of musical audio (such as a tape-part in electroacoustic music) is typified by two extremes on a continuum: On the one end are signal-based notations, such as sonograms, waveforms, etc. -which however don’t provide information about musical semantics. On the other end are symbolic notations, such as icons, characters, etc. -which in turn don’t provide information about the sonic details. Both aspects, however, are important for the interpretation of a musical work, in particular for contemporary music, which does not rely on common musical idioms. Using dictionary-based approaches (OM-Pursuit) an audio signal can be analyzed / transcribed based on a user-defined collection of smaller-scale sound files (so-called ‘atoms’). The individual atoms can be associated with arbitrary visual representations (symbolic, iconic, etc.). The user can then design simple programs to algorithmically arrange these visual elements on a 2D-canvas into a kind of cartographic map of the higher-level sonic content in the signal. Below are some screenshots from early experiments with the system. There's also one from my piece for 2 pianists and tape "6 Fragments on 5’05” - 1 Act of cleaning a piano".
 +
 +This work was presented at the 1st Symposium on Music Notation at Université IV, Paris Sorbonne, organized by AFIM/MINT-OMF workgroup [[http://notation.afim-asso.org|Les nouveaux espaces de la notation musicale]].
 +
 +----
 +{{software:ompursuit-notation-1.png?640|}}
 +
 +{{software:ompursuit-notation-4.png?640|}}
 +
 +{{software:ompursuit-notation-5.png?640|}}
 +----
 +
 +
 +===== Sound Examples =====
 +
 +See the set on [[https://soundcloud.com/marleynoe/sets|soundcloud]] for some audio examples.
Line 149: Line 133:
---- ----
 +
 +
 +
Line 156: Line 143:
OM-Pursuit has been used for the composition of a number of works, most notably: OM-Pursuit has been used for the composition of a number of works, most notably:
 +
 +  * "Spielraum" for Violin, Cello, 2 dancers and New Digital Musical Instruments (Marlon Schumacher, 2013). \\ Commissioned for the research-creation project [[projects:gestes|"Les Gestes"]]. Premiered at [[http://www.agoradanse.com/en/spectacles/2013/les-gestes|Agora de la danse]], Montreal, Canada, March 2013.
  * "Spin" for extended V-Drums, Interactive Video and Live-Electronics (Marlon Schumacher, 2012). \\ Commissioned by Codes D'Acces. Premiered at the [[http://www.codesdacces.org/2012/03/20/prisma-et-les-messagers/|"Prisma et Les Messagers"]] concert, Usine-C, Montreal, Canada, April 2012.   * "Spin" for extended V-Drums, Interactive Video and Live-Electronics (Marlon Schumacher, 2012). \\ Commissioned by Codes D'Acces. Premiered at the [[http://www.codesdacces.org/2012/03/20/prisma-et-les-messagers/|"Prisma et Les Messagers"]] concert, Usine-C, Montreal, Canada, April 2012.
  * "Ab-Tasten" for Disklavier, 4 Virtual Pianos and Sound Synthesis (Marlon Schumacher, 2011). \\ Commissioned by [[http://www.cirmmt.mcgill.ca/|CIRMMT]]. Premiered at the live@CIRMMT concert [[http://www.cirmmt.mcgill.ca/activities/live-cirmmt/clavisphere/?searchterm=clavisphere|Clavisphere]], Montreal, Canada, 2011. \\   * "Ab-Tasten" for Disklavier, 4 Virtual Pianos and Sound Synthesis (Marlon Schumacher, 2011). \\ Commissioned by [[http://www.cirmmt.mcgill.ca/|CIRMMT]]. Premiered at the live@CIRMMT concert [[http://www.cirmmt.mcgill.ca/activities/live-cirmmt/clavisphere/?searchterm=clavisphere|Clavisphere]], Montreal, Canada, 2011. \\
 +    * [[http://www.music.mcgill.ca/~marlon/audio/Abtasten/Abtasten%20(for%20Disklavier%20and%20Electronics).m4a|Binaural Live-Recording]] McGill MultiMedia Room, April 14, 2011.
---- ----
-{{tag>"computer-aided composition" "computer-aided orchestration" "sparse decomposition" "audio transcription"}} +{{tag>"computer-aided composition" "computer-aided orchestration" "sparse decomposition" "audio transcription" Mapping Audio-Musaicing}}