The Concatenator samples sound grains from a corpus to match a target audio stream in real time using a particle filter. Like Driedger's Let It Bee technique, The Concatenator preserves the timbre of the corpus, while matching the pitch and rhythm of the target. Keep scrolling for videos and examples!
Click here to download our paper on this, which is part of the proceedings of ISMIR 2024. If you use this in your work, please cite as follows:
Christopher J. Tralie and Ben Cantil. ``The concatenator: A bayesian approach to real time concatenative musaicing.'' In Proceedings of the 25th Conference of the International Society for Music Information Retrieval (ISMIR 2024).
Click here to download our python prototype, which includes a real time option! Have a look at the Readme.md
file for dependencies and instructions for use.
Below is a video showing the potential of our plugin to be used real time "in concert" (pun intended) with Ableton Live (Click here to download the video file).
DataMind Audio, a music AI startup, is working on a proprietary plugin with a much fancier user interface based on the ideas in this paper. Below is a demo of this plugin, courtesy of Dr. Martin Parker, showing how to create live foley of someone walking through mud using The Concatenaton on top of this video https://www.youtube.com/watch?v=ZYXzzRb--gI (BASCfilms, dir. 2011. Mud Walking). (Click here to download the video file of Martin's demo)
Below are some of the corpus/target matrices we created for our qualitative analysis (corpora displayed on rows, targets displayed on columns). All results are presented at 44.1khz stereo. Every result was generated using P=1000 particles, p=5 activations per particle, τ=10, and pd=0.95. With this number of particles, all of these examples run at real time or faster on commodity hardware.
Note that a 4096 window has a better frequency resolution, which is particularly noticeable at lower pitches. But it may also miss some transients using the same parameters as above.
Below we highlight some of our favorite examples
Corpus | Target | Result |
---|---|---|
Beethoven - Symphony No. 5 in C Minor |
Beatles_LetItBe |
|
Beatles_LetItBe |
Notice how a window of 4096 does better with the bassoon due to increased frequency resolution |
|
Ripple Contiuum - filt saw - 3 octaves |
|
|
Mr. Bill-Spectra Sample Pack Excerpt |
Drums Bass Test |
|
Vocals |
Pink noise sweep |
|
Percussive |
Beatbox Test |
|