The Concatenator

Christopher J. Tralie, Ben Cantil

The Concatenator samples sound grains from a corpus to match a target audio stream in real time using a particle filter. Like Driedger's Let It Bee technique, The Concatenator preserves the timbre of the corpus, while matching the pitch and rhythm of the target. Keep scrolling for videos and examples!


Paper

Click here to download our paper on this, which is part of the proceedings of ISMIR 2024. If you use this in your work, please cite as follows:

Christopher J. Tralie and Ben Cantil. ``The concatenator: A bayesian approach to real time concatenative musaicing.'' In Proceedings of the 25th Conference of the International Society for Music Information Retrieval (ISMIR 2024).


Code

Click here to download our python prototype, which includes a real time option! Have a look at the Readme.md file for dependencies and instructions for use.


Poster


Demo Videos

Below is a video showing the potential of our plugin to be used real time "in concert" (pun intended) with Ableton Live (Click here to download the video file).

DataMind Audio, a music AI startup, is working on a proprietary plugin with a much fancier user interface based on the ideas in this paper. Below is a demo of this plugin, courtesy of Dr. Martin Parker, showing how to create live foley of someone walking through mud using The Concatenaton on top of this video https://www.youtube.com/watch?v=ZYXzzRb--gI (BASCfilms, dir. 2011. Mud Walking). (Click here to download the video file of Martin's demo)


Qualitative Analysis

Below are some of the corpus/target matrices we created for our qualitative analysis (corpora displayed on rows, targets displayed on columns). All results are presented at 44.1khz stereo. Every result was generated using P=1000 particles, p=5 activations per particle, τ=10, and pd=0.95. With this number of particles, all of these examples run at real time or faster on commodity hardware.

winLength = 2048

winLength = 4096

Note that a 4096 window has a better frequency resolution, which is particularly noticeable at lower pitches. But it may also miss some transients using the same parameters as above.

Our Favorites

Below we highlight some of our favorite examples

Corpus Target Result

Beethoven - Symphony No. 5 in C Minor

Beatles_LetItBe

win=2048

win=4096

1.6 hours of woodwinds

Beatles_LetItBe

win=2048

win=4096

Notice how a window of 4096 does better with the bassoon due to increased frequency resolution

1.6 hours of woodwinds

Ripple Contiuum - filt saw - 3 octaves

win=2048

win=4096

Mr. Bill-Spectra Sample Pack Excerpt

Drums Bass Test

win=2048

win=4096

Vocals

Pink noise sweep

win=2048

win=4096

Percussive

Beatbox Test

win=2048

win=4096