A while back, I tried recreating the sound of a dialup modem from scratch, which involved a dive into multiple PDFs of telecommunication standards. (I am not the first person to do this.) Very helpful was Windy Tan’s beautiful analysis breaking down a spectrogram of a real dialup handshake. A particularly famous bit is the nasal mid-low “whine” at the end of the handshake, and if you’ve ever wondered what that is, you’re hearing line probing signals L1 and L2, which are synthesized using additive synthesis. The exact partial frequency, phases, and amplitudes are specified in section 10.1.2.4 of ITU-T V.34. They’re at 9 seconds in this recording of a modem by thearchiveguy99 on Freesound:

WARNING: Audio files in this post are loud and piercing. Please turn up your speakers to extremely painful levels for optimal telecommunication.

The whine is legendary, but I want to turn your attention to high-frequency gurgles before it, starting at about 4 seconds in, some of them resembling slurping a last bit of soda through a straw in a McDonald’s cup. These tones, specified at a high level in ITU-T V.8bis, are encoded using frequency-shift keying (FSK) as given in the lower-level ITU-T V.21 spec. In its simplest form, FSK uses a single sine wave to encode a binary signal by modulating its frequency. Following V.21’s description of a “channel 1” signal, at regular intervals of 1/300 seconds, a single bit is transmitted. If the bit is 0, the sine wave’s frequency is set to 1180 Hz. If the bit is 1, the frequency is set to 980 Hz.

Yep, that’s a dialup gurgle. The graph below demonstrates this visually (for different parameters — I had to lower the carrier for the 1 bits so that the distinction between the two is obvious). Above we have the audio waveform, below it the data signal.

Designing that dialup imitation made me realize that I really like how FSK sounds purely as a sound design tool. This got me looking into its various siblings, which are known as the “digital modulation” schemes for transmission of digital signals over an analog medium such as radio. In contrast, in “analog modulation” the signal being transmitted is continuous, encompassing familiar techniques like FM, AM, and single-sideband modulation. Today we’re going to honor the long tradition of adapting telecommunications methods for creative use by seeing what kind of sounds we can design by directly listening to signals produced with FSK and related methods. I call this approach Digital Modulation Synthesis, a name I’m not in love with, but I couldn’t think of a better one. It may be abbreviated to the cooler-sounding “DM Synthesis.”

In telecommunications, modulation is only half the story, and you also need a demodulator which takes the transmission signal and gets the data back. Engineering modulators and demodulators in a way that’s robust to noise is an interesting problem subject to nearly a century of study, but here we’re going to be the weirdos who just want to listen to the modulated signal itself. I will spend no time here discussing the engineering tradeoffs of these digital modulation schemes in a telecommunication settings; those problems are very interesting and deep but not relevant to this post.

This post is pretty light on math, and the DSP isn’t fancy here, as our concentration is more on sound design and synthesis. All this should be doable in SuperCollider, Pd, Max, etc. with pre-existing units.

I didn’t research this post as thoroughly as I would have liked, so it’s not impossible that I have misrepresented things from the telecommunications literature. If there are mistakes, sorry about that.

Read more…

Synthesizer forums have a long-running joke about “Gear Acquisition Syndrome” that pictures the stereotypical hobbyist with all their Hainbach-endorsed gear, seemingly oblivious to the fact that they’re intended for making music. Trite as the joke is, the culture that reduces synthesizers to boutique furniture deserves all the mockery it gets. I’m a bit annoyed that I don’t remember who said this: Eurorack is millennials’ equivalent of model trains.

A lot of the audience of this blog is electronic music DIYers and open source fanatics, and many of us have good reasons to sneer at this culture. I myself own very little music hardware and basically never use VSTs, and I admit I feel a little smug when I read about how much money people can blow on synths and even plugins. Then I remember my own history with electronic music.

When I was first getting into SuperCollider — my first ever experience making electronic music — I found my way to DSP conference papers and became totally obsessed. I read every DAFx paper that seemed remotely interesting to me; I learned C++ to implement these in my own UGens. I was pretty certain that with enough effort I could replace any VST plugin I read about.

I thought I was being clever circumventing GAS by building everything myself, but what really happened is that I had fallen into my own version of GAS: I had System-Building Syndrome. System-building can be personally enriching, and no doubt more cost-effective than buying roomfuls of synths, but if your mission is to produce and release music then getting into engineer mode will actively hinder your efforts to do so. When I sat down to make music with all the nice fancy UGens I made, my music sucked. It discouraged me from making more music, so I’d spend more time building UGens since clearly I didn’t have enough of them.

Read more…

(original)

(processed)

I would like to congratulate wavesets (not wavelets) for entering their 30th year of being largely ignored outside of a very small circle of computer music nerds. Introduced by Trevor Wishart in [Wishart1994] and popularized by Microsound [Roads2002] and the Composers Desktop Project, a waveset is defined as a segment of an audio signal between two consecutive upward zero crossings. For simple oscillators like sine and saw waves, wavesets divide the signal into pitch periods, but for general periodic signals there may be any number of wavesets per period. For signals containing noise or multiple pitches at once, waveset segmentation is completely unpredictable.

Many simple audio effects fall out of this idea. You can reverse individual wavesets, omit every other waveset, repeat each waveset, sort them, whatever.

I like waveset-based effects best on input signals that are monophonic (having only one pitch) and low in noise. Synthetic signals can make for particularly interesting results. Much as the phase vocoder tends to sound blurry and phasey, waveset transformations also have their own “house style” in the form of highly digital glitches and crackles. These glitches are particularly pronounced when a waveset-based algorithm is fed non-monophonic signals or signals containing strong high-frequency noise. Wavesets are extremely sensitive to any kind of prefiltering applied to the input signal; it’s a good idea to highpass filter the signal to block dc, and it’s fun to add pre-filters as a musical parameter.

Today, we’re putting a possibly new spin on wavesets by combining them with basic statistical learning. The idea is to perform k-means clustering on waveset features. The steps of the algorithm are as follows:

1. Segmentation: Divide a single-channel audio signal into \(N\) wavesets.

2. Analysis: Compute the feature vector \(\mathbf{x}_i\) for the \(i\)-th waveset. I use just two features: length \(\ell_i\), or number of samples between the zero crossings, and RMS \(r_i\) of the waveset’s samples. All the lengths are compiled into a single size-\(N\) vector \(\mathbf{\ell}\) and the RMSs into \(\mathbf{r}\).

3. Normalization: Scale \(\mathbf{\ell}\) and \(\mathbf{r}\) so that they each have variance 1.

4. Weighting: Scale \(\mathbf{\ell}\) by a weighting parameter \(w\), which controls how much the clustering stage emphasizes differences in length vs. differences in amplitude. We’ll talk more about this later, but \(w = 5\) seems to work as a start.

5. Clustering: Run k-means clustering on the feature vectors \(\mathbf{x}\), producing \(k\) clusters.

6. For each cluster, pick one representative waveset, the one closest to the centroid of the cluster.

7. Quantization: In the original audio signal, replace each waveset with the representative waveset from its cluster.

Implementation is very lightweight, clocking in at about 30 lines of Python with scikit-learn. There are only two parameters here other than the input audio: \(w\) and \(k\). [1] The length of the input audio signal is important too, so for musical reasons let’s not think in terms of \(k\) but rather “clusters per second” \(c\), which is \(k\) divided by the signal length in seconds. As we will see, with only two-dimensional control we can produce a tremendous variety of sounds.

Read more…

Let’s open this post with music. If you aren’t familiar with the late composer Roland Kayn’s work, listen to one of his Electronic Symphonies, or at least sample a few minutes. The album cover links to his official Bandcamp.

I found out about Kayn’s music through a thread on Lines, and it gets only more interesting as I looked into his story. Kayn was an exponent of combining music with cybernetics, a field I gloss as “the study of feedback systems.” At its heart, it’s a mathematical study; Norbert Wiener’s original text Cybernetics is chock full of integrals and differential equations. As the field developed, it overlapped significantly with medicine, social sciences, humanities, and especially critical theory. I do hope to read Wiener’s Cybernetics and The Human Use of Human Beings, but haven’t gotten around to them, so consider my understanding of the field surface-level — I’m mainly concerned with cybernetics as it relates to Kayn’s music.

Kayn began making cybernetics-influenced electronic works in university electronic music studios in the 60’s. We don’t know what exact tools, but Pickles’ 2016 dissertation Cybernetics in music surmises that the equipment back then was something along the lines of tape machines and basic analog processors and generators: oscillators, noise, filters, reverbs, envelope generators, ring modulators. More intriguingly, recordings of his works were entirely real time — a stark contrast to the offline tape-based processes of the contemporary musique concrète movement. Critic Frans van Rossum wrote in 2011 that Kayn worked with a “network of electronic equipment” and a “a system of signals or commands that it can obey and execute” (quote also via Pickles).

EDIT 2023-12-20: An earlier version of this post credited the technology to Kayn alone, but I have since been properly schooled: it was Jaap Vink who actually built the system, as Sascha Frere-Jones wrote in 4Columns. Given the intimacy that cybernetic musicians necessarily have with their tools, his fingerprints are deeply embedded in Kayn’s electronic works. The sources I looked at didn’t make this clear to me, and most don’t even mention Vink by name. Vink referred to himself as a technician, not a composer, and it’s possible that the concealment of his status was his preference.

That these works were likely made on 60’s equipment, were recorded in real time, and sound varied and dynamic over long stretches of time should impress anyone familiar with modular synths. The implications that his musical systems had some kinds of self-generating and autonomous properties makes this even more remarkable. Now Kayn’s output is more than just great music, it’s now a puzzle. How did he make this music? Can we recreate part of his process?

I don’t think we’ll ever get definitive answers. Kayn died in 2011, Vink in January 2023. While there’s a small body of writing about them, little of it is technical. We can only guess, but we have some resources that helps us with the guessing process — the Lines thread I mentioned, and also La Synthèse Humaine, a YouTube channel run by musician Émile Zener a.k.a. Gunnar Haslam. He takes direct inspiration from Kayn, and unlike me has extensively studied the cybernetics literature. I won’t take the time to resummarize the Lines thread or Zener’s work — just go check them out for yourself — instead I am interested in expanding on them.

Making a cybernetic LFO

It’s perhaps easiest if I first walk you through a concrete example of building a cybernetic patch. We begin with the simple harmonic oscillator, which can be implemented with the system of ordinary differential equations

\begin{equation*} \begin{align*} \dot{a} &= -kb \\ \dot{b} &= ka \\ \end{align*} \end{equation*}

and the following block diagram:

This comprises two integrator stages in a feedback loop. An integrator is a unit that continuously accumulates the input signal (you can get something like a leaky integrator with a one-pole lowpass filter with a very low cutoff). The triangles are simple amplifiers multiplying by constants; one must be negative and the other positive. There are two outputs, A and B, which produce a sine and cosine wave respectively. The system does need an initial “kick” to generate sine waves, by e.g. initially setting \(a(0) = 1\) and \(b(0) = 0\).

Assume that the oscillator is tuned to oscillate at a slow 1 Hz. It is an “LFO” in the literal sense that it oscillates at a low frequency, but critically we aren’t generating it with an actual oscillator module, and the oscillation is an emergent property of the system that we built. We can turn this into an audible synthesis patch by using the two outputs, A and B, as amplitudes, multiplying them with VCAs by two saw oscillators with static frequencies. Mixing the oscillators together, we have a simple looping alternation between two “notes” even though we have no actual sequencer here.

However, the oscillators and VCA’s aren’t part of a loop, and their output does not influence the rest of the system. For reasons that I’ll justify later, we loop in their VCAs by running an envelope follower on the output of the VCAs, and using it to modulate the coefficients of other VCAs that are in the loop.

(I’m misusing notation here a bit; the arrows into the loop VCAs are not actually setting the gain, but wiggling around the gain it had in the previous diagram. The intentions of this diagram are more romantic than schematic — the point is to illustrate the closing of loops, not to specify an exact algorithm.)

This isn’t an actual simple harmonic oscillator anymore, but it mimics the core behavior of one. However, if we start changing any parameter, the core behavior of the “LFO” starts changing too. Everything interacts with everything, and nothing is isolated or purely upstream or purely downstream.

We’ll return to this patch and expand it into something more compelling in sound design terms. Let’s go theoretical for a bit and explain what we’re actually trying to do here.

Read more…

As 2023 ends, it seems like a good time to reflect and allow myself some vanity. It’s also a few days shy of the 15th anniversary of the first thing I created on the Internet (a certain wiki I have mentioned before). Amusingly, I’m really doing the same thing as I was back then: publicly writing about niche technical subjects. Back then it was mathematics, now music technology.

I set a goal for myself to release at least one substantial technical blog post every month this year, and succeeded:

• An Approach to Sound Synthesis with L-Systems

• A Closer Look at Clarence Barlow’s ISIS

• Negative Compression

• Opinionated Advice for SuperCollider Beginners

• Audio Texture Resynthesis

• A Preliminary Theory of Sound Design

• The Duration Trick

• An Intro to Wavelets for Computer Musicians

• Composing with Accelerating Rhythms

• Making Synthesized Sounds More Acoustic

• Audio Effects with Cepstral Processing

• Cybernetic Synthesis and Roland Kayn

Outside of writing, I also released a full-length album, began performing live, and gave some talks and guest lectures.

Meeting the monthly deadline required pushing myself a little, but I’m glad I set this schedule, because the self-imposed time constraints forced me to scope projects realistically and suppress perfectionism. Often all I needed to do for a blog post was to find an exciting topic and demonstrate its value with some nice sounds; there’s a time and place for thorough research and literature review, but I’m okay without it for most projects here. I also felt the variety is beneficial, since it helped me gauge my level of interest in various music tech topics and, more importantly, my audience’s. I don’t have analytics or a comment section, but I do hear from readers by email occasionally (thank you!), and this helps me understand what kind of writing is most wanted in the music tech community.

Speaking of engagement, now’s a good time as any to announce that I will be pivoting to topics suggested to me by this free online Blog Idea Generator:

Read more…

Thanks to all who checked out my album Haywire Frontier. Yesterday, I gave a remote talk for the NOTAM SuperCollider meetup on the project. The talk wasn’t recorded, but I decided to rework it into prose. This is partially for the benefit of people that missed the event, but mostly because I’m too lazy to research and write a new post wholly from scratch this month.

It’s not necessary to listen to the album to understand this post, but of course I would appreciate it.

// NB: Full aggregated code from example, plus SynthDefs, are at the end of the post.
Routine({
    loop {
        s.bind { Synth(\kick) };
        rrand(0.03, 0.6).wait;
    };
}).play;

I wasn’t able to get this post fully complete in time for my self-imposed monthly deadline. I have decided to put it up in an incomplete state and clean it up in early September. I hope it is informative even in its current condition, which gets increasingly sketchy towards the end. Open during construction.

Among DSP types, those unfamiliar with wavelets often view them as a mysterious dark art, vaguely rumored to be “superior” to FFT in some way but for reasons not well understood. Computer musicians with a penchant for unusual and bizarre DSP (for instance, people who read niche blogs devoted to the topic) tend to get particularly excited about wavelets purely for their novelty. Is the phase vocoder too passé for you? Are you on some kind of Baudelairean hedonic treadmill where even the most eldritch Composers Desktop Project commands bore you?

Well, here it is: my introduction to wavelets, specifically written for those with a background in audio signal processing. I’ve been writing this post on and off for most of 2023, and while I am in no way a wavelet expert, I finally feel ready to explain them. I’ve found that a lot of wavelet resources are far too detailed, containing information mainly useful to people wishing to invent new wavelets rather than people who just want to implement and use them. After you peel back those layers, wavelets are surprisingly not so scary! Maybe not easy, but I do think it’s possible to explain wavelets in an accessible and pragmatic way. The goal here is not to turn you into a wavelet guru, but to impart basic working knowledge (with some theory to act as a springboard to more comprehensive resources).

Before we go further, I have to emphasize an important fact: while wavelets have found many practical uses in image processing and especially biomedical signal processing, wavelets are not that common in audio. I’m not aware of any widely adopted and publicly documented audio compression codec that makes use of wavelets. For both audio analysis alone and analysis-resynthesis, the short-time Fourier transform and the phase vocoder are the gold standard. The tradeoffs between time and frequency resolution are generally addressable with multiresolution variants of the STFT.

There is no one “wavelet transform” but a huge family of methods. New ones are developed all the time. To limit the scope of this post, I will introduce the two “classical” wavelet transforms: the Continuous Wavelet Transform (CWT) and Multiresolution Analysis (MRA). I’ll also go over popular choices of individual wavelets and summarize their properties. There are other wavelet transforms, some of more musically fertile than CWT or MRA, but you can’t skip the fundamentals before moving on to those. My hope is that demystifying wavelet basics will empower more DSP-savvy artists to learn about these curious creatures.

Read more…