Constructive Synthesis

Ken Wessen

ken.wessen@gmail.com

Preface

Synthesis is the electronic generation of audio signals for use in music or as sound effects, or quite possibly a combination of both. Subtractive Synthesis is one important technique of electronic sound production, that was first used in the 1960s, and became increasingly available (and affordable) as analogue synthesisers were produced during the 1970s and beyond.

My aim, through the presentations in these pages, is to lead you to a detailed and well-rounded understanding of the theory of subtractive synthesis, make you comfortable with the application of this theory across a wide variety of synthesisers, and also provide some exposure to the process of building such a synthesiser using modern digital hardware.

Although just one of many approaches to electronic sound creation, the analogue subtractive approach boasts a primary attraction of constructing sounds right down at the level of the waveform. It is not particularly trying to reproduce the sounds of other instruments, certainly not precisely, but supports making new sounds — sounds that are dynamic and recognisably electronic in nature. These may still be closely related to sounds produced by other instruments, or may be entirely novel.

To the uninitiated, the controls of an analogue, or as is found more frequently these days, a virtual analogue synthesiser, can appear as a daunting nest of knobs, sliders and wires, giving rise to the impression that the process is a very complicated one. Such an impression is often reinforced when initial efforts to produce sounds by random knob fiddling tend to fail utterly. However, it is not really such an arcane practice, but is based on combining a few well defined sub-systems (we will call them modules) in a way that is consistent across nearly all examples from the myriad of synthesisers produced since the 1970s.

To do this requires an understanding of the physics of sound, in particular the nature of the sound waves produced by oscillators, since these provide the raw sonic material. Then there is the action of filters that selectively suppress or enhance features of the sound produced by the oscillators, envelopes that shape the amplitude of the sound and other signals, and low frequency oscillators that produce modulation - i.e. time varying features in the output sound.

Although my subject matter is primarily subtractive synthesis, I have chosen the term constructive for my title, and I mean constructive in few different senses:

• Construction of sound by creating a desired combination of harmonics;
• Construction of sound by using the different modules of a synthesiser;
• Construction of sound with digital hardware;
• Construction of sound in software.

I will not attempt to cover basic coding or electronics, since there are many alternative sources that concentrate on such things. The coding I use for the interactive web applications is the javascript webaudio library, and for the electronics I use the Teensy and it’s associated Audio Shield, along with the Teensyduino extension of the standard Arduino software environment. If you are interested in building low level circuits for true analogue synthesis, I recommend the book Make: Analog Synthesizers by Ray Wilson.

The presentation will proceed, more or less, module by module following the diagram above, with more than 30 webapps designed to illustrate the material in a visual and interactive way. The reason for the adjective subtractive will become apparent as we consider each of these building blocks of subtractive synthesis in turn. We will look at the theory behind them, the role they play in synthesis, and how they interact with other modules.

10 illustrative projects using the Teensy are also presented, showing all wiring and code, for readers interested in making their own hardware.

An easy to build and use, Teensy-based polyphonic synthesis kit, Mentor, is also available and is described in detail in chapter 9. If you wish to build the kit, it can also be used to illustrate each the important points raised in the earlier chapters.

1 Sound Waves and Oscillators

At the core of any subtractive synthesiser you will find one or more oscillators. These are the modules responsible for generating the original sound sources. As the name implies, subtractive synthesis is about taking things away, so what we get from the oscillator at the start is the major determining factor in what sounds we can make.

But what does it mean to “take away” from a sound?

To better understand sound and be able to answer this question, we need to look at the physics of sound waves in more detail.

1.1 Properties of Sound Waves

The two most important physical parameters that describe a sound wave are its frequency, i.e. how fast the wave is repeating itself, or oscillating, and its amplitude, i.e the size of the associated displacement.

The frequency of a sound wave corresponds to the pitch, with faster oscillations producing higher notes, and the oscillation itself is a process whereby particles in the air are variously compressed and rarefied in the direction the wave is travelling. The amplitude is a measure of how large a displacement of air is associated with each cycle, with larger displacements producing louder sounds.

Frequency is measured in hertz, abbreviated Hz, where 1Hz corresponds to exactly one cycle per second.

The human ear can hear sounds ranging from 20Hz up to 20kHz, and the frequency range on a standard 88 key piano is from 32.7Hz to 4186Hz. However, we will soon see how a musical sound at some particular frequency generally contains many additional higher frequencies that combine to produce the particular type of sound.

1.2 Wave Types

The next most significant feature of a sound wave is its shape, since this is what determines its timbre — the character of its sound. Music would be very boring if all sounds of a given pitch sounded the same, and when we hear the difference in the sounds produced by various instruments, it is a result of the different shapes of the sound waves produced.

For traditional instruments this is a result of physical aspects of the instrument and the manner in which it is played - e.g. blowing air through a wooden bore or brass tube, vibrating a reed, plucking or striking a string.

Synthesising sound by purely electronic means is of course quite different.

Subtractive synthesis begins with a circuit that generates a basic waveform as a starting point. The 4 most common waveforms generated by analogue synthesisers are:

Use the app below to listen to each of these, and the effect of varying frequency and amplitude.

1.2.1 Changing the tone

Musically, a sine wave sounds somewhat flute-like (though a real flute has many additional dynamics and thus sounds much more interesting). Let’s start with sine waves since they produce the cleanest tone, and see what happens when they are added together.

The following figure shows 3 sine waves with frequencies increasing in integer multiples (\(f, 2f, 3f\)) and amplitudes decreasing by the same factor (\(1, \frac{1}{2}, \frac{1}{3}\)).

Added together, these waves result in something that looks a little like a sawtooth.

Explore this process further using the following app that allows you to add up to 25 sine waves of increasing frequency, and listen to the result. The plot on the left shows the waveshape of the sum, i.e. the amplitude over time, and the plot on the right shows the spectrum, plotted as decibels vs frequency.

1.3 Harmonics

What we saw in the previous section illustrates the fact that, because of its periodic nature, any particular waveform can be understood as a weighted sum of sine waves of different frequencies. The lowest frequency sine wave is called the fundamental, and determines the frequency of the note, and the higher frequency terms are called the overtones. Generally they will be integer multiples of the fundamental (the multiple is known as the wave number), and in this case they are known as harmonics, but for bell-like and percussive sounds, the higher frequencies may be non-integer multiples of the fundamental.

The presence or absence of particular harmonics, and their relative amplitude, is what creates a waves particular character, or timbre. So, for example, when seeking to achieving a string or brass sound, a sawtooth wave is the most appropriate starting point. Single reed instruments (such as a clarinet) are more like square waves, and a flute is closest to a pure sine wave.

The set of harmonics in a wave, and their amplitudes, is known as its spectrum. Let’s revisit the wavetypes app but this time we will add in a spectrum view so we can also look at the harmonic content of the waves.

In the app, the waves are digitally generated and thus not perfect examples of their type. Additionally, the harmonic amplitudes are plotted in decibels, so the scale is logarithmic and the degree of drop off in amplitude of the higher harmonics is not as obvious to the naked eye. The figures below summarise the results for ideal waves, and show the harmonic amplitudes on a linear scale.

The key insight in subtractive synthesis is if you start with one of these waves, you get all their harmonics for free. You can then reduce or emphasize the relative amplitudes of selected harmonics through a process known as filtering to create new sounds. Because a sawtooth wave has all harmonics, it is often the richest starting point for synthesis.

1.4 Pulse Waves

We can generalise a square wave by changing the relative amount of time at the upper and lower amplitudes. This is known as a pulse wave, and is described by its pulse width.

Use the following app to explore how the sound deviates from a square wave tone as the pulse width is increased. What do you think will happen as it reaches the maximum?

1.5 Playing Notes and MIDI

While a synthesiser can certainly be used to generate sound and sound effects without any need of a keyboard, it is far more common to treat them as a kind of keyboard instrument and have them get their primary frequency input from the notes played.

There are two main ways to communicate played note information to the oscillator in a synthesiser. The first is control voltage and gate, where changes in an input voltage indicate changes of frequency, with the usual standard being a 1V change in voltage signifying a one octave change in pitch. The gate signal indicates when a key is pressed and released. This approach is limited in that support for multiple keys requires additional signal voltages, so CV synths are generally monophonic — they support only a single note at a time (though the control voltage may be passed to 2 or even 3 independent oscillators). Nevertheless, the CV standard allows for some very interesting interoperability between synthesiser modules since all control, be it oscillators, filters, modulators, envelopes etc, use the same interface, and so can be interconnected for an enormously wide variety of complicated interactions and effects.

The second approach uses a serial protocol called MIDI (Musical Instrument Digital Interface), that can communicate note values, keyboard velocity, and many other facets of control of an electronic musical instrument. MIDI has become the standard means of connecting keyboard controllers, synthesisers, electric pianos and computers, and it is the technology we will employ for the hardware projects presented in subsequent chapters.

The MIDI protocol is based on messages. Here are some of the most important:

note on

Note on messages are sent when a keyboard note is played and contain the note number from 21 (A0) to 108 (C8), a velocity value from 0 to 127 indicating how fast the key was pressed, and a channel number from 1 to 16.

Converting a note number \(n\) to a frequency in hertz is according to the formula \[f_n = 440 \times 2^{\frac{1}{12}(n - 69)}.\]

note off

A note off message is sent when a key is lifted, and carries note, velocity and channel data in the same way as the note on message.

control change

Controls on a device may be assigned a specific control number (CC), and then when their position is changed, an associated MIDI message carrying the CC number and a value from 0-127 is generated to indicate the new value for the control. Alternatively, if this message is sent to a device, it will adjust the value of the corresponding control to the sent value.

clock

A MIDI clock message is used to synchronise timing across connected devices, at a rate of 24 clocks per quarter note.

Synthesisers often have quite small keyboards - many serious performance machines only provide 3 octaves. One way to bypass this constraint is to use a Controller Keyboard that connects via MIDI. However, many of these are similarly small.

A standard control on a synth is a range control, that will be marked with lengths in feet in powers of 2 — at least 4’, 8’ and 16’, and often 2’ and 32’ as well. These lengths are a reference to the lengths of pipes in pipe organs, and their effect is octave shifting. Doubling the length of a pipe (or a string) will drop its frequency by one octave, and conversely halving will raise the pitch in the same ratio. It is customary to control octave shifts on an electronic instrument by selecting a virtual pipe ‘length’. The 8’ range is unshifted.

A range control allows quick adjustment of independent oscillators without compromising the tuning, and, when the keyboard is small, the larger shifts allow playing notes over a much greater range. For example, a 3 octave keyboard can reach 2 lower octaves using a 32’ range, and 2 higher octaves with a 2’ range, making 7 octaves in all — just less than a standard piano keyboard.

Each of the earlier apps in this chapter supported quasi-continuous frequency change via a slider control. In contrast, the following app uses a keyboard to determine the required oscillator frequency.

1.6 Summary

• Oscillators produce periodic waveforms with particular shapes, the standard ones used in synthesis being sine, triangle, sawtooth, square and pulse.
• The pitch is determined by the fundamental frequency of the wave, but, apart from a pure sine wave, each of these waveforms contain higher frequency harmonics that are responsible for their specific tonal qualities.
• The MIDI protocol supports communication of note and control information between keyboard controllers, synthesisers and computers.

Glossary

Frequency

The rate at which a periodic wave cycles, corresponding to the pitch of the sound.

Amplitude

The maximum displacement level of a periodic wave, corresponding to the loudness of the sound.

Waveshape

The shape of the displacement for a periodic wave over one cycle.

Harmonic

A pure sine of frequency \(nf\) for integer \(n\) that is a component of a periodic wave with frequency \(f\).

Spectrum

The set of all harmonics in a sound.

Timbre

The tonal character of a particular waveshape.

Pulse width

The percentage of time a (generalised) square wave spends at its maximum level.

MIDI

Musical Instrument Digital Interface — a serial protocol for communicating timing, notes and controls between devices.

Monophonic

Describes a device that can only produce sounds associated with a single note at a time.

2 Combining Oscillators

2.1 Mixing Wave Types

Many synthesisers support mixing the basic waveshapes to provide a broader sonic palette. For example, adding in a sawtooth wave to an otherwise smoother waveform adds some “buzz”.

The following app supports continuously varying the waveform from triangle to sawtooth and then to square. See how the features merge in the waveshape, and watch the change in the harmonics as the combined wave transforms.

2.2 Phase

Another important parameter describing a sound wave is the phase, measured in degrees and describing how far a wave is through its cycle (just as with a circle, 360 degrees is a full cycle). Unlike frequency and amplitude, this is not something we hear in isolation, but it can have a big effect when waves add.

For example, consider the sum of two sine waves of frequencies \(f\) and \(2f\), in the ratio \(1\) and \(\frac{1}{2}\).

This is the first step of the sum we look at earlier when making a sawtooth wave.

In the previous example, both waves are in phase, i.e. they start their cycles at exactly the same time. Let’s shift the phase of the higher frequency wave by 1/4 of a cycle (so they are 90 degrees out of phase).

You can see that the resulting wave shape is quite different, but surprisingly, the perceived sound is unaltered. What we hear is determined by the harmonic components alone, and in this case the two harmonic components are still present in the same amounts.

The picture is a little different when considering the effects of phase difference when adding identical waves. The following figure shows the result of summing two identical sine waves with increasing phase separation (perhaps due to a small delay). The results show that starting from an initial in-phase doubling, as the phase shift increases the amplitude of the resulting wave will decrease in amplitude. This is due to increasing cancellation, culminating in silence when the phase shift is exactly half a cycle — for a wave of 440Hz this corresponds to a delay of a little more than 1ms.

In the case of more complex waves, this summing of sines still happens at each of the harmonics, but the same time delay leads to a different phase separation at each harmonic, and so the effect is a much richer interaction where some harmonics are enhanced and others are attenuated.

In each of these examples the phase difference was fixed. In the next section we will see how a changing phase relationship between two waves leads to very interesting dynamic features in the sound produced.

2.3 Detuning

When two waves that are very close in frequency, say \(f-\delta\) and \(f+\delta\) for small \(\delta\), are added, the interesting phenomenon of beating occurs. The pitch will be at \(f\), but the envelope will vary at the much slower rate of \(2\delta\).

As the waves go in and out of phase, their combination changes from a sum to a cancellation. A synth with multiple voices will usually provide a unison option where all voices play exactly together to make a rich, thick sound, and also an option to detune, where small frequency shifts can be used to introduce slow phase effects in the overall amplitude.

When the detune amount is small, say a few cents, this tends to soften the sound in much the same way that the sound of the strings section in an orchestra is much smoother than that of a string quartet.

2.4 Noise

Another audio source that is regularly found in a synthesiser is a noise generator. Unlike the clean, repeating cycles of the standard oscillator waveshapes, a noise signal is random and produces sounds like hissing, crackling, or rushing wind or water.

These sounds can be used to create sound effects, percussive sounds, or just mixed in to some degree with the other more tuneful sounds to make a more interesting synthesised sound.

A noise signal is not a repeating cycle like the other waves we have looked at, but it is still an audio signal with specific spectral properties. Its spectrum is not made up of regularly spaced harmonics, but is rather a mix of all frequencies. Nevertheless, the overall shape of spectrum still determines important aspects of the noise’s audible character.

Traditionally noise is described in colour terms. White noise is a harsh sounding noise with an even spectral distribution. Reducing the higher frequencies softens the sound, producing pink noise, that, because of the nature of human perception, actually sounds more even. Making the higher frequency part of the spectrum drop off even faster produces brown noise, that, with its stronger bass component, can sound like the ocean.

Tilting the noise spectrum in the other direction, so the higher frequencies are enhanced, produces blue noise, a particularly harsh sounding noise reminiscent of hissing or rushing water.

The following app lets you switch between these 4 kinds of noise, blending it in with a standard oscillator, and both hearing and seeing the result on the sound, waveshape and spectrum.

2.5 Summary

• Waves can be added, combining their harmonics, to make more complicated waveshapes.

• Adding waves that are close in frequency produces an envelope that oscillates at the difference of the two frequencies.

• When sine waves of the same frequency are added together, they may either amplify or cancel to various degrees depending on the phase difference.

• When more complicated out of phase waves are added, this process happens differentially with any matching harmonics.

• Noise contains harmonics at all frequencies, and the ‘sound’ of the noise still depends on the overall shape of the spectrum.

Glossary

Phase

The position within a cycle of a periodic wave.

Detune

Make two oscillators play slightly out of tune with each other in order to generate dynamic effects associated with the time-varying envelope of their sum.

Noise Generator

A source of any particular kind of noise signal that can be used alongside the oscillators.

White Noise

Flat spectrum noise — a harsh sounding noise.

Pink Noise

A filtered noise signal with reduced higher frequencies — a smoother and more even sounding noise for human perception.

Brown Noise

Similar to pink noise, but even softer since the higher frequency components drop off even faster.

Blue Noise

A particularly harsh sounding noise where the lower frequency components are attenuated and the high frequency ones unaltered.

3 Audio and the Teensy

As its name implies, the Teensy is very small, but it is nonetheless a remarkably powerful microcontroller development system: USB powered, easily programmed using the Teensyduino software, and breadboard compatible. Add in the Teensy Audio Shield, and you have support for 16 bit, 44.1 kHz sample rate audio with stereo output and input. The Teensyduino software includes the necessary Audio Library.

The projects that follow do not require a specific version of Teensy, but for speed and memory reasons I strongly recommend using the Teensy 4 or later, though the Teensy 3 variants have 12-bit digital to analogue converters (DACs) for true analogue output which may sometimes be an important consideration. The only additional components we will require are potentiometers and switches for simple external controls, and optionally a MIDI shield or breakout board for external connectivity. Experienced users of Teensy (or Arduino) may wish to add in support for an LCD for showing the values currently set via the potentiometers.

The polyphonic synthesis kit, Mentor, presented in chapter 9, is in some respects simply the union of the many Teensy projects presented over the course of the next 6 chapters. It is easy to build, and can be used instead of the breadboard circuits if desired.

3.1 The Teensy Pinout

My examples use the Teensy 4.1 and its matching Audio Shield, but I will only use pins that are common across all the recent variants.

The Audio Shield can be stacked directly on top of the Teensy as follows.

3.2 Setup on a breadboard

We will want some controls to interact with the Teensy. Let’s start with two potentiometers, connected to analogue pins A0 and A1 on the Teensy. These allow us to read input values from 0 to 1023, depending on their rotation.

The position of the Audio Shield is indicated by the transparent blue rectangle, since it would otherwise obscure the Teensy pins.

This arrangement is all that is required for the next 4 projects. The Teensy audio code will change, as will the function of the two potentiometers, but the connections will remain the same.

3.3 Project 1: One oscillator - control pitch and amplitude

The Audio Library Design Tool functions as a graphical design tool for Teensy audio projects, with code import and export, and as an API reference for the Audio library. This first project is basically the simplest possible controllable “synthesiser”, and this simplicity is clear in the layout shown by the graphical design tool — a single oscillator, an output (i2s) and an audio controller (an SGTL5000 — the low power audio codec provided on the Audio Shield).

Use the Teensyduino software to make a new project called SimpleOscillator, and enter the following code (which includes the header and declaration code exported by the Audio Design Tool).

   #include <math.h>
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       oscillator;      //xy=252,335
   AudioOutputI2S           i2s;           //xy=451,335
   AudioConnection          patchCord1(oscillator, 0, i2s, 0);
   AudioConnection          patchCord2(oscillator, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;     //xy=462,266
   // GUItool: end automatically generated code
   
   #define PITCH_POT A0
   #define AMP_POT A1
   
   int pitch = 220;
   int amplitude = 0.5;
   int waveform = WAVEFORM_SINE;
   
   // pitch scales logarithmically
   float inputToPitch(int input)
   {
       int n = map(input, 0, 1023, 21, 108);
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the pitch and amplitude controls
       pinMode(PITCH_POT, INPUT);
       pinMode(AMP_POT, INPUT);
   
       // configure and start the oscillator object
       oscillator.amplitude(amplitude);
       oscillator.frequency(pitch);
       oscillator.begin(waveform);
   }
   
   void loop() 
   {
       // read the pitch pot position
       int newpitch = analogRead(PITCH_POT);
       // has it changed?
       if (newpitch != pitch)
       {
           // update if it has
           pitch = newpitch;
           oscillator.frequency(inputToPitch(newpitch));
       }
   
       // read the amp pot position
       int newamp = analogRead(AMP_POT);
       // has it changed?
       if (newamp != amplitude)
       {
           // update if it has
           amplitude = newamp;
           oscillator.amplitude(1.0f * amplitude / 1024);
       }
   }

The code contains a little bit of Teensy and Audio Library initialisation code, and the output of the single oscillator is determined by setting the frequency, amplitude and waveform (recall from the previous chapter that these are the three fundamental aspects of a sound wave).

In this project, the frequency and amplitude of the output sound changes in response to adjustments in their corresponding potentiometer.

3.4 Project 2: One oscillator - control pitch and wavetype

For this second project we will change the function of the 2nd potentiometer from amplitude to selecting the waveform. Since this is not a straightforwardly numerical parameter to the oscillator, a little extra code is required to define the waveforms we will support, and to manage the transitions. The Teensy supports a variety of predefined waveforms, and also allows you to define your own. We will employ standard sine, triangle, sawtooth and square waves, as well as two fixed predetermined pulse waves — with widths and 25% and 10%.

Use the Teensyduino software to make a new project called WaveType, and enter the following code.

   #include <math.h>
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       oscillator;      //xy=252,335
   AudioOutputI2S           i2s;           //xy=451,335
   AudioConnection          patchCord1(oscillator, 0, i2s, 0);
   AudioConnection          patchCord2(oscillator, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;     //xy=462,266
   // GUItool: end automatically generated code
   
   #define PITCH_POT A0
   #define WAVETYPE_POT A1
   
   typedef enum 
   {
       SINE, TRIANGLE, SAWTOOTH, SQUARE, PULSE_25, PULSE_10
   } wavetype;
   
   // map our wavetype to teensy waveform (note there are two pulse waves)
   int wavetypes[] = {
       WAVEFORM_SINE, WAVEFORM_TRIANGLE, WAVEFORM_SAWTOOTH, 
       WAVEFORM_SQUARE, WAVEFORM_PULSE, WAVEFORM_PULSE
   };
   
   #define N_TYPES 6
   
   int pitch = 220;
   wavetype waveform = SAWTOOTH;
   
   // pitch scales logarithmically
   float inputToPitch(int input)
   {
       int n = map(input, 0, 1023, 21, 108);
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   wavetype getWaveTypeFromValue(int inputValue)
   {
       int n = map(inputValue, 0, 1023, 0, N_TYPES - 1);
       return (wavetype)n;
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the pitch and amplitude controls
       pinMode(PITCH_POT, INPUT);
       pinMode(WAVETYPE_POT, INPUT);
   
       // configure and start the oscillator object
       oscillator.amplitude(0.5);
       oscillator.frequency(pitch);
       waveform = getWaveTypeFromValue(analogRead(WAVETYPE_POT));
       oscillator.begin(waveform);
   }
   
   void loop() 
   {
       // read the pitch pot position
       int newpitch = analogRead(PITCH_POT);
       // has it changed?
       if (newpitch != pitch)
       {
           // update if it has
           pitch = newpitch;
           oscillator.frequency(inputToPitch(newpitch));
       }
   
       // read the wavetype pot position
       int v = analogRead(WAVETYPE_POT);
       wavetype newwave = getWaveTypeFromValue(v);
       // has it changed?
       if (newwave != waveform)
       {
           // update if it has
           waveform = newwave;
           oscillator.begin(wavetypes[waveform]);
           if (waveform == PULSE_25)
               oscillator.pulseWidth(0.25);
           else if (waveform == PULSE_10)
               oscillator.pulseWidth(0.1);
           else    // just to be safe
               oscillator.pulseWidth(0.5);
       }
   }

3.5 Project 3: Three oscillators - control pitch and wavetype mix

In section 2.1 of the previous chapter, we looked at smoothly transitioning between triangle, sawtooth and square wavetypes. To do this with Teensy requires three oscillators, one for each wavetype, and we manage the output by adjusting their amplitudes appropriately and mixing their outputs, using 3 channels of the standard 4-to-1 mixer object, to get the final result.

The maximum sum of the output amplitudes must not exceed 1.0, or else clipping, i.e. distortion in the generated sound due to exceeding the amplitude limit, will occur.

The first potentiometer still controls the frequency, but this time the second potentiometer is used to allocate across the wavetypes — fully anti-clockwise to the middle position mixes triangle and sawtooth, and middle to fully clockwise mixes sawtooth and square.

Use the Teensyduino software to make a new project called WaveTypeMix, and enter the following code.

   #include <math.h>
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       triangle;      //xy=252,289
   AudioSynthWaveform       square; //xy=253,388
   AudioSynthWaveform       sawtooth; //xy=254,337
   AudioMixer4              mixer;         //xy=448,338
   AudioOutputI2S           i2s;           //xy=612,338
   AudioConnection          patchCord1(triangle, 0, mixer, 0);
   AudioConnection          patchCord3(sawtooth, 0, mixer, 1);
   AudioConnection          patchCord2(square, 0, mixer, 2);
   AudioConnection          patchCord4(mixer, 0, i2s, 0);
   AudioConnection          patchCord5(mixer, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;     //xy=599,273
   // GUItool: end automatically generated code
   
   #define PITCH_POT A0
   #define MIX_POT A1
   
   #define TRIANGLE 0
   #define SAWTOOTH 1
   #define SQUARE 2
   
   int pitch = 220;
   float mixf = 1.0;
   
   // pitch scales logarithmically
   float inputToPitch(int input)
   {
       int n = map(input, 0, 1023, 21, 108);
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   void applyMixValue(float mixf)
   {
       // allocate across the wavetypes
       // 0 to 1 mixes triangle and sawtooth, 1 to 2 mixes sawtooth and square
       if (mixf < 1)
       {
           mixer.gain(TRIANGLE, 1 - mixf);
           mixer.gain(SAWTOOTH, mixf);
           mixer.gain(SQUARE, 0);
       }
       else
       {
           mixer.gain(TRIANGLE, 0);
           mixer.gain(SAWTOOTH, 2 - mixf);
           mixer.gain(SQUARE, mixf - 1);
       }
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the pitch and amplitude controls
       pinMode(PITCH_POT, INPUT);
       pinMode(MIX_POT, INPUT);
   
       // configure and start the three oscillators
       triangle.frequency(pitch);
       triangle.amplitude(0.5);
       triangle.begin(WAVEFORM_TRIANGLE);
   
       sawtooth.frequency(pitch);
       sawtooth.amplitude(0.5);
       sawtooth.begin(WAVEFORM_SAWTOOTH);
   
       square.frequency(pitch);
       square.amplitude(0.5);
       square.begin(WAVEFORM_SQUARE);
   
       applyMixValue(mixf);
   }
   
   void loop() 
   {
       // read the pitch pot position
       int newpitch = analogRead(PITCH_POT);
       // has it changed?
       if (newpitch != pitch)
       {
           // update if it has
           pitch = newpitch;
           triangle.frequency(inputToPitch(newpitch));
           sawtooth.frequency(inputToPitch(newpitch));
           square.frequency(inputToPitch(newpitch));
       }
   
       // read the mix pot position
       float newmix = 2 * analogRead(MIX_POT) / 1023.0;
       // has it changed?
       if (newmix != mixf)
       {
           // update if it has
           mixf = newmix;
           applyMixValue(mixf);
       }
   }

3.6 Project 4: Two oscillators - control pitch and detune

In section 2.3 of the previous chapter, we saw and heard the interesting dynamic results of detuning — mixing two waves of close frequency. With two oscillators and the Teensy, we can generate the same effect.

The first potentiometer controls the frequency, and the second one controls the detuning, from -100 to 100 cents (i.e. a semitone in each direction).

Use the Teensyduino software to make a new project called Detune, and enter the following code.

   #include <math.h>
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       oscillator1;      //xy=252,289
   AudioSynthWaveform       oscillator2; //xy=254,337
   AudioMixer4              mixer;         //xy=448,338
   AudioOutputI2S           i2s;           //xy=612,338
   AudioConnection          patchCord1(oscillator1, 0, mixer, 0);
   AudioConnection          patchCord2(oscillator2, 0, mixer, 1);
   AudioConnection          patchCord3(mixer, 0, i2s, 0);
   AudioConnection          patchCord4(mixer, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;     //xy=599,273
   // GUItool: end automatically generated code
   
   #define PITCH_POT A0
   #define DETUNE_POT A1
   
   int pitch = 220;
   int detune = 0;
   int detunepotvalue = 0;
   int waveform = WAVEFORM_SAWTOOTH;
   
   // pitch scales logarithmically
   float inputToPitch(int input, int detune)
   {
       int n = map(input, 0, 1023, 21, 108);
       float f = 440 * pow(2, (n - 69) / 12.0);
       // calculate the detuned frequency using the fact that there are 1200 cents in an octave
       return f * pow(2, detune / 1200.0);
   }
   
   /*
    * Because pot reading can be noisy, we will work with an average of the most recent 'n' readings.
    * This is a useful utility function for working with a Teensy.
    */
   int smoothpot(int newvalue, int curvalue, int n)
   {
     return floor(((n - 1) * curvalue + newvalue) / n);
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the pitch and amplitude controls
       pinMode(PITCH_POT, INPUT);
       pinMode(DETUNE_POT, INPUT);
   
       // configure and start the oscillators
       oscillator1.frequency(pitch);
       oscillator1.amplitude(0.5);
       oscillator1.begin(waveform);
       oscillator2.frequency(pitch);
       oscillator2.amplitude(0.5);
       oscillator2.begin(waveform);
   }
   
   void loop() 
   {
       // read the pitch pot position
       int newpitch = analogRead(PITCH_POT);
       // has it changed?
       if (newpitch != pitch)
       {
           // update if it has
           pitch = newpitch;
           oscillator1.frequency(inputToPitch(newpitch, 0));
           oscillator2.frequency(inputToPitch(newpitch, detune));
       }
   
       // read the detune pot position
       detunepotvalue = smoothpot(analogRead(DETUNE_POT), detunepotvalue, 16);
       int newdetune = detunepotvalue / 5 - 102; // range is -102 to 102 cents
       // make 0 a bit sticky since it is an important value and there is 
       // a lot of noise when reading a pot like this
       if (newdetune > 0)
           newdetune = max(0, newdetune - 2);
       else if (newdetune < 0)
           newdetune = min(0, newdetune + 2);
       // has it changed?
       if (newdetune != detune)
       {
           // update if it has
           detune = newdetune;
           oscillator1.frequency(inputToPitch(newpitch, 0));
           oscillator2.frequency(inputToPitch(newpitch, detune));
       }
   }

3.7 Project 5: MIDI input

To use MIDI we need to wire in a MIDI connector. A standard MIDI connector is a 5 pin DIN, and a complete MIDI in/out circuit uses just 2 connectors, 4 resistors, a diode and an opto-coupler to ensure the connected systems are electrically isolated. The easiest option is to use one of the readily available MIDI shields, or a MIDI I/O breakout board, and just connect the 4 wires as shown below.

The MIDI library included with the Teensyduino software makes the communication with a MIDI controller easy, and we will use it for this project.

First connect up the input from a MIDI board/shield to the RX pin (we only receive MIDI in this example, but it is well worth extending it to try transmitting as well). Since the input note determines frequency we no longer need a frequency potentiometer, so one pot will be for wavetype, and the other will be a range selector, supporting the octave ranges 32’, 16’, 8’, 4’ and 2’.

Use the Teensyduino software to make a new project called MIDI, and enter the following code. The audio library object connections are the same as used in the WaveType circuit.

   #include <math.h>
   #include "MIDI.h"
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       oscillator;      //xy=252,335
   AudioOutputI2S           i2s;           //xy=451,335
   AudioConnection          patchCord1(oscillator, 0, i2s, 0);
   AudioConnection          patchCord2(oscillator, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;     //xy=462,266
   // GUItool: end automatically generated code
   
   #define RANGE_POT A0
   #define WAVETYPE_POT A1
   
   typedef enum 
   {
       SINE, TRIANGLE, SAWTOOTH, SQUARE, PULSE_25, PULSE_10
   } wavetype;
   
   int wavetypes[] = {
       WAVEFORM_SINE, WAVEFORM_TRIANGLE, WAVEFORM_SAWTOOTH, 
       WAVEFORM_SQUARE, WAVEFORM_PULSE, WAVEFORM_PULSE
   };
   
   #define N_TYPES 6
   
   float volume = 0.5;
   int pitch = 220;
   int octaveshift = 0;
   int curnote = 0;
   wavetype waveform = (wavetype)100;  // illegal value guarantees update on first loop
   
   // Define the callbacks for handling MIDI note on and note off messages
   void handleNoteOn(byte channel, byte pitch, byte velocity);
   void handleNoteOff(byte channel, byte pitch, byte velocity);
   
   // Create a hardware MIDI instance on Serial1 (pins 1 and 2)
   MIDI_CREATE_INSTANCE(HardwareSerial, Serial1, MIDI);
   
   // pitch scales logarithmically
   float inputToPitch(int n)
   {
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   wavetype getWaveTypeFromValue(int inputValue)
   {
       int n = map(inputValue, 0, 1023, 0, N_TYPES - 1);
       return (wavetype)n;
   }
   
   void playNote(bool updateWaveType)
   {
       if (updateWaveType)
       {
           oscillator.begin(wavetypes[waveform]);
           if (waveform == PULSE_25)
               oscillator.pulseWidth(0.25);
           else if (waveform == PULSE_10)
               oscillator.pulseWidth(0.1);
           else    // just to be safe
               oscillator.pulseWidth(0.5);
       }
       if (curnote)
       {
           oscillator.amplitude(volume);
           oscillator.frequency(inputToPitch(curnote + octaveshift * 12));
       }
   }
   
   void handleNoteOn(byte channel, byte pitch, byte velocity)
   {
       // set the current active note
       curnote = pitch;
       playNote(false);
   }
   
   void handleNoteOff(byte channel, byte pitch, byte velocity)
   {
       if (pitch == curnote)
       {
           // turn off the oscillator because the active note has been lifted
           oscillator.amplitude(0);
           curnote = 0;
       }
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the range and wavetype controls
       pinMode(RANGE_POT, INPUT);
       pinMode(WAVETYPE_POT, INPUT);
   
       // Setup the MIDI listening on channel 1
       MIDI.begin(1);
       MIDI.setHandleNoteOn(handleNoteOn);
       MIDI.setHandleNoteOff(handleNoteOff);
   }
   
   void loop() 
   {
       // read the range pot position
       int newrange = map(analogRead(RANGE_POT), 0, 1023, -2, 2);
       // has it changed?
       if (newrange != octaveshift)
       {
           // update if it has
           octaveshift = newrange;
           playNote(false);
       }
   
       // read the wavetype pot position
       int v = analogRead(WAVETYPE_POT);
       wavetype newwave = getWaveTypeFromValue(v);
       // has it changed?
       if (newwave != waveform)
       {
           // update if it has
           waveform = newwave;
           playNote(true);
       }
   
       // check for MIDI messages (the callbacks will handle them if there are any)
       MIDI.read();
   }

3.8 Summary

• Using a Teensy, the Teensy Audio Shield, and its associated Audio library, we can create oscillators able to generate sounds of desired frequency and waveshape, and combine them in various ways.
• Potentiometers, buttons and LEDs can be integrated to provide controls and feedback.

4 Filtering

The simplest way to think about a filter is as a device that alters the spectrum of a sound by attenuating particular frequencies. Any harmonics of the sound wave that fall in the affected frequency range will have their amplitude reduced to some degree, while other frequencies will pass through unaltered.

There is an enormous variation of filter types and implementations used in synthesis. We will look in detail at the two most common general classes of filter: low pass and high pass, characterised by a cut-off frequency that quantifies the starting point of significant attenuation, but there are also band pass filters, notch filters, comb filters, and even an “all pass” filter that doesn’t attenuate any frequencies!

Filters may also be resonant, in which case frequency components around their cut-off frequency will be amplified.

However, filters make other changes to the character of the sound since the process of filtering also affects the phase of the sound wave. As we saw in section 2.2, adding identical but phase shifted waves affects the harmonics differently. Since this process happens internally to the filter, the timbre of the output can be quite drastically altered, and this is why the filter of any given synthesiser is the component most responsible for its characteristic “sound” (and also why an all pass filter is still useful).

A useful analogy is with a stringed instrument: the string is the oscillator — responsible for producing the raw sound, but the body of the instrument, be it a violin, guitar, or piano say, is akin to the filter in the synth, selectively attenuating and enhancing harmonics. This is why the highest quality instruments have the most attention paid to this aspect of their design and construction.

The central importance of filtering to the technique of sound production we are studying is, of course, captured in its name — subtractive synthesis.

4.1 Low Pass Filtering

A low pass filter reduces the harshness or brightness of a sound by attenuating the higher frequency harmonic components. With regard to the waveshape, this has the effect of rounding off any sharp edges. Thinking in terms of the harmonics, the resulting wave becomes like the sum of fewer and fewer sine waves, and we get the reverse effect to that which we saw with the Adding Sines app in section 1.2.1. Eventually only the fundamental frequency will remain and all waveshapes are reduced to a pure sine wave under enough filtering. Going below the fundamental removes all sound.

Often a filter is described as being open or closed, depending on how many harmonics are being let through or attenuated.

The frequency response curve describes the rate of attenuation vs frequency for a filter, and the following figure shows a curve for a 12db/octave low pass filter. The frequency axis has octave steps labelled (doubling frequency each step) and the gain axis is in steps of 12dB (12dB/octave and 24dB/octave are the most commonly used attenuation rates).

The following app lets you see and hear all these aspects of low pass filtering on different wave shapes.

4.2 Resonance

Far and away the most commonly employed filter in subtractive synthesis is a resonant low pass filter, that combines a low pass filter as above with a controllable degree of resonance — emphasis of harmonics around the cut-off frequency.

Hear and see the effect of resonance with the following app.

Cut-off and resonance combine to create much of the signature sound of subtractive synthesis.

For some filters, when resonance is at a maximum, the filter itself will oscillate. This is called self-oscillation and produces a sine wave. Combined with key tracking (which we will see in section 4.5 below) you can actually “play” the filter as a sine wave oscillator. Without key tracking, you can still alter the frequency using the cut-off control, and replicate the sounds of a theremin.

4.3 High Pass Filtering

As clearly indicated by its name, high pass filtering has the opposite effect to low pass filtering. For a high pass filter, the cut-off frequency determines the point at which lower frequency harmonics begin to be significantly attenuated.

This can be useful when you need to reduce the muddiness or rumbling within a sound, and can lead to some very interesting waveforms as the following app shows.

4.4 Project 6: Teensy Filtering

For this project we need more control than the 2 potentiometers we used in the previous chapter, so we will add an additional 2 potentiometers, as well as a button and associated LED to manage state. The full set of connections needs to be as shown below:

The Teensy Audio Library provides a 12dB/octave filter object that supports resonance, and has three outputs — one each for a low pass, band pass and high pass filtered output. As in Project 2, one potentiometer will control the pitch, and another will control the wave type. The additional 2 potentiometers will control the cut-off frequency and the resonance, and the button will cause the output signal to switch between the low pass and the high pass filtered signal. A mixer object is placed after the filter and used to allow only the desired signal through.

   #include <math.h>
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       oscillator;     //xy=83.5,203
   AudioFilterStateVariable filter;        //xy=237,210
   AudioMixer4              mixer;         //xy=408,215
   AudioOutputI2S           i2s;            //xy=586,212
   AudioConnection          patchCord1(oscillator, 0, filter, 0);
   AudioConnection          patchCord2(filter, 0, mixer, 0);
   AudioConnection          patchCord3(filter, 2, mixer, 1);
   AudioConnection          patchCord4(mixer, 0, i2s, 0);
   AudioConnection          patchCord5(mixer, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;       //xy=538,138
   // GUItool: end automatically generated code
   
   #define PITCH_POT A0
   #define WAVETYPE_POT A1
   #define CUTOFF_POT A2
   #define RES_POT A3
   
   #define LED 3
   #define HIGH_LOW_SW 4
   
   typedef enum 
   {
       SINE, TRIANGLE, SAWTOOTH, SQUARE, PULSE_25, PULSE_10
   } wavetype;
   // map our wavetype to teensy waveform (note there are two pulse waves)
   int wavetypes[] = {
       WAVEFORM_SINE, WAVEFORM_TRIANGLE, WAVEFORM_SAWTOOTH, 
       WAVEFORM_SQUARE, WAVEFORM_PULSE, WAVEFORM_PULSE
   };
   
   #define N_TYPES 6
   
   int pitch = 220;
   wavetype waveform = SAWTOOTH;
   int cutoff = 2000;
   int resonance = 0;
   bool isLowPass = true;
   
   // pitch scales logarithmically
   float inputToPitch(int input)
   {
       int n = map(input, 0, 1023, 21, 108);
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   // input is from 0 to 127
   void setCutoff(int u)
   {
       // Use an exponential curve from 50Hz to about 12kHz
       float co = 50 * exp(5.481 * u / 127.0);
       filter.frequency(co);
       filter.octaveControl(log2f(12000.0 / (float)co));
   }
   
   // input is from 0 to 127
   void setResonance(int u)
   {
       // Convert to an appropriate range for the Teensy Audio Library filter object
       filter.resonance(u * 4.3 / 127.0 + 0.7);
   }
   
   wavetype getWaveTypeFromValue(int inputValue)
   {
       int n = map(inputValue, 0, 1023, 0, N_TYPES - 1);
       return (wavetype)n;
   }
   
   void updateFilter()
   {
       digitalWrite(LED, isLowPass);
       // turn on the correct channel from the mixer
       mixer.gain(0, isLowPass ? 1 : 0);
       mixer.gain(1, isLowPass ? 0 : 1);
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the pitch and amplitude controls
       pinMode(PITCH_POT, INPUT);
       pinMode(WAVETYPE_POT, INPUT);
       // and the two pins for the filter pots
       pinMode(CUTOFF_POT, INPUT);
       pinMode(RES_POT, INPUT);
       
       // and the switch/LED pins
       pinMode(LED, OUTPUT);
       pinMode(HIGH_LOW_SW, INPUT_PULLUP);
       updateFilter();
       
       setCutoff(cutoff / 127.0);
       setResonance(resonance / 127.0);
   
       // configure and start the oscillator object
       oscillator.amplitude(1);
       oscillator.frequency(pitch);
       waveform = getWaveTypeFromValue(analogRead(WAVETYPE_POT));
       oscillator.begin(waveform);
   }
   
   void loop() 
   {
       static long lastpress = 0;
       if ((digitalRead(HIGH_LOW_SW) == 0) && (millis() - lastpress > 200))
       {
           // switch is on (line pulled low)
           lastpress = millis();
           isLowPass = !isLowPass;
           updateFilter();
       }
       
       // read the pitch pot position
       int newpitch = analogRead(PITCH_POT);
       // has it changed?
       if (newpitch != pitch)
       {
           // update if it has
           pitch = newpitch;
           oscillator.frequency(inputToPitch(newpitch));
       }
   
       // read the wavetype pot position
       int v = analogRead(WAVETYPE_POT);
       wavetype newwave = getWaveTypeFromValue(v);
       // has it changed?
       if (newwave != waveform)
       {
           // update if it has
           waveform = newwave;
           oscillator.begin(wavetypes[waveform]);
           if (waveform == PULSE_25)
               oscillator.pulseWidth(0.25);
           else if (waveform == PULSE_10)
               oscillator.pulseWidth(0.1);
           else    // just to be safe
               oscillator.pulseWidth(0.5);
       }
       
       // read the cutoff pot position
       int newcutoff = analogRead(CUTOFF_POT) >> 3;
       // has it changed?
       if (newcutoff != cutoff)
       {
           // update if it has
           cutoff = newcutoff;
           setCutoff(cutoff);
       }
       
       // read the resonance pot position
       int newres = analogRead(RES_POT) >> 3;
       // has it changed?
       if (newres != resonance)
       {
           // update if it has
           resonance = newres;
           setResonance(resonance);
       }
   }

4.5 Key Tracking

With a fixed filter cut-off frequency, the effect on a note’s harmonics will depend on the frequency of that note. For example, a cut-off of 800Hz will only significantly attenuate the 8th and higher harmonics of the note G2 at 98Hz, but will be strongly attenuating all harmonics above the second for A4 at 440Hz. Going even further, the fundamental for C6 at 1047Hz is well above the cut-off frequency.

This shows how the filter effects the timbre (or, equivalently, the waveshape) of a note differently depending on its frequency.

Key tracking overcomes this by adjusting the cut-off frequency with pitch, making the cut-off frequency change and keeping the timbre (i.e. the spectral profile of the harmonics) more consistent across the keyboard. This may or may not be desired, and so it is generally adjustable.

4.6 Summary

• Through filtering and resonance, the character of the synthesised sound can be altered, giving far more variation than is possible through wave shape selection alone.
• Filters are described by their frequency response curves, i.e. the curves that determine their attenuation or enhancement of input frequencies.
• The cut-off frequency is the frequency at which the attenuation amount is 3dB.
• Resonance acts to enhance harmonics at (and around) the cut-off frequency.
• To understand the action of the filter, you need to consider its impact on the harmonics of the input sound.

Glossary

Cut-off Frequency

The frequency at which a low pass or high pass filter is applying a 3dB attenuation to the input signal.

Resonance

A measure of the amount of amplification of harmonics at and around the cut-off frequency.

Frequency response curve

A curve that describes a filter’s response for all input frequencies.

Low pass filter

A filter that attenuates frequencies above a particular cut-off frequency,

High pass filter

A filter that attenuates frequencies below a particular cut-off frequency,

Band pass filter

A filter that attenuates frequencies within a particular frequency range.

Key tracking

Adjusting the cut-off frequency inline with the frequency of the played note.

Self-oscillation

When a maximum resonance filter produces a sine-wave due to feedback.

5 Envelopes

An envelope is a single-cycle time-varying amplitude. Starting and ending at zero, it is usually used to control the change in loudness of a played note over time, but is also regularly applied to the cut-off frequency of a filter, and sometimes even to the pitch of an oscillator.

The loudness envelope is one of the most recognisable aspects of the sound produced by a physical instrument. For a bowed instrument, such as violin or cello, the musician can control the loudness at all times through their use of the bow. Conversely, on a piano or acoustic guitar, when a string is played there is a very rapid rise to a volume determined by the force with which the note was struck or the the string plucked, followed by a slow decay at a rate that is outside of the player’s control. An organ note has a very simple loudness contour — immediately reaching it’s fixed volume, and remaining there as long as the key is held, before immediately dropping to silence when the key is released. A brass sound can rise to its maximum fast or slow, and then tends to drop down a little to a holding level, before stopping immediately when blowing ceases.

In this chapter we will look at the way that envelopes are usually modelled in subtractive synthesisers, specifically with the four stage ADSR model introduced by Bob Moog in the 1960s.

In our generic synthesiser diagram the envelope module is connected to the keyboard, since the variation is triggered when a key is pressed.

This also brings up an interesting point regarding polyphony — i.e. supporting the playing of multiple notes simultaneously. In a truly polyphonic synthesiser, each note is completely independent and follows its own envelope (and has its own filter as well). In an analogue synthesiser this means substantial additional electronics, and thus expense, for each voice. One way that polyphony is achieved more simply is to support independent oscillators for each voice, but have them share the envelope and filter. A synthesiser that works this way is called paraphonic, and any group of held notes will rise and fall following a single envelope, even if they were not played at the same time.

5.1 Attack and Decay

As an initial model, we will think about the volume profile for our synthesised notes as being comprised of two stages. The initial stage is the attack, and this determines how quickly the volume rises to the maximum. Upon reaching maximum, the second stage, the decay, begins, and this stage determines how quickly the note subsequently falls to zero.

Each stage is characterised by a length of time, generally up to a maximum of a few seconds, and the associated volume change during each transition may be either linear or exponential.

The figure below illustrates an AD envelope with a linear attack and an exponential decay.

Use the following app to see and hear the results of changing the AD envelope parameters. Press the desired waveform button to simulate playing a note.

5.2 Sustain and Release

We can further refine the loudness profile by adding a third and fourth stage. The third stage, sustain, determines the fixed loudness of the sound while the key is being held down (and after any transients due to attack and decay have completed). Note that when sustain is included in the envelope, the decay stage doesn’t reduce to zero, but rather to the sustain level. The final stage, release, determines how quickly a note fades to silence once the key is lifted.

The signal that a synthesiser relies on to manage the envelope process is called a gate. It usually is linked to a keyboard and remains on during the entire duration of a key press. The attack stage of an ADSR envelope starts when the gate signal starts, and the release stage starts when the gate signal ends.

The attack, decay and release parameters each determine the length of time the envelope spends in the corresponding stage, whereas sustain specifies an amplitude level, since the time spent in the sustain stage is determined by the duration of the keypress.

The following app includes an adjustable ADSR envelope. Use it to see and hear the results of changing the envelope’s shape. Press the desired waveform button to simulate playing a note, and hit the Stop button to release the note.

5.3 Project 7: Teensy ADSR Envelope

To experiment with an ADSR envelope on the Teensy we will need to connect 4 potentiometers, one for each envelope control, as well as MIDI input so we can use an external controller to send in notes for the pitch and the triggering of the envelope. We will also make use of the button to enable and disable the envelope shaping of the output sound.

Fortunately, the Teensy Audio Library includes an ADSR envelope object (actually it implements a six-stage DAHDSR envelope, where the additional stages are an initial delay time and a hold time between the attack and decay) that makes coding the desired behaviour much easier.

The analogue inputs on the Teensy are 10 bit, meaning they return values in the range 0 to \(2^{10} - 1 = 1023\). By shifting out the 3 least significant bits to make the reading more stable to noise we get a useful default scale for the control values of 0 to \(2^7 - 1 = 127\) (this also corresponds to the range used in MIDI messages).

For our attack, decay and release values, we will scale this value based on a desired maximum time length of 10 seconds using

#define ENV_MILLIS 10000

and calculating the time in milliseconds from the value as

pot_val / 127.0 * ENV_MILLIS

as required by the library object.

The connections for the library objects are as shown in the following diagram. We will use a mixer to choose between the output with or without the shaping provided by the envelope.

   #include <math.h>
   #include "MIDI.h"
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       oscillator;     //xy=83.5,203
   AudioEffectEnvelope      envelope;      //xy=240,167
   AudioMixer4              mixer;         //xy=386,207
   AudioOutputI2S           i2s;            //xy=555,206
   AudioConnection          patchCord1(oscillator, envelope);
   AudioConnection          patchCord2(oscillator, 0, mixer, 1);
   AudioConnection          patchCord3(envelope, 0, mixer, 0);
   AudioConnection          patchCord4(mixer, 0, i2s, 0);
   AudioConnection          patchCord5(mixer, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;       //xy=511,138
   // GUItool: end automatically generated code
   
   #define A_POT A0
   #define D_POT A1
   #define S_POT A2
   #define R_POT A3
   
   #define LED 3
   #define ENABLE 4
   
   #define ENV_MILLIS 10000 // max transition time is 10s
   
   float volume = 0.5;
   int pitch = 220;
   int curnote = 0;
   
   int a_val = 30;
   int d_val = 60;
   int s_val = 100;
   int r_val = 60;
   
   bool enable = true;
   
   // Define the callbacks for handling MIDI note on and note off messages
   void handleNoteOn(byte channel, byte pitch, byte velocity);
   void handleNoteOff(byte channel, byte pitch, byte velocity);
   
   // Create a hardware MIDI instance on Serial1 (pins 1 and 2)
   MIDI_CREATE_INSTANCE(HardwareSerial, Serial1, MIDI);
   
   // pitch scales logarithmically
   float inputToPitch(int n)
   {
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   void handleNoteOn(byte channel, byte pitch, byte velocity)
   {
       // set the current active note
       curnote = pitch;
       oscillator.amplitude(volume);
       oscillator.frequency(inputToPitch(curnote));
       envelope.noteOn();
   }
   
   void handleNoteOff(byte channel, byte pitch, byte velocity)
   {
       if (pitch == curnote)
       {
           // turn off the oscillator because the active note has been lifted
           if (!enable)
               oscillator.amplitude(0);
           curnote = 0;
           envelope.noteOff();
       }
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup four pins for listening to the ADSR controls
       pinMode(A_POT, INPUT);
       pinMode(D_POT, INPUT);
       pinMode(S_POT, INPUT);
       pinMode(R_POT, INPUT);
   
       // and the switch/LED pins
       pinMode(LED, OUTPUT);
       pinMode(ENABLE, INPUT_PULLUP);
       updateEnable();
       
       oscillator.begin(WAVEFORM_SQUARE);
   
       // Setup the MIDI listening on channel 1
       MIDI.begin(1);
       MIDI.setHandleNoteOn(handleNoteOn);
       MIDI.setHandleNoteOff(handleNoteOff);
   }
   
   bool checkPot(int *curValue, int pin)
   {
       int newValue = analogRead(pin) >> 3;  // reduce to between 0 to 127
       // has it changed?
       if (newValue != *curValue)
       {
           *curValue = newValue;
           return true;
       }
       return false;
   }
   
   void updateEnable()
   {
       digitalWrite(LED, enable);
       // turn on the correct channel from the mixer
       mixer.gain(0, enable ? 1 : 0);
       mixer.gain(1, enable ? 0 : 1);
       if (!enable && curnote == 0)
           oscillator.amplitude(0);
   }
   
   void loop() 
   {
       static long lastpress = 0;
       if ((digitalRead(ENABLE) == 0) && (millis() - lastpress > 200))  // switch is on (line pulled low)
       {
           lastpress = millis();
           enable = !enable;
           updateEnable();
       }
   
       if (checkPot(&a_val, A_POT))
           envelope.attack(a_val / 127.0 * ENV_MILLIS);  // attack time in ms
   
       if (checkPot(&d_val, D_POT))
           envelope.decay(d_val / 127.0 * ENV_MILLIS);  // decay time in ms
   
       if (checkPot(&s_val, S_POT))
           envelope.sustain(s_val / 127.0);  // sustain is a level, not a time
   
       if (checkPot(&r_val, R_POT))
           envelope.release(r_val / 127.0 * ENV_MILLIS);  // release time in ms
   
       // check for MIDI messages (the callbacks will handle them if there are any)
       MIDI.read();
   }

5.4 Linking the Envelope and the Filter

As mentioned earlier, envelopes are regularly applied to the cut-off frequency of a filter, thus causing the harmonic content of the wave to vary over time. This can be a very pleasing effect and, although it is also an important feature of traditional musical instruments, it is one of the signature sounds of synthesiser music.

The envelope shape causes the cut-off frequency to increase from its set value over the period of attack, then drop to the sustain amount during the decay stage. Just like with the loudness envelope, the value remains static during the sustain stage before dropping back to its set value over the release.

An amount setting will be used to control the extent to which the envelope effects the cut-off, and the behaviour in the release stage depends on the note remaining audible due to also having some release in its loudness envelope. For many synthesisers the same envelope is used for both loudness and filter cut-off.

Use the following app to see how the envelope affects the shape of the filter and, correspondingly, the harmonic content of the output sound.

5.5 Inverting the Filter Envelope

It is not uncommon for a synth to support inverting the action of the envelope on the filter. When inverted, the attack stage corresponds to the filter closing, before the decay stage where it partially opens to a degree controlled by the sustain level, and finally returning to its original level over the release period.

Use the following app to see and hear the difference between a filter that opens while following the envelope compared to an inverted one that closes.

5.6 Summary

• An envelope is a single-cycle time-varying amplitude.
• It is most commonly used to shape the loudness of the sound.
• An envelope can also be used to modulate other parameters, such as filter cut-off or pitch.
• ADSR is the most common envelope model, describing 4 stages — attack, decay, sustain and release.
• When using synthesis to approximate the sound of specific instruments, the envelope can be used to reproduce the required loudness contour and harmonic content of the sound.

Glossary

Envelope

A time varying amplitude countour.

Trigger

A signal that starts an envelope’s process.

Gate

A signal that indicates the duration of a key press. The attack stage of an ADSR envelope starts when the gate signal starts, and the release stage starts when the gate signal ends.

ADSR

The standard 4-stage synthesiser envelope, where A stands for attack, D for decay, S for sustain, and R for release.

Attack

The initial stage of an ADSR or related envelope where the amplitude rises from 0 to its full volume in response to a key down or other trigger.

Decay

The second stage of an ADSR or related envelope where the amplitude drops from its maximum to its sustain level.

Sustain

The third stage of an ADSR or related envelope where the amplitude remains steady at a specified level.

Release

The final stage of an ADSR envelope where the amplitude drops from the sustain level to 0, after key release.

Polyphonic

A device that can produce independent sounds in response to multiple notes at a time.

The maximum number of notes that can be played simultaneously is called the polyphony of the device.

Paraphonic

A reduced approach to polyphony where simultaneously held notes share a filter and are constrained to follow a single envelope.

6 Modulation

6.1 The LFO

In the previous chapter we saw how an envelope can control the loudness profile when a note is hit and released, or how it can be used to control the cut-off frequency of a filter. In general terms, a modulator is any component that introduces dynamic changes to the basic sound, and an envelope is one such component.

An ADSR envelope is an example of a one time curve — it starts when triggered, progresses through the attack and decay stages, becomes static while the note is held, and finally decaying after the note is lifted. The other common type of modulator is a Low Frequency Oscillator, or LFO, that, unlike the oscillators we discussed earlier, is not used to make an audio signal, but rather to change (i.e. modulate) other parameters. The range of frequencies supported by an LFO begins well below audible frequencies, maybe as low as 0.1Hz (or equivalently 10s to complete a full cycle), and rarely goes higher than 100Hz.

To use an LFO for modulation you must specify not only the rate of oscillation, but also an amount that determines the maximum range of the variation. An LFO may be bipolar, i.e. the modulation acts to both increase and decrease the input value, or unipolar, where the modulation acts in one direction only.

Two standard such aspects in traditional music are vibrato — the slow modulation of pitch, and tremolo — the slow modulation of amplitude. An LFO is also routinely used to modulate a filter’s cut-off frequency, often called a sweep when the rate is slow and the amount is high, or a wah-wah or wobble when the rate is faster.

6.2 Pitch Modulation

If the LFO destination is the pitch, then the pitch modulates around the base value at the LFO rate, producing the vibrato effect.

Use the following app to apply low frequency pitch modulation to the sound from a standard oscillator. Adjust the rate and the amount to hear the difference in the resulting signal.

6.3 Filter Modulation

If the LFO destination is the filter, then it is the cut-off frequency that is modulated around the base value at the LFO rate.

Use the following app to apply low frequency filter cut-off modulation to the sound from a standard oscillator. Adjust the rate and the amount to hear the difference in the resulting signal. The resonance is not modulated, but by adjusting the resonance level you can listen to the interesting ways that it interacts with the varying cut-off frequency.

6.4 Project 8: LFO Modulation

For the next Teensy project, we will combine both of these kinds of modulation, using 4 potentiometers to control the oscillator wavetype, pitch, LFO rate & amount, and a button to choose between pitch modulation and filter modulation.

In the Teensy code we will multiply the LFO oscillator output with a constant scaling factor set by the amount potentiometer to produce an adjustable level of modulation. (In fact we will use two, one for pitch and one for the filter.)

   #include <math.h>
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveformDc     pitchAmt;            //xy=90,133
   AudioSynthWaveform       LFO;      //xy=90,194
   AudioSynthWaveformDc     filterAmt; //xy=90,255
   AudioEffectMultiply      multiply;      //xy=247,160
   AudioEffectMultiply      multiply1; //xy=250,233
   AudioSynthWaveformModulated oscillator;   //xy=396,166
   AudioFilterStateVariable filter;        //xy=541,224
   AudioOutputI2S           i2s;            //xy=707,208
   AudioConnection          patchCord1(pitchAmt, 0, multiply, 0);
   AudioConnection          patchCord2(LFO, 0, multiply, 1);
   AudioConnection          patchCord3(LFO, 0, multiply1, 0);
   AudioConnection          patchCord4(filterAmt, 0, multiply1, 1);
   AudioConnection          patchCord5(multiply, 0, oscillator, 0);
   AudioConnection          patchCord6(multiply1, 0, filter, 1);
   AudioConnection          patchCord7(oscillator, 0, filter, 0);
   AudioConnection          patchCord8(filter, 0, i2s, 0);
   AudioConnection          patchCord9(filter, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;       //xy=618,140
   // GUItool: end automatically generated code
   
   #define PITCH_POT A0
   #define CUTOFF_POT A1
   #define RATE_POT A2
   #define AMOUNT_POT A3
   
   #define LED 3
   #define PITCH_FLT_SW 4
   
   int pitch = 220;
   int cutoff = 2000;
   int rate = 0;
   int amount = 0;
   bool isPitch = true;
   
   // pitch scales logarithmically
   float inputToPitch(int input)
   {
       int n = map(input, 0, 1023, 21, 108);
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   // input is from 0 to 127
   void setCutoff(int u)
   {
       // Use an exponential curve from 50Hz to about 12kHz
       float co = 50 * exp(5.481 * u / 127.0);
       filter.frequency(co);
       filter.octaveControl(log2f(12000.0 / (float)co));
   }
   
   void setLFORate(int u)
   {
   // convert log scale 0 to 127 to a linear rate
   // f(1) = 0.5Hz, f(127) = 40Hz
       float f = exp(u / 25.0) / 4.0;
       LFO.frequency(f);
   }
   
   void updateLFO()
   {
       digitalWrite(LED, isPitch);
       pitchAmt.amplitude(isPitch ? amount / 127.0 : 0);
       filterAmt.amplitude(isPitch ? 0 : amount / 127.0);
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the pitch and amplitude controls
       pinMode(PITCH_POT, INPUT);
       pinMode(CUTOFF_POT, INPUT);
       pinMode(RATE_POT, INPUT);
       pinMode(AMOUNT_POT, INPUT);
       
       // and the switch/LED pins
       pinMode(LED, OUTPUT);
       pinMode(PITCH_FLT_SW, INPUT_PULLUP);
       updateLFO();
   
       // configure and start the oscillator object
       oscillator.amplitude(0.5);
       oscillator.frequency(pitch);
       oscillator.begin(WAVEFORM_SAWTOOTH);
       oscillator.frequencyModulation(1);
   
       setLFORate(rate);
       LFO.amplitude(1);
       LFO.begin(WAVEFORM_TRIANGLE);
   }
   
   void loop() 
   {
       static long lastpress = 0;
       // are we modulating pitch or filter?
       if ((digitalRead(PITCH_FLT_SW) == 0) && (millis() - lastpress > 200))  // switch is on (line pulled low)
       {
           lastpress = millis();
           isPitch = !isPitch;
           updateLFO();
       }
       
       // read the pitch pot position
       int newpitch = analogRead(PITCH_POT);
       // has it changed?
       if (newpitch != pitch)
       {
           // update if it has
           pitch = newpitch;
           oscillator.frequency(inputToPitch(newpitch));
       }
       
       // read the cutoff pot position
       int newcutoff = analogRead(CUTOFF_POT) >> 3;
       // has it changed?
       if (newcutoff != cutoff)
       {
           // update if it has
           cutoff = newcutoff;
           setCutoff(cutoff);
       }
       
       // read the LFO rate pot position
       int newrate = analogRead(RATE_POT) >> 3;
       // has it changed?
       if (newrate != rate)
       {
           // update if it has
           rate = newrate;
           setLFORate(rate);
       }
       
       // read the amount pot position
       int newamount = analogRead(AMOUNT_POT) >> 3;
       // has it changed?
       if (newamount != amount)
       {
           // update if it has
           amount = newamount;
           updateLFO();
       }
   }

6.5 LFO Wave Shapes

Throughout the above discussion we haven’t explicitly considered the waveshape for the LFO, but it actually matters a great deal. The audible effect of the modulation varies greatly with the shape of the LFO.

The most common LFO waveshapes are sine, triangle, square, sawtooth and ramp (or reverse sawtooth).

This is similar to the audio oscillators, but there is one very important difference. For an LFO, it is not the harmonic content of the wave that matters, but its shape. For example, ramp and sawtooth sound exactly the same, and so an audio oscillator will only have one of them. But as modulators, they are very different, since a ramp is a rapid rise followed by a slow fall, whereas a sawtooth is a slow rise followed by a rapid fall.

With the app below you can hear the different results that arise from the different LFO shapes, for both pitch and filter modulation.

6.6 Pulse Width Modulation

Recall the pulse waveshape we looked at in section 1.4, a generalised square wave where the relative amount of time at the upper and lower amplitude limits was determined by the pulse width. Pulse width is a common target for LFO modulation, since cycling the pulse width makes for a harmonically interesting, dynamic sound.

Use the following app to listen to the different sounds generated by a dynamic pulse width of various kinds.

6.7 Sample and Hold

A particularly interesting extension to LFO modulation is Sample and Hold. This works by sampling an incoming signal at regular intervals, and holding the value constant in between samples. If the sample rate is high compared to an incoming signal, this acts to reduce the resolution and outputs a quantised (or stepped) version of the input signal.

Often the input signal is white noise, creating a stepped random output as shown in the figure below.

Use the following app to see how an output signal so produced can be used to modulate pitch or cut-off in the same way as any other LFO waveshape.

6.8 Project 9: LFO Shapes

For this Teensy project we will re-use the pitch modulation approach from earlier in this chapter.

However, instead of using the second potentiometer to change the oscillator waveshape, it will instead change the shape of the modulating waveform produced by the LFO (and the oscillator waveshape will stay fixed as a sawtooth wave). The code supports 6 different LFO waveshapes — the standard 5 LFO waveshapes of sine, triangle, sawtooth, ramp and square, as well as sample and hold.

As in the previous project, the 3rd potentiometer controls the LFO rate and 4th controls the modulation amount.

   #include <math.h>
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveformDc     pitchAmt;       //xy=107,193
   AudioSynthWaveform       LFO;            //xy=107,254
   AudioEffectMultiply      multiply;       //xy=264,220
   AudioSynthWaveformModulated oscillator;     //xy=414,227
   AudioOutputI2S           i2s;            //xy=595,229
   AudioConnection          patchCord1(pitchAmt, 0, multiply, 0);
   AudioConnection          patchCord2(LFO, 0, multiply, 1);
   AudioConnection          patchCord3(multiply, 0, oscillator, 0);
   AudioConnection          patchCord4(oscillator, 0, i2s, 0);
   AudioConnection          patchCord5(oscillator, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;       //xy=534,166
   // GUItool: end automatically generated code
   
   #define PITCH_POT A0
   #define SHAPE_POT A1
   #define RATE_POT A2
   #define AMOUNT_POT A3
   
   #define LED 3
   
   // map our wavetype to teensy waveform
   int shapes[] = {
       WAVEFORM_SINE, WAVEFORM_TRIANGLE, WAVEFORM_SAWTOOTH, 
       WAVEFORM_SAWTOOTH_REVERSE, WAVEFORM_SQUARE, WAVEFORM_SAMPLE_HOLD
   };
   
   #define N_TYPES 6
   
   int pitch = 220;
   int shape = 0;
   int rate = 0;
   int amount = 0;
   
   bool ledstate = false;  // toggle LED to give feedback for shape changes
   
   // pitch scales logarithmically
   float inputToPitch(int input)
   {
       int n = map(input, 0, 1023, 21, 108);
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   int getShapeFromValue(int inputValue)
   {
       return map(inputValue, 0, 1023, 0, N_TYPES - 1);
   }
   
   void setLFORate(int u)
   {
   // convert log scale 0 to 127 to a linear rate
   // f(1) = 0.5Hz, f(127) = 40Hz
       float f = exp(u / 25.0) / 4.0;
       LFO.frequency(f);
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       AudioMemory(20);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the pitch and amplitude controls
       pinMode(PITCH_POT, INPUT);
       pinMode(SHAPE_POT, INPUT);
       pinMode(RATE_POT, INPUT);
       pinMode(AMOUNT_POT, INPUT);
       
       pinMode(LED, OUTPUT);
   
       // configure and start the oscillator object
       oscillator.amplitude(0.5);
       oscillator.frequency(pitch);
       oscillator.begin(WAVEFORM_SAWTOOTH);
       oscillator.frequencyModulation(1);
   
       setLFORate(rate);
       LFO.amplitude(1);
       shape = getShapeFromValue(analogRead(SHAPE_POT));
       LFO.begin(shape);
   }
   
   void loop() 
   {
       // read the pitch pot position
       int newpitch = analogRead(PITCH_POT);
       // has it changed?
       if (newpitch != pitch)
       {
           // update if it has
           pitch = newpitch;
           oscillator.frequency(inputToPitch(newpitch));
       }
   
       // read the wavetype pot position
       int v = analogRead(SHAPE_POT);
       int newshape = getShapeFromValue(v);
       // has it changed?
       if (newshape != shape)
       {
           // update if it has
           shape = newshape;
           LFO.begin(shapes[shape]);
           digitalWrite(LED, ledstate);
           ledstate = !ledstate;
       }
       
       // read the cutoff pot position
       int newrate = analogRead(RATE_POT) >> 3;
       // has it changed?
       if (newrate != rate)
       {
           // update if it has
           rate = newrate;
           setLFORate(rate);
       }
       
       // read the cutoff pot position
       int newamount = analogRead(AMOUNT_POT) >> 3;
       // has it changed?
       if (newamount != amount)
       {
           // update if it has
           amount = newamount;
           pitchAmt.amplitude(amount / 127.0);
       }
   }

6.9 Summary

• Modulation adds a time-varying aspect to the generated sound.
• A low frequency oscillator (LFO) is a source of cyclic modulation, with adjustable rate and degree of modulation.
• The modulation from an LFO is usually applied to pitch and filter cut-off, but can also modulate other parameters.
• The shape of the LFO makes a big difference to the audible nature of the modulation.
• LFO modulation plays an important part in both musical sound generation and sound effects.

Glossary

Modulator

A component that introduces a time-dependent change to an existing signal.

Amount

The amplitude of the low frequency modulation for a given parameter.

Bipolar

Describes an LFO that oscillates between a positive value and its negative opposite, resulting in modulation of an equal amount in each direction.

Unipolar

Describes an LFO that oscillates between a positive value and zero, with resulting modulation in one direction only.

Vibrato

Slow modulation of oscillator pitch.

Tremolo

Slow modulation of oscillator amplitude.

Sample and Hold

Create an output signal by sampling an incoming signal (often white noise) at regular intervals, and holding the value constant in between samples.

7 The Amplifier

As can be seen in our generic synthesiser module diagram, the Amplifier is the component where the outputs of all the previous components converge. The primary signal generated by the oscillator has been filtered and modulated, and now it needs its loudness applied.

In chapter 5 we learned about the amplitude envelope. This is the point at which it is applied, along with any additional loudness modulations, e.g. cyclic modulation by the LFO.

It is actually a bit of a misnomer to call this component an amplifier, since it also attenuates. In fact, attenuation is often it’s most significant task. Nevertheless, when building a synth, care must be taken to ensure the signal is clean in all earlier stages, specifically that there is no overloading leading to the harshly distorted audio resulting from clipping. The amplifier can then be used to raise the volume to the level required for effect input and audio output.

The amplifier component in a synthesiser will often support switching between an unshaped output signal, where the loudness simply follows the gate (so, like an organ, the sound is at maximum level while a key is pressed and silent at other times), or the envelope-shaped signal.

7.1 LFO Amplitude Modulation

In the same way that an LFO can be used to modulate pitch and/or filter cut-off (as we saw in the previous chapter), the LFO output can be applied to the amplifier input signal and thus cyclically modulate the loudness. (This kind of interoperability is a key feature of modular synthesisers in particular, where the user is responsible for manually constructing with physical cables all signal paths.)

Use the app below to see and hear the result of modulating the signal amplitude to different degrees, and with different LFO shapes.

7.2 Ring Modulation

Another form of amplitude modulation that is often provided in a synthesiser is ring modulation.

Consider the product of two sine waves — a carrier of frequency \(f\) and a modulator of frequency \(g\). Using the identity

\[\sin(f)\sin(g)=\frac{\cos(f-g)-\cos(f+g)}{2}\]

we see that the result is a combination of two new sine waves (\(\cos(x)\) is just \(\sin(x)\) phase shifted by 90 degrees), with frequencies given by the sum and the difference of the original two frequencies. Note especially that neither of the original frequencies is present in the output, and only in special cases will the two output frequencies be in harmonic relationship.

The effect on the waveshape is as shown in the following figure.

For other waveshapes the same thing happens: here is an example involving a square wave carrier modulated by a triangle wave.

Although these two figures appear quite similar, the situation in the second example is actually far more complex, since every single harmonic in the carrier wave is split in two by every single harmonic in the modulating wave. When neither the carrier nor the modulator is a sine wave, this process creates an extraordinarily rich array of harmonics in the output wave.

This kind of modulation is known as ring modulation and can produce marvellous raspy inharmonic, metallic, and robotic effects. Indeed, this is the process used to generate the famous Dalek voices from Doctor Who. Use the following app to explore the incredible range of waveforms that can be generated in this way.

7.3 Summary

• The amplifier is the component where the outputs of all the previous components converge.
• The amplifier can attenuate or amplify an incoming signal.
• It is responsible for shaping the loudness with the amplitude envelope each keypress.
• Cyclic loudness modulation can be applied using the LFO.
• It can also be used to uniformly raise or lower the level of the incoming signal.
• Ring modulation is a form of amplitude modulation that significantly alters the harmonic content of the incoming waves.

Glossary

Clipping

When a signal goes over the maximum supported amplitude, the wave form is truncated and harsh audio artefacts are introduced.

Modular Synthesiser

A synthesiser composed of independent modules, where the user must use physical cables to manually create all signal paths.

Ring Modulation

Multiplying two signals to produce an output that contains the sum and difference of their frequencies as harmonics.

8 Effects

In some respects it might be more reasonable to consider the Effects component as being external to the synthesiser proper, since audio effects are essentially modifications of a fully generated signal that can quite satisfactorily stand on its own. Yet, since an effects module is part of most modern synthesisers, and the sounds they produce are integral to a modern understanding of synthesis, I am including them as part of our generic synthesiser and approach to subtractive synthesis.

Many effects had their genesis in physical devices - e.g. physically interfering with tape reels, echo chambers, overdriving amplifiers etc. But, with the advent of digital technology, the breadth and power of the available effects, and the degree of control possible, has increased massively.

An effect adjusted signal, known as the wet signal, is often mixed back with the original signal, known as the dry signal, to make an adjustable balance.

8.1 Portamento

Portamento is a smooth glide of the frequency between one note and the next — a very simple effect that was a common feature of many of the earliest synthesisers. Since this was quite a novel effect for a keyboard instrument, it became strongly identified with the classic monophonic electronic sound. The rate of frequency change is generally adjustable, and portamento is regularly used to add expression to an otherwise discrete melody line.

8.2 Reverb

When a sound is made in a 3D space, the waves propagate in all directions, reflecting off the various surfaces in the space until they lose all their energy and decay to zero. A listener will hear the original (dry) sound, followed by the many reflections (the wet sound) very slightly delayed. The net effect is perceived as a single sound with reverb.

Synthesisers reproduce this effect digitally, using algorithms based on specific physical models. Hence the reverb effect will often be described in terms such as hall, room, plate, or spring (sometimes using a physical spring to produce).

The effect can be shaped using various parameters, such as:

• delay – the time for the echoes to decay,
• level – the amount of the reverb signal to mix into the output,
• diffusion – the density of the reflections,
• damping – the degree of drop off in the reflection of higher frequencies.

The following app adds a simple reverb effect to the Four Tone Organ from chapter 1.

8.3 Delay

A delay is also an echo effect, but unlike reverb it is without any sense of space. The effect can have many layers, from a signal being mixed with a delayed copy of itself, to several delayed copies at different times and levels.

A useful trick is to add some feedback, so, just like your reflection when standing between two mirrors, each delay is delaying the original signal plus the delay, and so on.

The delay with feedback approach is used in the following app. You can control the delay time and level of feedback and hear the difference in outcome. It is also interesting to hear how delay interacts with the other modulations we discussed earlier, so this app also supports LFO modulation of the filter cut-off.

8.4 Chorus, Flanger, Phaser

A very popular class of effects arises from mixing a signal with a delayed version of itself, and modulating the delay with an LFO.

The first such effect we will look at is a chorus, where the signal flow is as shown in the diagram below. (The delay time and LFO rate values shown are indicative only, but serve to help differentiate this effect from those that follow.)

You can see that the chorus effect is generated precisely as described above — mix the input signal with a delayed copy of itself, while slowly modulating the delay time (between say 20ms and 50ms) with an LFO. The result is a pleasing ambience, suggesting multiple instruments or voices separated in space, and thus heard at slightly different times.

If we reduce the delay time to 20ms or less, slow the modulation even further, and add in an adjustable feedback, we get a flanger effect. (This effect is so named since it originated by pressing on the rim of a tape reel.)

To produce a third related effect, the delay in the flanger can be replaced by a series of all-pass filters. These filters allow all frequencies through, but as we saw in chapter 4, introduce phase shifting in the signal that in turn produces a frequency-dependent enhancement and attenuation amongst the harmonics when the original signal is added back in. This effect is called a phaser, and a slow LFO modulating the all-pass filter frequency introduces its characteristic cycling phasing effect.

One way to consistently parameterise the control of these three effects is:

• Level – how much of the wet signal is mixed back in,
• Speed – the rate of the internal LFO modulation,
• Depth – the amount of internal LFO modulation.

The following app supports controlling these effects using these three parameters.

8.5 Distortion

Distortion, in general, is something to be avoided, since it constitutes a degradation of the audio signal. Low quality equipment, poor connections, or levels too high can all introduce distortion to an audio signal that will manifest itself as a raspy, noisy output sound.

Nevertheless, it is sometimes desirable to add some distortion in a controlled way to generate a particular sound. Specifically, it can be used to add harshness or warmth depending on the nature and level of the distortion.

In particular, overdrive will push a signal beyond the supported voltage limits leading to clipping where the peaks of the waves are truncated. The sharp edges this causes introduce or enhance higher frequency components and lead to distortion in the sound.

When first looking at waveshapes way back in chapter 1, we saw how moving from sine to triangle to square to sawtooth waveshapes increased the harshness of the tone as the higher frequency harmonics became more prevalent. Increasing the higher frequency harmonics even further exaggerates this effect, and doing so on a signal is thus a source of controlled distortion.

The following app lets you do exactly this; increase the level of the higher harmonics already present in the signal (except in the case of a pure sine wave) and listen to the increasing distortion that results. The effect on the wave shape and the spectrum is also shown.

8.6 Project 10: Delay Effect

The Teensy Audio Library comes with a large number of effects that can be inserted directly into the signal path. For this final Teensy project, we will use the delay effect object in a more complicated arrangement involving feedback as shown below.

As above, we will control the delay using the three parameters: level, speed and depth, so this uses 3 of our 4 potentiometers. The remaining potentiometer (the code uses potentiometer 1) can then be used to select a wavetype. The button is not used.

   #include <math.h>
   #include "MIDI.h"
   
   #include <Audio.h>
   #include <Wire.h>
   #include <SPI.h>
   #include <SD.h>
   #include <SerialFlash.h>
   
   // GUItool: begin automatically generated code
   AudioSynthWaveform       oscillator;     //xy=146.5,227
   AudioMixer4              delaymixer;         //xy=350,288
   AudioEffectDelay         delayeff;         //xy=353,416
   AudioMixer4              mixer; //xy=520,249
   AudioOutputI2S           i2s;            //xy=694,257
   AudioConnection          patchCord1(oscillator, 0, delaymixer, 0);
   AudioConnection          patchCord2(oscillator, 0, mixer, 0);
   AudioConnection          patchCord3(delaymixer, delayeff);
   AudioConnection          patchCord4(delaymixer, 0, mixer, 1);
   AudioConnection          patchCord5(delayeff, 0, delaymixer, 1);
   AudioConnection          patchCord6(mixer, 0, i2s, 0);
   AudioConnection          patchCord7(mixer, 0, i2s, 1);
   AudioControlSGTL5000     sgtl5000;       //xy=610,348
   // GUItool: end automatically generated code
   
   #define WAVETYPE_POT A0
   #define DELAY_LVL_POT A1
   #define DELAY_TIME_POT A2
   #define DELAY_FB_POT A3
   
   typedef enum 
   {
       SINE, TRIANGLE, SAWTOOTH, SQUARE, PULSE_25, PULSE_10
   } wavetype;
   
   int wavetypes[] = {
       WAVEFORM_SINE, WAVEFORM_TRIANGLE, WAVEFORM_SAWTOOTH, 
       WAVEFORM_SQUARE, WAVEFORM_PULSE, WAVEFORM_PULSE
   };
   
   #define N_TYPES 6
   
   float volume = 0.5;
   int pitch = 220;
   int curnote = 0;
   wavetype waveform = (wavetype)100;  // illegal value guarantees update on first loop
   
   int delaylevel = 0;
   int delaytime = 500;  // millis
   int delayfeedback = 63;
   
   // Define the callbacks for handling MIDI note on and note off messages
   void handleNoteOn(byte channel, byte pitch, byte velocity);
   void handleNoteOff(byte channel, byte pitch, byte velocity);
   
   // Create a hardware MIDI instance on Serial1 (pins 1 and 2)
   MIDI_CREATE_INSTANCE(HardwareSerial, Serial1, MIDI);
   
   // pitch scales logarithmically
   float inputToPitch(int n)
   {
       return 440 * pow(2, (n - 69) / 12.0);
   }
   
   wavetype getWaveTypeFromValue(int inputValue)
   {
       int n = map(inputValue, 0, 1023, 0, N_TYPES - 1);
       return (wavetype)n;
   }
   
   void playNote(bool updateWaveType)
   {
       if (updateWaveType)
       {
           oscillator.begin(wavetypes[waveform]);
           if (waveform == PULSE_25)
               oscillator.pulseWidth(0.25);
           else if (waveform == PULSE_10)
               oscillator.pulseWidth(0.1);
           else    // just to be safe
               oscillator.pulseWidth(0.5);
       }
       if (curnote)
       {
           oscillator.amplitude(volume);
           oscillator.frequency(inputToPitch(curnote));
       }
   }
   
   void handleNoteOn(byte channel, byte pitch, byte velocity)
   {
       // set the current active note
       curnote = pitch;
       playNote(false);
   }
   
   void handleNoteOff(byte channel, byte pitch, byte velocity)
   {
       if (pitch == curnote)
       {
           // turn off the oscillator because the active note has been lifted
           oscillator.amplitude(0);
           curnote = 0;
       }
   }
   
   void setup() 
   {
       // reserve some memory for the audio functions
       // need alot of extra memory for a delay effect
       AudioMemory(400);
       // enable the audio control chip on the Audio Shield
       sgtl5000.enable();
       sgtl5000.volume(0.5);
   
       // setup the two pins for listening to the range and wavetype controls
       pinMode(WAVETYPE_POT, INPUT);
       pinMode(DELAY_LVL_POT, INPUT);
       pinMode(DELAY_TIME_POT, INPUT);
       pinMode(DELAY_FB_POT, INPUT);
   
       // Setup the MIDI listening on channel 1
       MIDI.begin(1);
       MIDI.setHandleNoteOn(handleNoteOn);
       MIDI.setHandleNoteOff(handleNoteOff);
       
       delaylevel = 0.5;
       delayeff.delay(0, delaytime);
       delaymixer.gain(0, 0.75);
       delaymixer.gain(1, delayfeedback / 127.0);
       mixer.gain(0, 1 - delaylevel / 127.0);
       mixer.gain(1, delaylevel / 127.0);
   }
   
   void loop() 
   {
       // read the delay level pot position
       int newlevel = analogRead(DELAY_LVL_POT) >> 3;
       // has it changed?
       if (newlevel != delaylevel)
       {
           // update if it has
           delaylevel = newlevel;
           mixer.gain(0, 1 - delaylevel / 127.0);
           mixer.gain(1, delaylevel / 127.0);
       }
       
       // read the delay time pot position
       int newtime = analogRead(DELAY_TIME_POT) ; // up to 1s (well just over)
       // has it changed by at least 20ms?
       // (really need to avoid jitter here because it will have significant audible consequences)
       if (abs(newtime - delaytime) > 20)
       {
           // update if it has
           delaytime = newtime;
           delayeff.delay(0, delaytime);
       }
       
       // read the delay feedback pot position
       int newfeedback = analogRead(DELAY_FB_POT) >> 3;
       // has it changed?
       if (newfeedback != delayfeedback)
       {
           // update if it has
           delayfeedback = newfeedback;
           delaymixer.gain(1, delayfeedback / 127.0);
       }
   
       // read the wavetype pot position
       int v = analogRead(WAVETYPE_POT);
       wavetype newwave = getWaveTypeFromValue(v);
       // has it changed?
       if (newwave != waveform)
       {
           // update if it has
           waveform = newwave;
           playNote(true);
       }
   
       // check for MIDI messages (the callbacks will handle them if there are any)
       MIDI.read();
   }

8.7 Summary

• Effects come at the end of the sound production chain.
• Digital effect generation provides an enormous range of sonic alteration possibilities.
• Effects generally combine delay, modulation and distortion in various ways.
• Effects add significant sonic interest to an audio signal, and have thus become signature components of the characteristic sounds of synthesis.

Glossary

Audio Effect

Any creative manipulation of an input sound.

Portamento

A smooth frequency glide when changing from one note to the next.

Wet signal:

An audio signal with an effect applied.

Dry signal:

An audio signal without an effect applied.

Reverb

The effect that results from a sound echoing in space.

Delay

Record and play back a signal after a specific period of time.

Chorus

Add an effect to simulate multiple identical sources separated in space.

Flanger

Add a delayed signal with feedback into the original signal, with a slow modulation of the delay.

Phaser

Use oscillating phase cancellation to filter various parts of a signal’s spectrum.

Distortion

Add harshness or grit to a signal by amplifying the higher harmonics.

Overdrive

Drive a signal beyond its limits so that the clipping occurs, adding harshness to the signal.

9 Introducing Mentor

Now we’ve looked at all the necessary modules to make a subtractive synthesiser, and how to build and control them using a Teensy and a breadboard, it’s time to put it all together and build a complete polyphonic synthesiser.

In this chapter we will look at a single board synthesiser kit based on a Teensy 4.0 and Teensy Audio Shield, called Mentor, due to it’s focus as an instrument for learning subtractive synthesis.

All circuit and firmware source files are available here.

9.1 Specifications

Mentor is a 16 voice polyphonic synthesiser, with an architecture that matches exactly the 6 module structure we have been working through.

It features a 6x4 control matrix, with each row corresponding to one of the modules: Oscillator, Filter, Envelope, LFO, Amplifier and Effects, and a ‘Select’ encoder for selecting between them. Four additional encoders are provided for setting the parameters relevant to the selected module, and an LCD gives feedback on the parameters under adjustment.

The final encoder is used to manage presets and system settings, and can also be used for any custom extensions made to the firmware.

The full specifications are shown in the following table:

A special drone/noise-maker mode is available where the synth will produce a continuous note with (quasi)continuously adjustable frequency. This is useful for experimenting with sounds, and for making sound effects.

Three further ‘Demo’ modes are provided, one consisting of a repeating chord, another a simple looping monophonic melody, and the third a simple looping polyphonic tune. These are similarly useful for experimenting with settings without having to play a keyboard, or even have one attached.

9.2 Firmware

Each of the 16 voices has the following structure — a separate oscillator for each supported waveshape, plus a mixer to combine them (voiceMixer), a low pass filter (voiceFilter), and dual outputs, one with envelope-shaping (voiceEnv) and one that is simply on or off (voiceRaw).

The voices integrate with the top-level synth modules — the high pass filter (HPF), the LFO and its pitch and filter amount controllers, before mixing prior to being sent to the effects.

The final set of objects and connections manage both the direct output and the output that passes through the reverb, chorus, delay and overdrive effects, prior to final mixing.

9.3 The Control Encoder

The Control encoder is for presets and system parameters. Specifically, it allows you to:

• Load ROM or user presets,
• Save user presets (up to 10).
• Manage any LFO external sync (supporting multiplying factors of 4, 3, 2, 1, 1/2, 1/3, 1/4).
• Play or stop a demo (single continuous note, repeating single chord, monophonic tune, polyphonic tune).
• Panic button — sends a MIDI “all notes off” message.
• Toggle support for aftertouch.

Rotate the encoder until the LCD shows the setting to adjust, then click and rotate to see the allowed values, before selecting one with a further click. The first option is always ‘Cancel’ for making no change.

9.4 The Control Matrix

As mentioned above, each row of the control matrix corresponds to one of the modules: Oscillator, Filter, Envelope, LFO, Amplifier or Effects, and four encoders are provided for setting the parameters relevant to each module.

The ‘Select’ encoder is for selecting a particular row, with a LED indicating the current selection, and the LCD shows a 4 value summary indicating the current settings for the module — each value being the current setting for the encoder in the corresponding position.

When a parameter is being changed, the LCD also displays the changing value in detail.

9.5 Oscillator Settings

9.6 Filter Settings

9.7 Envelope Settings

9.8 LFO Settings

9.9 Amplifier Settings

9.10 Effects Settings

10 Mentor Patch Presets

Mentor includes a bank of 12 sample presets in memory. The first preset is a basic initialisation patch, and the others are designed to provide a useful starting point for a range of standard synthesiser sounds.

Below I will briefly describe the patch creation process in each case, going through the settings I chose and the reasons why.

10.1 INIT

The INIT preset is a basic unmodulated and unshaped sawtooth wave, and provides a handy way to quickly reset the parameters.

10.2 Strings

One of the staple sounds of any polyphonic synthesiser is strings, giving it the ability to reproduce the lush overlay of an orchestral string section.

Listen to a sample: Strings

10.3 Brass

Another polyphonic staple is a generic brass ensemble sound.

Listen to a sample: Brass

10.4 Solo Brass

For our first solo preset, we again make a brass sound. It is interesting to consider the difference in settings between a sound that is reminiscent of an ensemble of similar instruments, and any of them in solo.

Listen to a sample: Solo Brass

10.5 Wobble

This next preset is our first unambiguously ‘electronic’ sound, i.e. an useful and interesting sound in its own right, and not an attempt to reproduce features of any particular traditional instrument.

Listen to a sample: Wobble

10.6 Smooth Bass

Synthesisers are regularly used to add bass. This preset is for a deep, smooth bass line.

Listen to a sample: Smooth Bass

10.7 Dirty Bass

A common alternative synthesiser bass is an even deeper and harsher sound.

Listen to a sample: Dirty Bass

10.8 Alien

Modulating on pitch with sample and hold makes for a classic sci-fi alien/robot sound.

Listen to a sample: Alien

10.9 Pulse

A common feature of synthesiser music is a pulsing sound that supports either single notes or chords. Square and ramp LFO waveshapes are usually at the core of such sounds, and the rate should be adjusted to suit the music being played — either manually or through the synthesiser’s LFO sync capability.

Listen to a sample: Pulse

10.10 Pad

Another classic synth sound — a very slow, rich, modulating, pad.

Listen to a sample: Pad

10.11 Piano

It is impossible to reproduce the complex sound of a piano with subtractive synthesis, and even electric piano sounds are difficult. Nevertheless, seeing it is such an important sound, it is important to at least attempt an acceptable version.

Listen to a sample: Piano

10.12 Squash

Another purely electronic sound, with limited polyphony but quite effective for leads. The sound varies dramatically when sustained.

Listen to a sample: Squash

11 Other Kinds of Synthesis

11.1 Additive Synthesis

We have seen how all the different waveshapes are built up from sine wave components, i.e. the harmonics. So, it follows that with enough independent sine wave oscillators, each harmonic may be directly generated at any required level. This is additive synthesis, and obviates the need for a filter.

The catch is of course that it is somewhat impractical to have the large number of oscillators that would be required to do this effectively. For example, with a low frequency sawtooth wave, say C1 on the piano at 32.7Hz, even the 100th harmonic is only a little over 3kHz - well within the range of human hearing. As a result, there are only very few synthesisers based on this type of synthesis.

Nevertheless, with a few appropriately chosen harmonic ratios, the generation of many interesting and varied timbres is possible using this additive approach.

Hammond have used a standard set of 9 drawbars on their organs for more than 80 years, pitched at 16’, 5⅓’, 8’, 4’, 2⅔’, 2’, 1⅗’, 1⅓’ and 1’, corresponding to intervals of a sub-octave, 5th, primary, octave, 12th, 15th (double octave), 17th, 19th and 22nd (triple octave) as shown in the next two figures.

The following app emulates the 9 drawbars. Use it to see and hear the effect on the waveform as the harmonic levels are altered.

11.2 Wavefolding

As seen in earlier chapters, when the amplitude of a wave increases beyond the maximum allowed, it clips — that is the out of range bits are chopped off and the wave is left with flattened peaks. This is generally an unwanted distortion in the sound.

Alternatively, the portion of the wave that exceeds the maximum threshold can be folded back, resulting in a different waveform with additional harmonics. Indeed, even a small amount of folding on a pure sine wave will immediately produce a raft of higher harmonics.

This idea is known as wavefolding, and, as the figure below shows, increasing the amount of folding is an efficient way to generate more and more complex waveforms, and thus a richer harmonic structure.

Folding works best on waveforms with smooth edges, and this presents a problem for sawtooth and square waves in particular. However, applying a low pass filter prior to folding (as shown in figure b below) smooths out the sharp edges on these waves, and enables folding to be applied very effectively.

One often mentioned dichotomy in synthesis is the division between the so-called East Coast Synthesis of Bob Moog, and the West Coast Synthesis of Don Buchla (see the brief discussion in section 12.3). Subtractive synthesis is most associated with the East Coast approach — i.e. start with a waveform rich in harmonics, and subsequently filter to sculpt the sound. In contrast, wavefolding is a way to add harmonics to a signal, and is most associated with the West Coast approach.

However, the similarities far outweigh the differences, and many synthesisers combine distinct features from each approach.

Use the following app to experiment with wavefolding — both the usual standard waveforms, and their filtered versions. Use the Overdrive control to increase the amplitude of the source wave and push it into an increasingly folded state.

11.3 Wavetables

Another significant and specifically digital approach to synthesis is the use of wavetables — numerical tables defining the shape of a wave by providing the value of its amplitude sampled sufficiently frequently over a single cycle.

In this method of synthesis, the standard sine, triangle, sawtooth etc oscillators are replaced with waveforms digitally generated from the wavetables. Additionally, the generated waves can be made to evolve over time, by interpolating between wavetables, or by introducing modulation by having related wavetable definitions for the attack, decay, sustain and release stages of an envelope for instance.

The PPG Wave from 1981 was the best early example of such a synthesiser. Shown below are 4 examples from the PPG Wave wavetables.

Use the following app to experiment with wavetable synthesis by drawing any desired waveshape, listening to its sound, and seeing its spectrum. In particular, see the effect of smoothing on the resulting sound, and relate it to the change in higher frequency harmonics.

It is also interesting to start with a standard waveshape, and then corrupt it slightly (e.g. by clipping the peaks) and seeing the effect on the harmonics.

11.4 FM Synthesis

All the different approaches to synthesis are concerned with introducing and shaping harmonics to create new sounds. In subtractive synthesis, we start with a basic wave from an oscillator and apply a filter and modulation, whereas in additive and wavetable synthesis the harmonics are added via direct specification.

Frequency Modulation (FM) Synthesis is yet another approach, that in its most basic form just involves one sine wave modulating another, yet it is probably the most successful at efficiently generating complex and distinctive waveforms.

Any discussion of FM synthesis is necessarily more theoretical than is required for the other topics we have covered, since the ideas that it is based on are very mathematical in nature. However they are a natural extension of something we’ve already seen: the LFO modulation of pitch.

This is what a standard LFO-driven vibrato it looks like:

But what happens as the modulator increases into audio frequencies?

The following figure shows the result of taking a sine wave oscillating at 262Hz (middle C), and modulating it by another sine wave of frequency from 0 to 600Hz in steps of 200Hz.

A very complex, rich structure quickly becomes apparent, since, unlike with an LFO, the modulator is fast enough to interfere with individual cycles and thus change the waveshape.

Additionally, we can increase the amount of modulation. In the following figure the modulation frequency is fixed at 60Hz, and the unmodulated wave is shown along with one where modulation is applied, and subsequently with the modulation amount increased by a factor of 5 and by a factor of 15.

But what does it sound like?

Not like vibrato anymore, but a harmonically rich, new sound capable of producing metallic and bell-like tones, the tine tones of electric pianos, raspy bass sounds, and percussion.

These are the distinct strengths of FM synthesis, and are complementary to the strengths of subtractive synthesis.

Generally an FM synthesiser will not have a filter, but envelopes play an important role in shaping both the modulation and the audio output.

11.5 The Mathematics of FM

The mathematical expression \(A\sin(2\pi ft)\) describes a sine wave with constant frequency, \(f\), and constant amplitude, \(A\). Modulating the frequency is done by adding a time-varying component to the frequency term. Writing the base frequency for the main wave as \(C\) (for carrier), and modulating that frequency using another sine wave of frequency \(M\) (for modulator) and amplitude \(D\), the expression changes to \[A\sin(2\pi Ct+I\sin(2\pi Mt))\] where \(I = \frac{D}{M}\) is the index of modulation.

This is the basic equation of FM synthesis, and the ratio \(H=\frac{M}{C}\) is known as the harmonic ratio.

We have already seen and heard this when using an LFO to modulate the frequency of an oscillator — in such cases the modulation is quite slow and its vibrato nature easily heard. But, as we saw above, in FM synthesis the modulating frequency is not so constrained, and the effect on the audio output becomes much more complex.

The basic FM equation generates a cluster of harmonics centered about the carrier frequency \(C\), called side-bands, with a density that depends on the modulation frequency \(M\). When the ratio of these frequencies is an integer, an harmonic single-tone is produced. Departing from an integer relationship leads to strange and interesting harsh inharmonic sounds.

Specifically, the harmonics introduced have frequencies given by \(C\pm kM\) for integer \(k\). (This can become negative, but since \(\sin(-x) = \sin(x+\pi)\), the negative frequency terms are simply phase shifted versions of their positive frequency counterparts, and so it is the absolute value of their frequency that we consider in the spectrum). The amplitude of the harmonics drops rapidly, with the number of significant harmonics given approximately by \(I+1\).

Some informative specific cases are:

• If \(H = 1\), harmonics are simply \(kC\), i.e. purely harmonics of the carrier frequency.

• If \(H = \frac{1}{m}\), spacing between the harmonics is \(\frac{C}{m}=M\).

• If \(H = 2\), harmonics are \(C\pm2kC\), i.e. the odd harmonics only.

• If H is irrational, the spectrum will be denser since the negative, wrap-around harmonics will not land on any of the existing harmonics. Furthermore, as \(m\) increases the spacing decreases and FM synthesis becomes suitable for synthesising percussion.

The following three figures illustrate these features.

11.6 Exploring FM

The following app supports exploring the waveforms and sounds that can be produced by FM synthesis.

Some things to try include:

• With a very low harmonic ratio (1/128) and small modulator level you can hear the gentle vibrato as for the subtractive case with an LFO. Increasing the modulator level it is still like the LFO, though the range of the vibrato increases.

• With a reasonably high pitched carrier (say around 1000Hz), and the harmonic ratio increased to 1/8, the side-bands in the spectrum are clear. Check that the spacing of the side-bands changes with the harmonic ratio as expected from the mathematical analysis.

• Setting the harmonic ratio to 1 or 2, watch changes in wave form and peaks for varying modulator level. Then see the difference when the fine-tune is non-zero, and listen to the resulting inharmonic modulation.

• Choose a higher harmonic ratio and hear the introduction of bell-like tones from the high frequency harmonics.

11.7 FM Algorithms

Now we have covered the idea behind FM synthesis, it’s time to look at how it can be used in practice.

The basic building block of FM synthesis is an operator, which is just FM terminology for an oscillator (usually a sine wave) and an associated envelope. If the operator connects to the output and so produces sound, it is called a carrier. If instead it connects to another operator, it is called a modulator since its oscillations are not heard but are instead used to modulate other operators (carriers or modulators).

The overall routing of modulators and carriers is called the algorithm, and designing a sound with FM synthesis involves choosing an algorithm and subsequently adjusting the operator levels (volume for carriers, modulation amount for modulators) and frequencies. If the modulator and carrier frequencies are kept in integer relationships, harmonic results are generated. Departing from this introduces harshness or growl to the sound. This is one of the counterintuitive aspects of sound design with FM synthesis — fine adjustments to the harmonic ratio have the largest effect on the synthesised sound.

Most FM synthesisers use 4 or 6 operators, and come with a collection of predefined algorithms. The following figure shows the basic building blocks of an FM algorithm.

Figure (a) is the situation we have been exploring — a single carrier being acted upon by a single modulator.

Adding an additional operator to modulate the modulator as in figure (b) is a natural extension, though it is already getting quite tricky to visualise the interaction.

Figures (c) and (d) show how multiple modulators can be combined to act on a single carrier (the individual modulations will add), or a single modulator can simultaneously act on multiple carriers.

Figure (e) is the special case of feedback, where an operator modulates itself. We saw in section 1.2.1 that many sine waves must be added to generate an effective square wave or sawtooth. In FM synthesis feedback can be used to make a sawtooth wave from a single sine wave oscillator, since as the feedback is increased, the harmonics introduced approach a drop off rate of \(\frac{1}{n}\). FM synthesis can also produce quite a good square wave with just 2 sine waves, since a modulation frequency of double the carrier frequency includes only the odd harmonics and produces the wave shown in the following figure.

11.8 Samplers

Samplers generate their sound using digital techniques similarly to wavetable synthesisers, except, rather than using stored single cycles of periodic waves as the primary sound generation device, a sample will employ a large collection of much longer pre-recorded and digitised samples. When a note is played, these samples are retrieved and played to provide a realistic reproduction of the recorded instrument.

Depending on the available memory, a single sample can be used across a few notes (with pitch shifting), and a single note can be represented by multiple samples in order to capture different aspects of playing the sampled instrument — e.g. playback different samples depending on the key velocity.

With samplers, the focus is very much on the realistic reproduction of the sounds of other instruments. Nevertheless, samples need not be traditional instruments, but can be completely arbitrary sounds, often natural sounds like running water, or industrial sounds like machinery or breaking glass.

Once the sound is played back, it can then be modulated by LFOs, envelopes, and even filters if desired, just as in subtractive synthesis.

Initially extremely expensive, as the availability of greater processing power and memory capacity grew, these synthesisers gained support for more and longer samples, and time-dependent processing increased the accuracy of the virtual instruments.

Sometimes the term rompler is used to describe a sampler that has no audio recording capability and thus can only use pre-existing samples stored in its ROM.

Glossary

Additive Synthesis

Sound synthesis by direct specification of sine wave harmonic frequencies and levels.

Wavefolding

Wavefolding describes the process whereby a waveform is folded back on itself, possibly several times, when its amplitude crosses a threshold.

East Coast Synthesis

An approach to synthesis based on ideas by Bob Moog, where the central element is a filter that selectively removes harmonics from an harmonically rich source oscillator.

West Coast Synthesis

An approach to synthesis based on ideas by Don Buchla, focussing on generating harmonics through additive techniques and waveshaping.

Wavetable Synthesis

Sound synthesis by direct specification of waveshape amplitude data.

Frequency Modulation Synthesis

Sound synthesis by modulating sine waves by other sine waves of similar frequency.

Operator

FM term for an oscillator and associated envelope.

Carrier

An FM operator that is used to produce sound.

Modulator

An FM operator that is used to modulate other operators (either carriers or other modulators).

Harmonic Ratio

The ratio of the modulator frequency to the carrier frequency in FM synthesis.

Index of Modulation

A factor in the FM wave equation that quantifies the amount of modulation applied to the carrier wave.

Algorithm

A predefined routing of the modulator and carrier operators.

Feedback

When an FM operator is configured to modulate itself.

Exponential FM

If frequency modulation is applied using V/oct, then there will be a increase in the average frequency of the output as the modulation amount is increased.

Linear FM

If frequency modulation is applied as a fixed change of pitch in each direction, then the average frequency of the output will be independent of the modulation amount.

Sampler

Synthesis by playing stored digitised samples of sounds.

Rompler

A sampler that can only use pre-existing samples stored in its ROM.

12 A Brief History

12.1 Synthesiser Prehistory

Beginning at the turn of the 20th century, and especially in the 1920s, a stream of fantastically named electro-mechanical instruments were invented and used in performance and recording.

The first one of note was The Telharmonium, built by Thaddeus Cahill in 1897; an enormous, tonewheel controlled machine that was designed to pipe music via phone lines, and can be considered a progenitor of the Hammond Organ.

The Ondes Martenot built by Maurice Martenot in 1928, and the Trautonium by Friedrich Trautwein in 1929, each used a continuous wire for frequency selection, the first by moving a ring and the second by pressing with a finger (including sensitivity to the pressure). The Ondes Martenot could generate various different waveshapes, and the Trautonium included 2 resonant filters.

Harald Bode, who was to play an extremely significant role in the development of the synthesiser, built the Warbo Formant Organ in 1937, with 4 voices, an envelope and filters. The Hammond Novachord from 1939 had 72 note polyphony, LFO modulation, and generated sound using subtractive synthesis via 3 resonant bandpass filters.

Also deserving mention is the Electronic Sackbut built by Hugh Le Caine in 1948, that included voltage control, adjustable waveforms, and continuous modulation using a mod wheel.

Each of these instruments demonstrated techniques and devices that would evolve into standard synthesiser features. Simultaneously, there were many advances in the technology used in electronic organs, and, of course, the development of transistors and increasing miniaturisation in electronics. Not to be overlooked, the advent of programmable computers was primed to have a large impact on electronic music.

12.2 Evolution of Synthesisers

Analogue

By the mid 1960s voltage control, modular, subtractive synthesis was established, and the sounds were increasingly employed in popular music and film soundtracks. Nevertheless, the size, cost and complexity of the instruments kept mainstream commercial use minimal until the early 1970s when the first portable synthesisers began to appear.

Also around this time, instruments that were only semi-modular in nature, or even wired entirely internally, were produced, reducing the complexity of operation and thus increasing their broader appeal. As more musical artists and composers used these synthesisers, sometimes driven by the new sonic possibilities, sometimes due to the affordability of particular instruments, their new sounds came to increasingly wider mainstream attention.

Polyphony increased the appeal and utility of synthesisers, but it also increased the complexity and cost (since it essentially required a complete copy of the analogue electronics for each voice), and so even today many analogue synthesisers remain monophonic.

Digital

Advances in computing power led first to the integration of digital technology with analogue synthesisers, replacing some of the modules with digital versions, improving tuning and stability, and most importantly, storing of patches in memory, and controlling and playing via a computer.

Eventually purely digital synthesisers appeared, exploiting other methods of sound generation such as sampling, wavetables, frequency modulation, as well as software synths implemented entirely on a computer. These instruments were frequently multitimbral (i.e. they could simultaneously play multiple patches), and more easily provided additional features such as sequencers, arpeggiators and effects.

Virtual Analogue

Nevertheless, there remained significant interest in the sound generation approach used in analogue synthesisers, leading to virtual analogue instruments, i.e. synthesisers that digitally emulate analogue electronics. These instruments create their sound in the same way as analogue synthesisers, and provide the same kind of controls, yet retain all the advantages of a digital device.

This illustrates what can be considered a minor paradox inherent in the evolution of synthesisers — as technological advances improved the closeness of synthesised sounds to conventional instruments, the synthesisers became less interesting and consumer pressure developed to return to an earlier state. Another way this trend revealed itself was in attitudes about analogue vs digital sound generation. Specifically, the clarity of digital is often judged less desirable than what is called the ‘warmth’ of analogue. This has meant both a resurgence of interest in pure analogue, and a corresponding effort to make digital sound generation reproduce some of the imperfections inherent in analogue.

12.3 Timeline

1955

The first device to be actually called a synthesiser was the RCA Electronic Music Synthesizer Mark I.

This was a two-voice polyphonic system. Each voice supported patching of attack and decay, high pass and low pass filters, a low frequency modulator and resonators, in the manner that subsequently became standard with modular synthesisers. The electronics was built with vacuum tubes, and consequently the device filled a large room where it was housed at Columbia University.

It also included a sequencer since the aim was to use electronically generated sounds in recording, and control was via punched paper tape prepared offline.

1961

Harald Bode, an engineer working on electronic organs and an inventor of several pre-synthesiser instruments (as mentioned in the prehistory section above), presented a paper on using transistors for voltage control of electronic musical instruments, essentially inventing the Modular Synthesiser. The modular approach uses voltage for all aspects of signal generation and control, and the musician is responsible for manually constructing all signal paths with physical cables.

1964

The first commercial modular synthesisers arrived essentially simultaneously in 1964: The Moog Modular by Bob Moog (pictured left), and the Buchla 100 by Don Buchla (pictured right).

Buchla was very interested in experimental music, and wanted to break away from keyboards entirely since they were conceptually linked to strings, something he felt was inappropriate for electronic music. Moog, on the other hand, was not so idealist, and therefore his synthesisers were a bit more musician friendly and included a keyboard. Ultimately the Moog Modular proved to be more influential.

1967

Ikutaro Kakehashi, who went on to found Roland in 1972, released one of the first drum machines in 1967, the Rhythm Ace FR-1. The device could play preset rhythm patterns (either individually or in combination) across 4 voices — cymbal, claves, cowbell and bass drum. It also supported manual play, and enabling or disabling of the voices.

Also in this year John Chowning at Stanford University invented the FM synthesis algorithm, which he subsequently licensed to Yamaha in 1974.

1968

Wendy Carlos - Switched-On Bach: an award-winning, million-selling album of music by Bach performed on a Moog Modular. It demonstrated the novelty, expressiveness and musicality possible with the Moog, and had enormous influence on the subsequent adoption and use of synthesisers across all musical genres.

1970

Keith Emerson, of the progressive rock supergroup Emerson, Lake and Palmer, began using the Moog Modular and, starting with his solo to conclude their song ‘Lucky Man’, eventually became more associated with the Moog sound than any other musician. He even used his Moog Modular when touring — despite its size!

1971

The Minimoog (pictured left) was the first genuinely portable synthesiser. It was semi-modular, i.e. all its modules were connected internally, thus allowing the synthesizer to work without requiring external patch cables. Instead, knobs and switches were used to manage the connections.

Despite Moog not fully appreciating the significance of the change at the time, it can be convincingly argued that the Minimoog is the most influential synthesiser of all time, with the control workflow it presented being largely followed by all subsequent analogue synths.

Similarly, ARP reduced their large modular ARP 2500 to the portable ARP 2600 (pictured right), while retaining an impressive collection of features — including ring modulation, sample and hold, and reverb. Tom Oberheim, soon to be a leading synthesiser designer in his own right, made a duophonic kit for the ARP 2600, enabling the playing of two notes independently, each assigned to different oscillators.

This year also saw jazz pianist Herbie Hancock embrace synthesizers and electronic music with his Mwandishi albums.

1973

Laurie Spiegel developed the GROOVE system (named from Generated Realtime Operations On Voltage-controlled Equipment) for electronic music. This was a computer-based system, where the computer was programmed to play a collection of analogue synthesisers. Her interest was in developing and studying the logic required for algorithmic composition and improvisation, creating fascinating mathematically derived, polyrhythmic music along the way.

The Expanding Universe: is her 1980 album (though I recommend the expanded release from 2012) that captures these ideas and presents a marvellous rendition of the resulting intricate melodies and interweaving rhythms. Another of her compositions, Harmonices Mundi (Harmony of the World), was included on the ‘Sounds of Earth’ disc sent into space on Voyager — a mathematical composition based on ideas from Johannes Kepler in the 17th century.

1974

Kraftwerk - Autobahn: A hugely influential album of repetitive, pulsing electronic soundscapes using synthesisers, vocoders, drum machines and occasionally traditional instruments.

Tangerine Dream - Phaedra: An internationally successful album of experimental, atmospheric, futuristic, electronic music that mixes melody, arpeggios, effects and noise, and laid the foundations for the Berlin School approach to electronic music.

Isao Tomita - Snowflakes Are Dancing: Reworking Debussy’s tone paintings as ambient synth music using a Moog Modular, a Mellotron, and some additional effects units.

1976

Jean Michel Jarre - Oxygène: The first internationally successful album by possibly the most famous performer/composer of electronic music of all. Infused with classical, symphonic ambition, the bright and jumpy themes undergo rich development, resulting in a hypnotic mix that presented a new vision for electronic music.

1978

Several important advances are evident in the Prophet 5 by Sequential Circuits. Created by Dave Smith, it had five voice polyphony, with two oscillators per voice, and included a microprocessor and was thus programmable. Additionally, the Prophet 5 included patch memory that supported storing and restoring the value for every knob. Such a feature was a boon to performing synthesists, since they could prepare many preset sounds and switch between them instantly, and to sound designers, since they could experiment with different adjustments knowing they could always get back to the original state with ease.

1979

With dual 8-bit 6800 microprocessors, a light pen on monitor interface, and a lot of dedicated RAM and other computer circuitry, Peter Vogel and Kim Ryrie of Sydney created the Fairlight CMI — an 8-voice polyphonic sampling and sequencing synthesiser. Able to sample real world sounds (up to 1 second), manipulate wavetables, and interface to analogue synths, the Fairlight represented a major advance, and was used extensively by Stevie Wonder, Peter Gabriel, Kate Bush, and Herbie Hancock (who even demonstrated it in a dedicated segment on Sesame Street in 1983).

Two subsequent major revisions saw the capabilities expand over its 10 years of production.

1981

The original proposal for MIDI, or Musical Instrument Digital Interface, was presented at the Audio Engineering Society conference in 1981 by Dave Smith and Chet Wood. MIDI, designed as a standard communication protocol for electronic instruments, is a message-based serial protocol covering timing, the playing and releasing of notes, and setting controls. MIDI was standardised in 1983, and rapidly adopted by all manufacturers.

1982

The PPG Wave 2.2 combined a wavetable-based digital sound generation engine with analogue filters and amplifiers,and featured 8-voice polyphony, and an onboard sequencer that supported recording filtering and wavetable changes in real time. Built around the 8-bit 6809 CPU, the PPG Wave provided 32 wavetables, each containing 64 waves, providing a sizeable sonic palette for sound generation.

Vangelis - Blade Runner Soundtrack: Deep and foreboding polyphonic strings and pads, haunting layered melodies and pulsating, robotic bass, written to accompany Ridley Scott’s dark & futuristic film. Various synthesisers and traditional instruments are used, but the music is most associated with the CS-80, Yamaha’s monumental and temperamental flagship polyphonic synthesiser from the late 70s.

1983

Yamaha’s release of the fully digital DX7 synthesiser, based on the new paradigm of FM synthesis, was a game-changer. It’s extraordinary success heralded a sustained period of dominance by digital synthesisers. The DX7 was a 6 operator FM synthesiser with 32 algorithms, MIDI capable, and was famous for its electric piano, bell-like, and growly, funky bass sounds.

In a significant departure from the knobby, interactive interface of most analogue synthesisers, programming the DX7 was extremely complex, and managed via a menu and an LCD. Hence most players used the pre-defined sounds, and these are now very recognisable in much of the music of the period.

Brian Eno - Apollo: Atmospheres and Soundtracks: Employing ambient DX7 sounds and additional effects, this music was written to be the soundtrack accompanying a feature film compiled solely from footage of the Apollo moon missions.

1984

One of Roland’s most popular synthesisers was the Juno-106. A 6 voice analogue synthesiser, it made analogue polyphony affordable, and it’s minimal yet sufficient capability was presented in an extremely useful and appealing layout, making electronic sound generation and manipulation accessible to many new musicians.

It included patch memory, MIDI control and is famous for its in-built chorus effect. Although analogue, the oscillators were digitally controlled, meaning the tuning stability problems associated with many earlier analogues were absent.

1988

The digital revolution that gained momentum with the DX7 in 1983, and was accelerated by the Roland D-50 in 1987, reached its full dominance with the Korg M1 in 1988 (pictured left). A sample-based synthesiser with 4MB of 16-bit tones in ROM and multiple digital effects, it became the biggest selling synth ever.

The M1 was 16-part multitimbral, incorporated a multitrack sequencer, and can be considered the first digital workstation.

1995

Moving forward several years brings us to a period of resurgent interest in analogue synthesis. The Clavia Nord Lead managed to tap into that trend by implementing virtual analogue (or VA) synthesis — i.e. a digital emulation of analogue circuitry. This enabled them return to the analogue approach of sound generation without surrendering support for the digital advantages of MIDI, presets, effects etc.

The Nord Lead was 4 part multitimbral, and featured a return to a performance oriented interface with multiple, dedicated knobs for real-time parameter control.

1996

The motivating principle of modular synthesis is that the user should have total control of all signals and connections. Eurorack is a hardware form factor standard designed to facilitate mounting, powering and interconnecting combinations of modules from multiple companies, thus encouraging the creation of custom synthesisers of essentially unbounded size and complexity. Since its formal specification by Doepfer in 1996, it has become the standard for modular synthesis, with thousands of modules available, plus a thriving DIY ecosystem.

1997

Increasing DSP power saw the development of the Access Virus series of virtual analogue synthesisers — probably the best regarded of all the VA synths. A 16 part multitimbral synthesiser, the Virus supported wavetables, configurable filters, envelopes, multiple LFOs, multiple effects, and provided the user with a deep modulation matrix where up to 3 sources of modulation per patch were mappable to up to 6 modulation destinations — with every parameter a possible modulation destination.

2014

As computers became increasingly powerful, so did the quality of software synthesisers, and there was increasing interest in software emulations of the many classic synthesisers of the past. A relatively recent twist on this trend was provided by Roland with the Roland System 1 in 2014 (pictured right), and subsequently the System 8 in 2017, and that is the concept of a plug-out. These are fully functional, knob-per-control, virtual analogue synthesisers that also support downloading a ‘plug-out’ software synthesiser and transforming themselves into an (extremely accurate) emulation of that particular instrument.

Many of Roland’s classic monosynths and polysynths are available as plug-outs, making the instrument able to emulate a SH-101 or SH-2, say, or on the System-8, a Juno-106 or, most desired of all, a Jupiter-8.

Appendix — All the Apps

^† These apps include a self-test.

The source for these apps can be found here.

Index