Film Sound Mixing

On-Set Sound

It all begins with recording during filming. The sound team records what the actors say on set. This recording is, of course, very important, but it often doesn’t make it into the finished film unchanged.

Recording During Filming

The microphone usually hangs on a boom pole above the actors’ heads or is a lavalier microphone attached to their clothing. The boom pole provides a more natural sound, while the lavalier microphone offers greater protection against background noise. Both options are often used simultaneously to provide choices later. Not only speech is recorded, but everything audible on set.

Recording is usually done on multiple tracks simultaneously, allowing selection between microphones during post-production. All tracks carry a timestamp that later synchronizes them with the picture.

Limitations of Original Sound

However, this is precisely where the problem lies. A filming location is rarely quiet and silent. Scenery, a street, wind, or the crew itself can end up in the recording. Some scenes are also shot in locations where recording without background noise is simply not possible.

In addition to technical reasons, there are linguistic ones. Sometimes a line doesn’t quite work, a word gets lost, or an emphasis no longer fits the scene after editing. In such cases, the original sound is replaced later.

Dialogue Re-recording

If the original sound is not optimally usable, actors re-record their lines in the studio. This process is called ADR, usually short for Automated Dialogue Replacement. In German, it is also referred to as Nachvertonung or Dialogersatz.

ADR in the Studio

The actor watches the scene and repeats the line until it matches the lip movement. The recording is made in a quiet room, free of background noise. Therefore, it sounds extremely clean, but also different from the original sound.

This cleanliness must then be undone. A voice speaking in a hall, for example, will have artificial reverb added. Without this additional enhancement, the dialogue would not fit the scene.

Dubbed Versions for Other Languages

The same process happens again for a German dubbed version, only with different voice actors and a translated script. Here, too, the voice is created in the studio and then inserted into the existing mix.

Music and sound effects are often delivered as a separate mix without dialogue. This version is called M&E, meaning Music and Effects. It serves as the basis for adding new dialogues in other languages. Where sound effects were present under the original dialogue, they must, of course, be re-added.

Sound Effects, Atmosphere, and Music

In addition to dialogues, there are three other elements that are recorded or composed separately. They remain on their own tracks until the end. Only after they are combined do they form what is heard as the final film sound.

All four run on separate tracks until the mixing stage. For dialogue intelligibility, the relationship between speech and music is particularly crucial.

Foley and Foley Artists

Footsteps, doors, rustling clothes, or a glass being set down are specifically recorded while the scene plays. The technical term for this is Foley; in German, they are called Geräuschemacher (foley artists).

This work takes place in a studio with various floor coverings and a collection of objects. What you hear in a film as, for example, footsteps on gravel, is created there and not on set. On set, these sounds would be recorded along with everything else and could not be separated later.

Sound Design for Everything Else

Not every sound can be recorded. A spaceship or a monster does not have a real sound. Such sounds are assembled on a computer from various sources. Amusingly, often from sources that have nothing to do with the scene.

This step is called sound design and is distinct from sound recording, even if both ultimately end up in the same track group.

Background Atmosphere

Atmosphere, also called ‘atmo,’ is the continuous background sound of a scene. Traffic outside the window, bird calls, or other ambient noises. It ensures that a room sounds like a room and not like a studio recording.

Music and Speech

Film music is usually created last because the composer needs to see the edited film. It follows the plot and begins at specific points, which are determined beforehand with the director. These points are recorded via timecode to ensure music and visuals align later.

For the intelligibility of spoken words, music is the most challenging element. It can overlap with the frequency range of the human voice and often plays exactly when dialogue is present.

Mixing in the Sound Studio

Ultimately, all tracks converge. During the mixing process, a re-recording mixer determines the relative volume of each element. This is where it is decided whether dialogues stand out or get lost.

Separate Tracks

The individual elements remain separated into groups called stems. Typically, there are three: one for dialogues, one for music, and one for effects. This separation is practical because individual groups can later be exchanged or re-weighted.

This is a fundamental prerequisite for dubbed versions. Newer methods for improving speech intelligibility also focus on this by increasing the dialogue component relative to the rest.

The Relationship Between Speech and Everything Else

The difference in volume between dialogue and background is a creative decision. Some directors aim for a realistic impression and accept that speech may be less prominent in the scene. Others bring dialogues clearly to the forefront.

The Audio Engineering Society considers this relationship the decisive factor for dialogue intelligibility. According to EBU studies, too small a difference between speech and background is one of the most common causes of complaints about TV sound.

Loudness Standards

For television, there are fixed guidelines that specify the loudness levels at which programs should be mixed. This ensures, for example, that channels and commercials are comparable. However, this standard only regulates the overall loudness level of a program. It does not dictate how speech and music within that program relate to each other.

Consequences for TV Sound

A cinema film is mixed for a large, quiet auditorium with many loudspeakers. On a television, this mix then encounters a living room, two small speakers on the device itself, and ambient noise. What works as quiet dialogue in the cinema can often be lost in one’s own living room.

Cinema films, series, and movies are often mixed too dynamically for broadcast and streaming, because the difference between overall loudness and dialogue loudness is large. Therefore, separate versions with reduced dynamics are often created for television and streaming services.

In object-based formats, individual sound components can be transmitted as separate objects or groups with metadata. If dialogue and background are available separately, the dialogue component can fundamentally be treated differently or adapted to the listener.