AI vs Spectral Noise Reducer: Which to Use

Illustration comparing a voice AI noise reducer with spectral noise reduction on a waveform

The free Noise Reducer has two engines, and choosing between them comes down to one question: is the recording only a voice? If it is, use Voice mode, an AI speech model that keeps speech and removes everything else. If there's music, instruments, or anything you want to keep that isn't speech, use Music mode, a spectral noise reducer that turns down steady noise like hiss and hum across the full frequency range.

Both run in a Web Worker in your browser, so your audio never leaves your device. The rest of this article covers what each engine actually does, where each one goes wrong, and which to pick for the recordings people bring to it most often.

At a glance

Voice mode (AI) Music mode (spectral)
What it is GTCRN, an AI speech-enhancement model running in onnxruntime-web Spectral gating, hand-written DSP, no AI
Best for Podcasts, interviews, voice memos, phone and call recordings Songs, full mixes, sung vocals, tape and vinyl transfers
Wrong for Anything with instruments: it treats them as noise Busy, changing noise like traffic or chatter
Noise it handles Changing, non-steady noise around a voice Steady noise: hiss, hum, fan and AC rumble, broadband noise
Output Mono, content up to about 8 kHz, upsampled back to the file's rate Stereo preserved, full band at the source sample rate
Download The ~0.5 MB model plus the engine that runs it, once, then cached None
Noise profile Learned by the model during training Built automatically from the quietest ~20% of moments in the file
Typical artifact when pushed Voice sounds colored or processed Warbly, chirpy "musical noise"

In both modes you get a wet/dry Mix slider after processing (it starts at 75%; 0% is the original recording, downmixed to mono in Voice mode), and you export through the same WAV, MP3 or Opus selector.

How voice AI noise reduction decides what to keep

Voice mode doesn't measure your noise. It runs a model that was trained on large amounts of speech mixed with noise, and it learned what speech looks like. Whatever doesn't fit that pattern gets suppressed. For the inside view of how models like this are built, see How AI Noise Reduction Works.

That approach has one big advantage. The model doesn't need the noise to hold still. A door closing, a dog barking twice, a car going past, keyboard clatter under a call: none of these are steady, and a method that relies on a fixed noise profile can't follow them. A speech model can, because it is tracking the voice, not the noise.

The limits come from the same design:

  • It treats anything that isn't speech as noise. That includes guitars, pads, drums, reverb tails and cymbals. The tool shows a warning in Voice mode for exactly this reason: don't run songs through it.
  • The output is mono. The input is resampled to 16 kHz mono before the model sees it.
  • It only carries content up to about 8 kHz. That's the ceiling for 16 kHz audio (half the sample rate). The result is upsampled back to your file's rate, but the detail above 8 kHz isn't restored.

For spoken word, the 8 kHz ceiling matters less than you'd think. Intelligibility lives well below it, and phone calls have long carried speech in a far narrower band. For a sung vocal headed into a mix, it matters a lot: the breath and air on top of a vocal sit above 8 kHz, and losing them is audible.

Voice mode shows real progress and an estimated time while it works. A long file takes a while, because the model processes the whole recording on your own machine.

How spectral noise reduction removes background noise from audio

Music mode is the classic approach, and it works on what it can measure in your file. It does three things.

It builds a noise profile automatically. Classic editors often ask you to select a stretch of noise-only audio first (Audacity's Noise Reduction calls this "Get Noise Profile"). Music mode skips that step. For each frequency band, it looks at the quietest ~20% of moments across the whole file and treats that level as the noise floor. The profile is then median-smoothed across frequency. Steady hiss is smooth across the spectrum, while a held note or synth pad is a narrow peak, so the smoothing keeps sustained tones, pads and drones from being learned as noise.

It gates each band. Where the signal sits close to the noise profile, that band is turned down. Where music or a voice rises well above it, the band stays open. The gain changes are smoothed across time and frequency, which is what keeps the result from sounding warbly. Spectral subtraction pushed hard produces "musical noise", a chirpy, underwater texture made of tiny leftover blips of noise, and the smoothing is there to prevent it.

It caps the cut. Maximum attenuation is limited to about 16 dB. That's conservative on purpose. Heavy noise gets reduced rather than erased, because erasing it is where spectral methods start to eat the music.

There's also an optional checkbox to remove 50/60 Hz electrical hum. It detects which mains frequency is present and notches the fundamental and its harmonics, which works better on hum than broadband gating does. The two guides for those specific problems go further: How to Remove Hiss from Audio and How to Remove Hum and Buzz from Audio.

The tradeoff is the mirror image of Voice mode. Spectral gating is good with noise that stays the same from start to finish and weak with noise that keeps changing. Traffic, chatter and clatter don't have one profile, so there's nothing stable to subtract.

Which noise reducer mode for which recording

Here's how the choice works out for common material. When a recording sits between two cases, try both. Each takes one click and one listen.

Podcast or interview

Voice mode. A podcast is speech, and the background noise in a home studio or on location is rarely just one steady sound. For the full workflow from raw file to export, see How to Remove Background Noise from Audio.

One exception: if your intro and outro music is already mixed into the file, Voice mode will strip that too. Clean the voice tracks before adding music, or use Music mode on the finished mix.

Voice memo or phone recording

Voice mode. Phone recordings are usually mono and already narrowband, so the speech-band output costs you little, and the noise around them (street, room, handling) tends to change. How to Clean Up a Phone Recording covers the rest of the cleanup.

Zoom or video call recording

Voice mode, for the same reasons: a voice, changing noise, and audio that has already been compressed by the call. How to Clean Up a Zoom Recording walks through it. If the call is saved as a video file, pull its soundtrack out first with Video to Audio.

A song or full mix with hiss

Music mode. This is the case Voice mode's warning exists for. Music mode keeps both channels and the full frequency range, and the automatic profile picks up the steady hiss under the quiet passages.

Cassette or vinyl digitization

Music mode, with the hum checkbox on if the transfer picked up a mains buzz. Tape hiss is steady broadband noise, which is exactly what spectral gating suits. Expect it to come down rather than disappear, given the 16 dB cap. Surface clicks and pops are short, sharp events, not steady noise, so don't expect either mode to treat them the way a dedicated declicker would.

A sung vocal or acapella destined for a mix

Music mode, even though it's a voice. Voice mode's mono output, cut off around 8 kHz, drops the air and breath that make a vocal sit well in a mix. Music mode keeps the vocal full-band and in stereo.

If the vocal came out of a finished song, the leftover noise is usually bleed from other instruments, not hiss, and that's a separation problem. How to Isolate Vocals From a Song Cleanly is the place to start.

Fan or air conditioner noise

It depends on what's under the fan. A fan or AC unit is about as steady as noise gets, so both engines handle it. On a speech-only recording, Voice mode can often clean more aggressively, since Music mode caps its cut at about 16 dB, at the cost of mono, speech-band output. If there's music in the file, or you want to keep stereo and the full range, use Music mode: a steady drone is exactly what its automatic profile measures well, especially if the file has a few seconds of room tone. Remove Fan and AC Noise With a Noise Reducer has more detail.

Wind

Wind gusts, so it changes all the time, and it hits hardest in the low end. On speech, Voice mode copes with that better than a fixed profile does. Bad wind distortion at the mic can't be fully repaired by either mode. See How to Reduce Wind Noise in Recordings.

A voice with music playing in the room

Voice mode. Here, treating music as noise is the point: the background track is what you want gone. It works best when the voice is clearly louder than the music. When they're close in level, expect some music to get through and the voice to sound more processed.

When neither engine is the right fix

Some problems sound like noise but aren't, and no noise reducer will touch them properly.

How to remove background noise without artifacts

Most bad noise reduction is too much noise reduction. A few habits help with both engines:

  1. Match the mode to the material using the guide above, instead of defaulting to AI because it sounds more advanced.
  2. Use the Mix slider. It starts at 75% for a reason. Blending some of the original back in often sounds more natural than the fully processed version, and a trace of steady background is far less distracting than warbling or a thin, processed voice.
  3. Compare before and after on the quiet parts and the loud parts. Artifacts hide in the gaps between words and in reverb tails.
  4. Export WAV if you're going to keep editing. Save MP3 or Opus for the final file.

If you can't tell whether your noise is steady or changing, look at it. Open the file in the Spectrogram: steady noise shows up as an even band or a few straight horizontal lines running the whole length of the file, while changing noise appears as blotches that come and go. How to Read a Spectrogram explains what you're seeing. Steady points to Music mode; changing noise under a voice points to Voice mode.

Frequently asked questions

Is AI noise reduction better than spectral noise reduction?

Neither is better overall; they suit different material. AI speech models like the one in Vocal Cut's Voice mode handle changing noise around a voice well but treat instruments as noise and output mono speech-band audio. Spectral noise reduction, used in Music mode, keeps music and stereo intact and handles steady noise like hiss and hum, but struggles with noise that keeps changing, such as traffic or chatter.

Can I use Voice mode on music?

You can, but it will remove most of the music. Voice mode runs a speech-enhancement model that treats everything that isn't a voice as noise, so guitars, drums, pads, reverb and cymbals get suppressed. Its output is also mono and limited to about 8 kHz. For songs, backing tracks, or any recording with instruments you want to keep, use Music mode instead.

Why does my cleaned audio sound warbly or underwater?

That sound is usually musical noise, an artifact of pushing spectral noise reduction too hard: small leftover fragments of noise flicker on and off and sound chirpy or watery. A voice that sounds thin or processed after AI cleanup is the equivalent problem in Voice mode. Lower the Mix slider to blend more of the original back, or try the other mode.

Does Music mode need a noise sample?

No. Music mode builds its noise profile automatically. For each frequency band it looks at the quietest ~20% of moments across the whole file and treats that level as the noise floor, then smooths the profile so sustained notes and pads aren't mistaken for noise. You don't need to select a noise-only section, which many classic editors ask for.

Is my audio uploaded when I use the Noise Reducer?

No. Both modes run in a Web Worker in your browser, so your audio never leaves your device. Voice mode downloads its AI model (about 0.5 MB) and the engine that runs it from Vocal Cut's CDN the first time you use it, then caches both. Music mode needs no download at all. There's no account and nothing to install.