What is voice activity detection?

By the Potok teamUpdated ~1 min read

What is voice activity detection? It means finding the parts of a recording that contain speech and skipping silence and noise. A detector labels each short frame as speech or not speech. The speech segments are then passed on, and the gaps are dropped or ignored.

Transcribe a recording

Send your voice message, audio or video to @potok_now_bot on Telegram and the text comes back in the chat.

Open @potok_now_bot in Telegram

Files up to 2 GB · export to TXT, MD, DOCX, PDF, SRT and VTT · speaker labels and timestamps · about 30 languages

Key facts

  • Simple detectors compare the loudness of a frame with a threshold, while stronger ones use a trained model that tells speech from other sounds.
  • Trimming silence saves time and cost in processing, since the recogniser does not read minutes of an empty room.
  • Speech detection also gives a rough start and end time for each phrase, which later steps can reuse for subtitle cues.
  • Mistakes go two ways: a quiet word cut off at the start of a phrase, or a cough or music passage treated as speech.
  • A detector that works on a quiet studio file may fail in a loud street, since the noise is as loud as the voice.

What it looks like

// list silent gaps longer than half a second
ffmpeg -i call.wav -af silencedetect=noise=-30dB:d=0.5 -f null -
//   silence_start: 12.4
//   silence_end: 15.9 | silence_duration: 3.5
//   silence_start: 41.0
// the stretches between these gaps are the speech segments

More

See the full list of languages, the tools and the glossary.

Frequently asked questions

Why detect voice before transcribing?

It removes silence and noise, so the recogniser works on speech only. This lowers the chance of made-up words in the gaps.

Is VAD the same as noise reduction?

No. Detection decides which parts to keep. Noise reduction changes the sound inside the parts. The two can be used together.

Can VAD cut off the start of a sentence?

Yes, when a speaker begins softly. Good systems add a short margin around each segment to avoid clipping the first sound.

Do I need to trim silence before sending to Potok?

No. Send the full recording as it is, up to 2 GB. Trimming only helps to reduce a very large file.

How can I find silent gaps in a file?

Use the silencedetect filter in ffmpeg. It prints the start and end of every quiet section for the threshold you choose.

Sources

Each source was opened by hand on the date shown. Pages change, so open the source if a detail matters to you.

  1. Voice activity detection (Wikipedia), Wikipedia · checked
  2. Background noise (Wikipedia), Wikipedia · checked

Try it on your own recording

Send your voice message, audio or video to @potok_now_bot on Telegram and the text comes back in the chat.

Open @potok_now_bot in Telegram