Click here to Skip to main content
15,867,704 members
Articles / Multimedia
Tip/Trick

How to Use FFmpeg Filters to Jazz Up Your Audio and Video Files

Rate me:
Please Sign up or sign in to vote.
5.00/5 (2 votes)
21 Jun 2021CPOL6 min read 19.1K   3   1
Cut, copy, convert, mix, rotate, flip, resize, crop, combine, compose, blur, sharpen, smoothen, side-by-side split, PIP inset, fade in/out using FFmpeg Filters
FFmpeg is a powerful utility for converting multimedia files but its real power is in the filters functionality. Learn about FFmpeg filters and save on apps or services that provide only a selection of the same functionality.

Introduction

You may have come across the FFmpeg utility before. It is that open-source command-line utility that can perform simple multimedia format conversions such as these:

ffmpeg -i video.avi video.mp4

ffmpeg -i audio.wav audio.mp3

FFmpeg is smart enough to pre-empt your intentions. For example, if you want to extract the audio from a video file, you do not have to strain yourself with details about codecs and stuff. FFmpeg will automatically choose the best settings:

ffmpeg -i video.mp4 audio.mp3

A lot more FFmpeg functionality is in the form of filters. The syntax for FFmpeg filters is obscure and the documentation is a bit cryptic. So, most people avoid them. Not anymore 'cos I'll learn ya.

Background

For several years, I did not bother with FFmpeg filters. The only filter I used was the volume filter, which I used (blindly) to amplify low-volume audio files. I had even written a FFmpeg tips-and-tricks article in a magazine without knowing any better. Last year, in the midst of the lockdown, I decided to write a follow-up for the article — sort of 'More FFmpeg tips and tricks'. When I went in depth about FFmpeg, I decided to write a book instead. It is now a 152-page full-colour paperback on Amazon - FFMPEG Quick Hacks. In the spirit of open source, I have made its ebook version free. The book has a full tutorial and is not just a collection of hacks. In fact, the tutorial forms much of this book. The hacks part is just one big chapter. I did not realize my mistake until it was too late.

For the book and this article, I used Linux. If you are on Windows, you need to replace the slashes (\) in multiline commands with carets.

FFmpeg Filter Syntax

FFmpeg has lots of filters - more than you can imagine. They are listed and described in detail at this link. Each one of these filters represents a processing phase that can be used individually or daisy-chained with other such filters. One or more multimedia streams (audio, video, subtitles) can be provided as input for the filter. Depending on the kind of processing done by the filter, it might have one or more output multimedia streams. The output from one filter can be used as the input for a subsequent filter.

Here is a simple example of a filter. It increases the volume of the audio stream by a third.

ffmpeg -i low.mp3 -af "volume=1.33" high.mp3

When I wanted to slightly rotate a video, I used this.

ffmpeg -y -i barbara.mp4 \
       -vf rotate=angle=-20*PI/180:fillcolor=brown \
       barb20.mp4

Image 1

The part in bold type in the above command is the filter definition. Most filter definitions in production may seem undecipherable and confusing but it is not rocket science. FFmpeg filters use a standard syntax:

[input label1][input label2]...[input_labelN]filter=key=value pair1:key=value 
 pair2...key=value pairN[output label1][output label2]...[output labelN]

Use the command switch -filter_complex before defining your filter. It handles all types of filters and eliminates confusing errors.

Multiple filters need to be separated by a semicolon (;). At the end of the string of filters though, there should be no semicolon. A filter may have one or more inputs and/or outputs. (Some special ones have none.) The inputs can be streams in the input files or any ad-hoc output label you specify in the previous filters. The output labels can be used as input for subsequent filters or for map assignments outside the filter definition. If the input(s) of a filter is/are same as the output(s) of the immediately previous filter, then a comma can be used instead of a semi-colon. Let me explain most of this with an example.

ffmpeg -i Tasmania-Nobrainia.mp4 \
       -filter_complex \
          "[0:a:0]atrim=start=60:end=120[trimmed]; 
           [trimmed]volume=volume=1.6[amped]" \
       -map '[amped]' \
       second-mminute-audio-amped.mp3

In this command, two filters are used — atrim and volume. The first filter is for extracting a particular section of its input audio stream. The second is for changing the volume level of its input audio stream. Before atrim, the input audio stream has been specified as [0:a:0]. This refers to the first audio stream in the first input file. The start and end key-value pairs specify the beginning and ending timestamps between which the extraction has to occur — the second minute. The extracted audio is then packaged as the output stream labelled as '[trimmed]'. This label is specified as the input for the next filter volume. The volume filter has a key-value pair, which is also named volume. This filter causes the volume of the input audio stream (labelled '[trimmed]') to be increased by two-thirds. The filter's output is labelled '[amped]'. The filter definition ends thereafter. [Note: There is no semicolon at the end of the filter definition.]

It is important that a filter only refers to existing input file streams or previously created output labels. Otherwise, FFmpeg will start making guesses. If it does not succeed, there will be errors. In the above command, the input audio stream specified as '[0:a:0]' is not really necessary. FFmpeg would have automatically chosen it. Given that the output format is an audio file, the last filter need not have labelled its output for the -map switch. FFmpeg would have automatically chosen it from the outputs of the last filter. [Note: DO NOT leave any outputs unused. The last outputs should match the streams required by the output file container (format).]

ffmpeg -i Tasmania-Nobrainia.mp4 \
       -filter_complex \
          "atrim=start=60:end=120[trimmed]; 
           [trimmed]volume=volume=1.6" \
       second-mminute-audio-amped.mp3

Going further, even the keys are not necessary if the values are in the order specified in the filter documentation.

ffmpeg -i Tasmania-Nobrainia.mp4 \
       -filter_complex \
          "atrim=60:120[trimmed]; 
           [trimmed]volume=1.6" \
       second-mminute-audio-amped.mp3

Most online examples of FFmpeg filters are like this. Without the filter documentation, filter definitions may look cryptic.

More FFmpeg Filter Examples

In some of his movies, Jim Carrey does a live rewind of a scene. Does he sound intelligible if you rewind that footage? Let us use FFmpeg filters and see.

ffmpeg -y -i ace-ventura-reverse.mp4 \
  -filter_complex \
     "[0:v:0]reverse[v]; [0:a:0]areverse[a]" \
  -map '[v]' -map '[a]' \
  ace-ventura-reverse-reversed.mp4

#Place them side-by-side
ffmpeg -y -i ace-ventura-reverse.mp4 -i ace-ventura-reverse-reversed.mp4 \
  -filter_complex  \
    "[0:v:0]pad=1280:360:0:0[frame];
     [frame][1:v:0]overlay=640:0[fullvideo];
     [fullvideo]drawtext=x=30:y=60:fontcolor=yellow:text=
     'Original audio on left speaker':fontsize=30:fontfile=Florentia.ttf[lefttext];
     [lefttext]drawtext=x=670:y=60:fontcolor=yellow:text=
     'Reversed audio on right speaker':fontsize=30:fontfile=Florentia.ttf[v];
     [0:a:0]channelsplit=channel_layout=mono[leftaudio];
     [1:a:0]channelsplit=channel_layout=mono[rightaudio];
     [leftaudio][rightaudio]join=inputs=2:channel_layout=stereo[a]" \
  -map '[v]' -map '[a]' ace-ventura-reversal-truth.mp4

The filters in the first FFmpeg command reverses the video and audio streams. [Note: These filters consume a lot of memory. Use short clips.] The filters in the second FFmpeg places the two videos side-by-side to see if they match up. Specifically, the audio stream from the original video is played on the left speaker while the audio from the reversed video is played on the second speaker. You may have to slide the audio balance of your volume control utility to either side to listen to only one of the audio streams.

Image 2

Some filters do not need any input. They may use certain built-in filter variables to generate audio or video output. In this command, the testsrc filter generates a video test pattern out of nothing. The aevalsrc generates a random white noise using an algebraic equation. [Note: The -f lavfi switch allows you to use the filter output as a pseudo input file.]

ffmpeg -f lavfi \
  -i "testsrc=size=320x260[out0];aevalsrc=random(0)/20[out1]" \
  -t 0:0:30 -pix_fmt yuv420p \
  test.mp4

Image 3

In the next example, the built-in filter variables iw and ih are used to resize a series of numbered images and create a slideshow video — the variables representing the actual dimensions of the current image being processed by the filter. [Note: A comma instead of a semicolon has been used to indicate that all outputs of the filter on the left are consumed by the filter on the right.]

ffmpeg -y -framerate 1/3 -i image%02d.jpg \
       -filter_complex "scale=eval=frame:w=640:h=480:force_original_aspect_ratio=decrease, 
              pad=640:480:(640-iw)/2:(480-ih)/2:yellow" \
       -pix_fmt yuv420p -framerate 24 \
       slide.mp4

Image 4

You will not crack FFmpeg filters on your first attempt. I certainly did not. After I understood how to use them though, a whole new world opened up to me. There is no need to use expensive software to create various effects. FFmpeg filters can do most of them for free.

If you did not understand something in this article, then read my free ebook. It has hundreds of examples and illustrations. Everything is explained in much more detail. After that, you can use the FFmpeg documentation and go nuts with it.

History

  • 21st June, 2021: Initial version
  • 22st June, 2021: Update about the use of comma, instead of a semicolon

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer www.VSubhash.in
India India
V. Subhash is an invisible Indian writer, programmer and illustrator. In 2020, he wrote one of the biggest jokebooks of all time and then ended up with over two dozen mostly non-fiction books including Linux Command-Line Tips & Tricks, CommonMark Ready Reference, PC Hardware Explained, Cool Electronic Projects and How To Install Solar. His book Quick Start Guide to FFmpeg has been published by Apress/SpringerNature in 2023. He wrote, illustrated, designed and produced all of his books using only open-source software. Subhash has programmed in more than a dozen languages (as varied as assembly, Java and Javascript); published software for desktop (NetCheck), mobile (Subhash Browser & RSS Reader) and web (TweetsToRSS); and designed several websites. As of 2023, he is working on a portable Javascript-free CMS using plain-jane PHP and SQLite. Subhash also occasionally writes for Open Source For You magazine and CodeProject.com.

Comments and Discussions

 
PraiseGreat! Thanks. Pin
Ken Lee 202211-Jun-22 21:02
Ken Lee 202211-Jun-22 21:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.