xstack - A Filter Worth Knowing
If you burrow into the documentation of ffmpeg, the excellent audio/visual toolkit, you will find, deep inside the section on Video Filters, the description of a filter called xstack.
This filter allows you to create a mosaic or matrix of video feeds, like a wall of videos:
The above figure shows how nine videos might look in such a matrix. The example feeds shown above are just a color source input, each one including a drawtext element that identifies the feed. The scripts in the code described in this techbits article created this exact image.
I’ve written up a How-To on the ffmpeg wiki on how to create and use the xstack filter. There, you can learn about the how the xstack layout works. The scripts in this collection create the required layouts automatically. (See the creatematrix.pl script for details.)
Let’s delve into the code used to create the examples. Grab the tarball here (sha256 checksum) and follow along. Obviously, you’ll need to install ffmpeg first. Follow the package installation instructions for your system to install ffmpeg.
You will also need:
- printf(1), the command line print formatting utility,
- perl(1) - a recent version of the perl scripting program.
Find a suitable working directory, and extract the tarball using your system’s tar command.
Let’s start with README.txt, where you can find the list of files and what they do:
colors.txt - a file containing the color definitions found in http://ffmpeg.org/ffmpeg-all.html#Color getcolor.pl - a perl script to extract and print a random color. Used in make1mkv.sh, play1mkv.sh make1mkv.sh - this script creates a 30 second .mkv video file and places it into ./videos. make1driver.sh - this script calls make1mkv.sh repeatedly to create and save .mkv files to ./videos play1mkv.sh - this script creates a .mkv video file and pipes it to ffplay to play for 10 seconds. the file is not saved. play1driver.sh - this script calls makeplay1mkv.sh repeatedly to play a .mkv video. No files are saved. creatematrix.pl - this perl script creates a shell script that displays a matrix of files in ./videos. the size of the matrix and choice of row or column major order can be specified as options.
The scripts that create and use videos will default to a directory named ./videos in your current directory which you will need to create (mkdir videos).
(Note: For understanding row major order and column major order, see this Wikipedia entry.)
Let’s start with make1mkv.sh. This script will create one mkv video file in the videos directory.
sh make1mkv.sh 01
To play this file, use the ffplay command as follows:
ffplay -i videos/01.mkv -t '10' -autoexit
This will play the video you just created for 10 seconds and then exit.
You could create multiple files using the make1mkv.sh1 script with different parameters by hand, but there is a driver script, make1driver.sh that will do this for you.
Two notes here - First, the make1driver.sh script uses printf(1), a command line print formatting program. Check your system documentation to make sure this program exists on your system. You may have to adjust the command syntax in some cases.
Second, by default, the script creates 36 files in the videos directory. The
files are small, just 20-25K bytes each, and, if printf(1) is working,
they will be named
If all that worked, you will see 36 files in ./videos. Now you can start creating your video matrix.
The creatematrix.pl perl script is the script used to create the video maxtrix. Use the “-h” option to get a usage message about the program:
perl creatematrix.pl -h Usage: creatematrix.pl [2|3|4|5|6] [row|col] where num is the size of the created matrix and row or col specifies major order (default is row) Sizes are as follows: 2 = vga (640x480) 2 videos (2x2) 3 = qvga (320x240) 9 videos (3x3 default) 4 = qvga (320x240) 16 videos (4x4) 5 = qqvga (160x120) 25 videos (5x5) 6 = qqvga (160x120) 36 videos (5x5) Files are looked for in the ./videos directory.
The size definitions of vga, qvga, qqvga are documented on the ffmpeg website.
With no options, the script will output a bourne shell script that creates a 3x3 video matrix in row major order:
#!/bin/sh # Start of Program # matrix_size =  # order_parm =  # matrixvar =  # ordervar = [row] # maxcount =  # path = [./videos] # There are 36 files in ./videos # We will only use  files. ffmpeg \ -i ./videos/01.mkv \ -i ./videos/02.mkv \ -i ./videos/03.mkv \ -i ./videos/04.mkv \ -i ./videos/05.mkv \ -i ./videos/06.mkv \ -i ./videos/07.mkv \ -i ./videos/08.mkv \ -i ./videos/09.mkv \ -filter_complex " \ [0:v] setpts=PTS-STARTPTS, scale=qvga [a0]; \ [1:v] setpts=PTS-STARTPTS, scale=qvga [a1]; \ [2:v] setpts=PTS-STARTPTS, scale=qvga [a2]; \ [3:v] setpts=PTS-STARTPTS, scale=qvga [a3]; \ [4:v] setpts=PTS-STARTPTS, scale=qvga [a4]; \ [5:v] setpts=PTS-STARTPTS, scale=qvga [a5]; \ [6:v] setpts=PTS-STARTPTS, scale=qvga [a6]; \ [7:v] setpts=PTS-STARTPTS, scale=qvga [a7]; \ [8:v] setpts=PTS-STARTPTS, scale=qvga [a8]; \ [a0][a1][a2][a3][a4][a5][a6][a7][a8]xstack=inputs=9:layout=0_0|w0_0|w0+w1_0|0_h0|w0_h0|w0+w1_h0|0_h0+h1|w0_h0+h1|w0+w1_h0+h1[out] \ " \ -map "[out]" \ -c:v libx264 -t '30' -f matroska - | ffplay -autoexit -left 10 -top 10 - # End of Program
You can copy and paste the output into a terminal command line session, or redirect to a temporary file and run that with the bourne shell (/bin/sh):
perl creatematrix.pl > t.sh /bin/sh t.sh
Ffmpeg will display a video matrix similar to Figure 1 above that will display for 30 seconds by default and then autoexit.
You can tweak the settings in the shell script, or to make scripts with your preferred values, edit the creatematrix.pl script. See below for additional discussion of the creatematrix.pl script.
With suitable parameters, you can create a 4x4, 5x5, or 6x6 matrix in either row or column major order:
perl creatematrix.pl 4 col > t.sh /bin/sh t.sh
perl creatematrix.pl 6 row > t.sh /bin/sh t.sh
To display larger sizes, you will need to edit the make1driver.sh script to create more files. And you need to edit the creatematrix.pl script to allow larger sizes, both left as an exercise for the reader. (Hint- in creatematrix.pl, look for where the variable $matrixvar is constrained.)
How high can you go? See if you can beat this “Matrix Madness” display: