Furkan Mudanyalı

A Brief Overlook into Audio and Video Codecs and Ways to Improve Them Using Filters

Today, in 27 October 2022, I gave a presentation about audio and video codecs for my Smart Multimedia Systems course given by Dr. Sibel Malkoş in Kadir Has University. I talked about containers, lossy and lossless codecs and some filters with examples to back them up. You can see the full presentation video that I got permission to share below, along with the examples I used.

Link to the PowerPoint presentation (PDF)
Link to the PowerPoint presentation (PPTX)

The Script I Used to Capture Footage

Link to the script


#!/bin/bash

# Device config
VIDEO_DEVICE=/dev/video2
VIDEO_CAPABILITIES="video/x-raw, format=YUY2, framerate=30000/1001, width=640, height=480, pixel-format-ratio=11/10"
AUDIO_DEVICE=hw:4,0
AUDIO_CAPABILITIES="audio/x-raw, rate=48000, channels=2"
TV_NORM=NTSC
TV_INPUT=1
FILENAME=$(date +"%Y%m%d_%H_%M_%S.mkv")

# Stream config
QUEUE="queue max-size-buffers=0 max-size-time=0 max-size-bytes=0"
DO_TIMESTAMP="do-timestamp=false"

# Encoding config
# Deinterlacing options:
# mode: 0 => send_frame / 1 => send_field
# parity: 0 => top field first / 1 => bottom field first
# deint: 0 => all frames / 1 only marked interlaced
DEINTERLACE="yadif=mode=1:parity=0:deint=0"
SCALE="scale=iw*2:-1:flags=experimental+full_chroma_int+full_chroma_inp+accurate_rnd"
CROP="crop=iw-0.06*iw:ih-0.06*ih"

# Set input for device
v4l2-ctl --device=$VIDEO_DEVICE -i $TV_INPUT

# Basically GStreamer will be the input for FFplay which will show the preview
# GStreamer will have 2 sinks, 1 for recording raw footage to file and the other
# a pipe for ffplay.
ffplay -i <(
gst-launch-1.0 -q \
  v4l2src device="$VIDEO_DEVICE" $DO_TIMESTAMP norm="$TV_NORM" \
  ! $VIDEO_CAPABILITIES \
  ! $QUEUE \
  ! tee name=videoTee \
  alsasrc device="$AUDIO_DEVICE" $DO_TIMESTAMP \
  ! $AUDIO_CAPABILITIES \
  ! $QUEUE \
  ! tee name=audioTee \
  matroskamux name=pipeMux \
  ! $QUEUE \
  ! fdsink fd=1 \
  matroskamux name=fileMux \
  ! $QUEUE \
              ! filesink location="$FILENAME" \
  audioTee. ! $QUEUE ! pipeMux. \
  audioTee. ! $QUEUE ! fileMux. \
  videoTee. ! $QUEUE ! pipeMux. \
  videoTee. ! $QUEUE ! fileMux.
) \
  -hide_banner \
  -fflags nobuffer \
  -flags +ildct+ilme \
  -analyzeduration 0 \
  -probesize 32 \
  -rtbufsize 128M \
  -threads 8 \
  -sync ext

Ratchet: Deadlocked (PS2) Footage with Weave Deinterlacing

Ratchet: Deadlocked (PS2) Footage with YADIF Line Doubling


ffmpeg –i input.mkv -vf "yadif=mode=0:parity=0:deint=0" yadif.mp4

Ratchet: Deadlocked (PS2) Footage with BWDIF Line Doubling


ffmpeg –i input.mkv -vf "bwdif=mode=0:parity=0:deint=0" bwdif.mp4

Big Buck Bunny passed through Minterpolate filter with basic blending


ffmpeg -i input.avi -vf "minterpolate=fps=48:mi_mode=blend" minterpolate_basic.mp4

Big Buck Bunny passed through Minterpolate filter with all the available options


ffmpeg -i input.avi -vf "minterpolate=fps=48:mi_mode=mci:mc_mode=aobmc:me_mode=bidir:vsbmc=1" minterpolate_advanced.mp4

Big thanks to the Blender Foundation for the Big Buck Bunny short film!
(c) copyright 2008, Blender Foundation / www.bigbuckbunny.org
Licensed under CC BY 3.0