Video Keyframe Extraction using FFMPEG

Abstract

using ffmpeg for fast and effective keyframe extraction

introduction

Processing videos are costly because of the large amount of images it contains. But luckily, thanks to video compression algorithm (as explained in much better details in this post) we can use ffmpeg to quickly get out the keyframes (aka I-frames).

I-frames are basically the highest quality images in a video. The frames surrounding them only stores the “differences” w.r.t. the I-frames.

I-frames Extraction

ffmpeg -skip_frame nokey -i your_video.mp4 \
    -vsync 0 -frame_pts true output/dir/iframe_%d.jpg

• skip_frame tells the decoder to process only keyframes
• -vsync 0 preserves timestamps
• -frame_pts set the output file’s %d to each keyframe’s presentation timestamp (PTS)
  • so to compute the actual timestamp, for example iframe_75.jpg with a fps of 30, the iframe’s timestamp would be 2.50 seconds
  • you can read more about it here and here

using a filter to select “Contextual Scene Change”

no need for OpenCV, ffmpeg can do this too with a little filter action:

ffmpeg -i your_video.mp4 \
    -vf "select='gt(scene,0.3)'"
    -vsync 0 -frame_pts true output/dir/kframe_%d.jpg

• the value for scene should be a float between 0 and 1 and it measures the pixel level difference between two frames. So here our threshold is 30%

i-frames already have “Contextual Scene Changing” frames

from personal testing, there are a lot more i-frames than there are “Contextual Scene Changing” frames but they also include them. So there are no need to run both.

If your want to save downstream processing time, then filter for “Contextual Scene Changing” Frames. Otherwise, just use i-frames to be as inclusive as possible!