FFMPEG - High-Interest Video Thumbnails

Posted at




ffmpeg -ss 100 -i my_movie.mp4 -vf "select=gt(scene\\,0.1)" -frames:v 50 -filter:v scale=200:-1,crop=iw:ih*0.8,fps=1/10 out%04d.jpg



for example this ~4minutes YouTube Video:




- the command line would probably won't give you as many as 50 thumbnails (address the command as "maximum 50 frames" it is depending on the length of the jump (-ss 100) which can be adjusted to much less on shorter videos (50, 20 or even 10),
- just bare in mind that lowering the jump rate will probably hit those black screens transitions (which are common in lammmmeeee video editing...)

Jump (-ss) 20:
icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0001

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0002

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0003

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0004

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0005

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0006

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0007

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0008

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0009

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0010

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0011

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0012

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0013

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0014

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0015

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0016

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0017

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0018

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0019

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0020

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0021

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0022

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_20__thumbnail_0023





raising jump (-ss) back to 100 will improve the storyline variety, but, naturally, will lower the number of thumbnails rendered:


icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0001

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0002

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0003

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0004

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0005

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0006

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0007

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0008

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0009

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0010

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0011

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0012

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0013

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0014

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_at_100__thumbnail_0015



oh, and, by the way- those black frames are about ~1.36KB while the other thumbnails are about ~4-9KB (depending on amount of colors in the image and placement near each other, which is how the JPEG compression algorithm works..).

another variation is the scene difference which is currently on 0.1 rendering a quite different scenes,
we can bring it around 0.4, and lower the jump to 20 to see the quality of interest, of result.
ffmpeg -ss 20 -i "PEOPLE ARE AWESOME! 2016 (UNBELIEVABLE).mp4" -vf "select=gt(scene\\,0.4)" -frames:v 50 -filter:v scale=200:-1,crop=iw:ih*0.8,fps=1/10 icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_%04d.jpg


now, those sort of videos are constructed of mostly multi-scenes with sharp transition so we would still (probably) get a nice result:


icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0001

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0002

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0003

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0004

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0005

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0006

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0007

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0008

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0009

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0010

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0011

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0012

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0013

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0014

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0015

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0016

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0017

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0018

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0019

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0020

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0021

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0022

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__thumbnail_0023


mmmm...
as you can see:

icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__comparing_2016-01-18_014240 icompile.eladkarako.com__ffmpeg_generate_high_interest_thumbnails__just_back_at_20__scene_at_0_4__comparing_2016-01-18_014408


the result is not as good as keeping the scene at lower-rates,
similar colors scenes are generated, which drop the interest rate down down down... :)

so it is always better keep the scene variable low, the jump rate (-ss) high (relative to video's length), the frames above 50, which will render more than enough thumbnails, which you can sort by size later, deleting those which are around 2KB which has low amount of colors, which also means they are boring!

this rule-of-thumb (ha! thumbnail/rule of thumb -- you see what I've did there?)
can be applied beyond removing black screens, if you'll always delete one tenth of your thumbnails, removing the ones which their size is lower first, you can alway assure a high amount of colors, which in combination of high rate of story line jump rate (-ss) and scene variation - WILL result with a high interest thumbnail scenes.

Nice!




May 23, 2016 Edit
I had great result when I've increased the bitrate tolerance using -bt and increased the frame-rate, so FFMPEG will look every 60 frame/seconds each time.

ffmpeg -threads 8 -ss 10 -i "myMovie.mkv" -bt 50M -vf "select=gt(scene\\,0.05)" -frames:v 20 -filter:v fps=1/60 out%04d.jpg


the scene difference is VERY LOW (meaning the scenes should be VERY DIFFERENT),
FFMPEG will take longer to calculate those.. BUT- you'll (probably) get a VERY different color/lighting theme, etc..
..which is a good thing :)))