Notes of Frame Extraction via FFMPEG
Published:
During former months I have did some work on keyframe extraction with FFMPEG, below will be some notes about how to using it for motion vector processing and frame extraction.
Video Frame
Normally the expressed video will contains different types of frames, including I-frame, P-frame, and B-frame, and combined into GOP (group of pictures). I-frame is the uncompressed one, usually used as video summarization. P and B frame will be divided into several macro-blocks, and each one connected to former or next frame, depends on its type (P or B block). P frame only includes P blocks while B frame can includes both types.
Motion Vector
The vector which present the motion of macro-blocks is so-called motion vector. In FFMPEG, motion vector can be extracted from compressed videos. All vector info can be printed and taken for further processing.
Below is the sample code of checking frame and macroblock type via FFMPEG.
static int decode_packet(const AVPacket *pkt)
{
framemvBuff[video_frame_count] == (-1.0);
int ret = avcodec_send_packet(video_dec_ctx, pkt);
if (ret < 0) {
fprintf(stderr, "Error while sending a packet to the decoder: %s\n", av_err2str(ret));
return ret;
}
int index = 0;
while (ret >= 0) {
ret = avcodec_receive_frame(video_dec_ctx, frame);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
break;
} else if (ret < 0) {
fprintf(stderr, "Error while receiving a frame from the decoder: %s\n", av_err2str(ret));
return ret;
}
if (ret >= 0) {
int i;
AVFrameSideData *sd;
sd = av_frame_get_side_data(frame, AV_FRAME_DATA_MOTION_VECTORS);
if (sd) {
// extarct all motion vectors of this frame
const AVMotionVector *mvs = (const AVMotionVector *)sd->data;
for (i = 0; i < sd->size / sizeof(*mvs); i++) {
const AVMotionVector *mv = &mvs[i];
// P block
if (mv->source == -1)
{
...
}
// B block
else if(mv->source == 1)
{
...
}
index++;
}
}
// I frame
else {
...
}
av_frame_unref(frame);
}
video_frame_count++;
}
return 0;
}
Frame Extraction
For extract specific frames from video via FFMPEG, below script works.
import os
for i in frame_index:
cmd = "ffmpeg -i " + filename + " -vf select='eq(n\,"+str(i)+")' -vframes 1 " + outpath + str(i) + ".jpg"
os.system(cmd)