在Rockchip平台上利用FFmpeg实现硬件解码与缩放并导出Python接口

一、为什么需要硬件加速？

在视频处理领域，4K/8K高分辨率视频的实时处理对CPU提出了极大挑战。传统软件解码在处理1080p视频时就可能占用超过50%的CPU资源。Rockchip芯片内置的RKMPP（Rockchip Media Process Platform）和RGA（Raster Graphic Acceleration）模块，可通过硬件加速实现：

1. 解码效率提升：H.264/H.265硬件解码功耗降低80%
2. 零内存拷贝：DRM_PRIME机制实现显存直通
3. 并行处理能力：专用硬件单元解放CPU资源

二、RK3588 Opencv-ffmpeg-rkmpp-rkrga编译与测试

组件	版本要求	说明
开发板	RK3568/RK3588	需支持RKMPP驱动
FFmpeg	4.3+	启用--enable-rkmpp 编译选项
Python	3.10+	需包含开发头文件

三、核心代码解释

3.1 初始化硬件上下文

av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_RKMPP, "hw", NULL, 0);

• 创建类型为RKMPP的硬件设备上下文
• 自动加载/usr/lib/libmpp.so驱动库

3.2 配置解码器

const AVCodec *decoder = avcodec_find_decoder_by_name("h264_rkmpp");
decoder_ctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);

• 使用专用解码器h264_rkmpp而非通用h264
• 硬件上下文绑定到解码器

3.3 构建滤镜链

const char *filter_descr = "hwupload=derive_device=rkmpp,scale_rkrga=w=1280:h=720:format=rgb24,hwdownload,format=rgb24";

• hwupload：将帧上传到GPU显存
• scale_rkrga：调用RGA进行硬件缩放
• hwdownload：将处理后的帧下载回系统内存

3.4 内存优化配置

frames_ctx->initial_pool_size = 20; // 显存池预分配帧数
av_hwframe_ctx_init(hw_frames_ref); // 初始化硬件帧上下文

• 避免运行时动态分配显存
• 确保内存连续满足DMA要求

3.5 Python接口封装关键

3.5.1 数据传递优化

PyObject *py_bytes = PyBytes_FromStringAndSize((const char*)filtered_frame->data[0],filtered_frame->width * filtered_frame->height * 3
);

• 直接传递RGB24数据指针
• 避免使用sws_scale转换格式产生额外拷贝

3.5.2 GIL锁处理

PyGILState_STATE gstate = PyGILState_Ensure();
/* 调用Python回调 */
PyGILState_Release(gstate);

• 在多线程环境中安全调用Python回调
• 防止与主解释器产生竞争条件

四、简单的DEMO

cat> e2e_demo.c <<-'EOF'
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavfilter/avfilter.h>
#include <libavfilter/buffersink.h>
#include <libavfilter/buffersrc.h>
#include <libavutil/opt.h>
#include <libavutil/hwcontext.h>
#include <libavutil/pixdesc.h>int main(int argc, char **argv) {AVFormatContext *in_format_ctx = NULL;AVCodecContext *decoder_ctx = NULL;AVFilterContext *buffersink_ctx = NULL;AVFilterContext *buffersrc_ctx = NULL;AVFilterGraph *filter_graph = NULL;AVBufferRef *hw_device_ctx = NULL;AVPacket *packet = NULL;AVFrame *frame = NULL, *filtered_frame = NULL;int video_stream_idx = -1;int loop_count = 5;int ret=0;// Initialize FFmpegav_log_set_level(AV_LOG_ERROR);// Initialize hardware deviceret = av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_RKMPP, "hw", NULL, 0);if (ret < 0) {fprintf(stderr, "Failed to create RKMPP hardware device\n");goto end;}// Open input fileret = avformat_open_input(&in_format_ctx, "/root/skiing.mp4", NULL, NULL);if (ret < 0) {fprintf(stderr, "Could not open input file\n");goto end;}// Find stream inforet = avformat_find_stream_info(in_format_ctx, NULL);if (ret < 0) {fprintf(stderr, "Failed to find stream information\n");goto end;}// Find video streamfor (int i = 0; i < in_format_ctx->nb_streams; i++) {if (in_format_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {video_stream_idx = i;break;}}if (video_stream_idx == -1) {fprintf(stderr, "No video stream found\n");ret = AVERROR(EINVAL);goto end;}// Find decoderconst AVCodec *decoder = avcodec_find_decoder_by_name("h264_rkmpp");if (!decoder) {fprintf(stderr, "Failed to find h264_rkmpp decoder\n");ret = AVERROR(EINVAL);goto end;}// Allocate decoder contextdecoder_ctx = avcodec_alloc_context3(decoder);if (!decoder_ctx) {fprintf(stderr, "Failed to allocate decoder context\n");ret = AVERROR(ENOMEM);goto end;}// Copy codec parameters to decoder contextret = avcodec_parameters_to_context(decoder_ctx, in_format_ctx->streams[video_stream_idx]->codecpar);if (ret < 0) {fprintf(stderr, "Failed to copy codec parameters to decoder context\n");goto end;}// Set hardware devicedecoder_ctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);// Open decoderret = avcodec_open2(decoder_ctx, decoder, NULL);if (ret < 0) {fprintf(stderr, "Failed to open decoder\n");goto end;}// Create filter graphfilter_graph = avfilter_graph_alloc();if (!filter_graph) {fprintf(stderr, "Failed to allocate filter graph\n");ret = AVERROR(ENOMEM);goto end;}// Create source filter with DRM_PRIME formatchar args[512];snprintf(args, sizeof(args),"video_size=%dx%d:pix_fmt=nv12:time_base=%d/%d:pixel_aspect=%d/%d",decoder_ctx->width