当前位置： 首页 > news >正文

GPU视频编解码：Jetson VPI+multimedia_api视频编解码入门（一）

news 来源：原创 2025/5/1 9:25:13

目录

一.Pipline与工具栈

二.硬件设备概况

三.GPU视频编解码框架

四.VPI编译使用实例

五. jetson_multimedia_api编译使用实例

一.Pipline与工具栈

二.硬件设备概况

三.GPU视频编解码框架

  1. jetson设备目前不支持VPF框架，关于VPF的使用我在下节PC段使用X86进行安装与演示
  2. jetson目前支持的GPU编解码框架为VPI和jetson_multimedia_api 
    #1.主机端
agx@ubuntu:~$ ls /usr/src/jetson_multimedia_api/
argus  data  include  LEGAL  LICENSE  Makefile  README  samples  tools
agx@ubuntu:~$ ls /opt/
containerd/  genymobile/  nvidia/      ota_package/ todesk/
agx@ubuntu:~$ ls /opt/nvidia/vpi2/
bin  doc  etc  include  lib  lib64  samples  share

#2.docker端
agx@ubuntu:~$ docker images
REPOSITORY                   TAG                  IMAGE ID       CREATED       SIZE
nvcr.io/nvidia/l4t-pytorch   r35.2.1-pth2.0-py3   853b58c1dce6   2 years ago   11.7GB
agx@ubuntu:~$ docker exec  -it nvpy bash
root@7666a2ca87d3:/# ls /usr/src/jetson_multimedia_api/
LEGAL  LICENSE  Makefile  README  argus  data  include  samples  tools
root@7666a2ca87d3:/# ls /opt/nvidia/vpi2/
bin  doc  etc  include  lib  lib64  samples  share

四.VPI编译使用实例

        1.运行结果

root@7666a2ca87d3:/opt/nvidia/vpi2# cd s
samples/ share/   
root@7666a2ca87d3:/opt/nvidia/vpi2# cd samples/
01-convolve_2d/           03-harris_corners/        05-benchmark/             07-fft/                   09-tnr/                   11-fisheye/               13-optflow_dense/         15-image_view/            17-template_matching/     assets/                   
02-stereo_disparity/      04-rescale/               06-klt_tracker/           08-cross_aarch64_l4t/     10-perspwarp/             12-optflow_lk/            14-background_subtractor/ 16-vpi_pytorch/           18-orb_feature_detector/  tutorial_blur/            
root@7666a2ca87d3:/opt/nvidia/vpi2# cd samples/01-convolve_2d/
root@7666a2ca87d3:/opt/nvidia/vpi2/samples/01-convolve_2d# python3 main.py --backend=cuda --input "/opt/nvidia/vpi2/share/backgrounds/NVIDIA_icon.png"
root@7666a2ca87d3:/opt/nvidia/vpi2/samples/01-convolve_2d#

        2.源码 

import sys
import vpi
import numpy as np
from PIL import Image
from argparse import ArgumentParser

# Parse command line arguments
parser = ArgumentParser()
parser.add_argument('--backend', choices=['cpu','cuda','pva'],default="cuda",
                    help='Backend to be used for processing')

parser.add_argument('--input',default="/opt/nvidia/vpi2/share/backgrounds/NVIDIA_icon.png",
                    help='Image to be used as input')

args = parser.parse_args();

if args.backend == 'cpu':
    backend = vpi.Backend.CPU
elif args.backend == 'cuda':
    backend = vpi.Backend.CUDA
else:
    assert args.backend == 'pva'
    backend = vpi.Backend.PVA

# Load input into a vpi.Image
try:
    input = vpi.asimage(np.asarray(Image.open(args.input)))
except IOError:
    sys.exit("Input file not found")
except:
    sys.exit("Error with input file")

# Convert it to grayscale
input = input.convert(vpi.Format.U8, backend=vpi.Backend.CUDA)

# Define a simple edge detection kernel
kernel = [[ 1, 0, -1],
          [ 0, 0,  0],
          [-1, 0, 1]]

# Using the chosen backend,
with backend:
    # Run input through the convolution filter
    output = input.convolution(kernel, border=vpi.Border.ZERO)

# Save result to disk
Image.fromarray(output.cpu()).save('edges_python'+str(sys.version_info[0])+'_'+args.backend+'.png')

         3.结果展示(上面用的是一个滤波)

五. jetson_multimedia_api编译使用实例

        1.cuda h264编码（bug警告，能编译通过·但是无法OSD，后续两个实验直接在jetson-dektop上面实验的，就行了）

root@ubuntu:/usr/src/jetson_multimedia_api/samples/03_video_cuda_enc# make clean
root@ubuntu:/usr/src/jetson_multimedia_api/samples/03_video_cuda_enc# make
Compiling: video_cuda_enc_csvparser.cpp
Compiling: video_cuda_enc_main.cpp
make[1]: 进入目录“/usr/src/jetson_multimedia_api/samples/common/classes”
Compiling: NvElementProfiler.cpp
Compiling: NvElement.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvBuffer.cpp
Compiling: NvLogging.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvUtils.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvBufSurface.cpp
Compiling: NvV4l2Element.cpp
make[1]: 离开目录“/usr/src/jetson_multimedia_api/samples/common/classes”
make[1]: 进入目录“/usr/src/jetson_multimedia_api/samples/common/algorithm/cuda”
Compiling: NvAnalysis.cu
Compiling: NvCudaProc.cpp
make[1]: 离开目录“/usr/src/jetson_multimedia_api/samples/common/algorithm/cuda”
Linking: video_cuda_enc
root@ubuntu:/usr/src/jetson_multimedia_api/samples/03_video_cuda_enc# ./video_cuda_enc ../../data/Video/sample_outdoor_car_1080p_10fps.yuv 1920 1080 H264 test.h264
段错误 (核心已转储)

        2. cuda h264解码

root@ubuntu:/usr/src/jetson_multimedia_api/samples/02_video_dec_cuda# ./video_dec_cuda ../../data/Video/sample_outdoor_car_1080p_10fps.h264 H264
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Starting decoder capture loop thread
Input file read complete
Video Resolution: 1920x1080
[INFO] (NvEglRenderer.cpp:110) <renderer0> Setting Screen width 1920 height 1080
Query and set capture successful
Exiting decoder capture loop thread
App run was successful

        3.cuda h264解码+tensorrt目标检测：

        GPU算法检测与结果缓存

root@ubuntu:/usr/src/jetson_multimedia_api/samples/02_video_dec_cuda# cd ../04_video_dec_trt/
root@ubuntu:/usr/src/jetson_multimedia_api/samples/04_video_dec_trt# ./video_dec_trt 2 ../../data/Video/sample_outdoor_car_1080p_10fps.h264  ../../data/Video/sample_outdoor_car_1080p_10fps.h264 H264 --trt-onnxmodel  ../../data/Model/resnet10/resnet10_dynamic_batch.onnx --trt-mode 0
set onnx modefile: ../../data/Model/resnet10/resnet10_dynamic_batch.onnx
Using cached TRT model
Deserialization required 13048 microseconds.
Total per-runner device persistent memory is 5632
Total per-runner host persistent memory is 45440
Allocated activation device memory of size 22138880
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Starting decoder capture loop thread
Input file read complete
Video Resolution: 1920x1080
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Resolution change successful
Starting decoder capture loop thread
Input file read complete
Video Resolution: 1920x1080
Resolution change successful
Time elapsed:1 ms per frame in past 100 frames
Time elapsed:1 ms per frame in past 100 frames
Time elapsed:1 ms per frame in past 100 frames
Time elapsed:1 ms per frame in past 100 frames
Time elapsed:1 ms per frame in past 100 frames
Time elapsed:1 ms per frame in past 100 frames

         CUDA-H264视频解码+OSD

root@ubuntu:/usr/src/jetson_multimedia_api/samples/02_video_dec_cuda# ./video_dec_cuda ../../data/Video/sample_outdoor_car_1080p_10fps.h264 H264 --bbox-file result0.txt 
ctx.osd_file_path:result0.txt
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Starting decoder capture loop thread
Input file read complete
Video Resolution: 1920x1080
[INFO] (NvEglRenderer.cpp:110) <renderer0> Setting Screen width 1920 height 1080
Query and set capture successful
Exiting decoder capture loop thread
App run was successful
root@ubuntu:/usr/src/jetson_multimedia_api/samples/02_video_dec_cuda# ls
Makefile  resuilt.txt  result0.txt  result1.txt  result.txt  videodec_csvparser.cpp  videodec_csvparser.o  video_dec_cuda  videodec.h  videodec_main.cpp  videodec_main.o
root@ubuntu:/usr/src/jetson_multimedia_api/samples/02_video_dec_cuda# ./video_dec_cuda ../../data/Video/sample_outdoor_car_1080p_10fps.h264 H264
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Starting decoder capture loop thread
Input file read complete
Video Resolution: 1920x1080
[INFO] (NvEglRenderer.cpp:110) <renderer0> Setting Screen width 1920 height 1080
Query and set capture successful

相关文章：

  • Effective C++ 剖析（条款1~9）
  • 【软件工程】03_软件需求分析
  • 速通大厂测开
  • 介绍如何使用Python构建引文网络，并计算结构洞指标
  • 深入解析 `SQL_SMALL_RESULT`：MySQL 的“小优化”大作用
  • 【unity实战】用unity封装一个复杂全面且带不同射击模式的飞机大战射击系统
  • Navicat 17.2 正式发布 | AI 助手、支持 Snowflake...超多新功能等你解锁
  • [贪心算法]-最大数（lambda 表达式的补充）
  • 4.JVM-垃圾回收介绍
  • 双链笔记新选择！使用Docker私有化部署Logseq知识库远程团队协作
  • Java学习------static、final、this、super关键字
  • 【最佳实践】Go 状态模式
  • 基于CNN的多种类蝴蝶图像分类
  • SAP学习笔记 - 用Deepseek 整理SAP 01 - MM模块中的常用表及主键
  • 自然语言处理编程文档
  • 8051汇编--条件转移指令
  • OceanBase 中，如何抓包分析应用连接超时的问题
  • SourceTree的安装与使用
  • 【Go每日一练】计算整数数组的最大子数组和
  • docker安装node部分问题
  • 美国务院宣布新一轮与伊朗相关的制裁
  • 神十九都带回了哪些实验样品？果蝇等生命类样品已交付科学家
  • 圆桌｜如何应对特朗普政府的关税霸凌？一种联合国视角的思考
  • 招商蛇口：一季度营收约204亿元，净利润约4.45亿元
  • 游客曝九寨沟打网约车被出租车围堵，官方：前者违规，后者做法不对
  • 4月份全国93个国家气象站日最高气温达到或突破极值