grounding dino 源码部署 cuda12.4 开放词汇目标检测（Open-Vocabulary Object Detection, OVOD）模型

参考

文本提示检测图像任意目标(Grounding DINO) 的使用以及全网最详细源码讲解_groundingdino-CSDN博客

IDEA-Research/GroundingDINO: [ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

grounding-dino-base · 模型库

CUDA Toolkit Archive | NVIDIA Developer

Microsoft C++ 生成工具 - Visual Studio

Error in the compilation of "ms_deform_attn_cuda.cu" when constructing the environment · Issue #405 · IDEA-Research/GroundingDINO

IDEA-Research/GroundingDINO | DeepWiki

zero-shot-object-detection-with-grounding-dino.ipynb - Colab

https://docs.nvidia.com/cuda/archive/

【Viusal studio新版本与cuda不兼容】_visual studio 2019与cuda12.8不兼容问题-CSDN博客

这个视频要看

Grounding_dino快速安装部署【教程】_哔哩哔哩_bilibili

教训：两年前的仓库就不要跑源码了

---

跟底下那个一样

没成功

./.venv/python -m pip uninstall torch torchvision torchaudio -y                                             

陈年老仓库

  ./.venv/python -m pip install torch torchvision torchaudio --index-url https://mirrors.nju.edu.cn/pytorch/whl/cu124

全选

ip/issues/11457Running setup.py develop for groundingdinoerror: subprocess-exited-with-error× python setup.py develop did not run successfully.│ exit code: 1╰─> [44 lines of output]        fatal: not a git repository (or any of the parent directories): .gitBuilding wheel groundingdino-0.1.0Compiling with CUDArunning developE:\GroundingDINO-main\.venv\lib\site-packages\setuptools\command\develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.!!********************************************************************************Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other        standards-based tools.See https://github.com/pypa/setuptools/issues/917 for details.********************************************************************************!!easy_install.initialize_options(self)E:\GroundingDINO-main\.venv\lib\site-packages\setuptools\_distutils\cmd.py:90: SetuptoolsDeprecationWarning: setup.py install is deprecated.!!********************************************************************************Please avoid running ``setup.py`` directly.Instead, use pypa/build, pypa/installer or other        standards-based tools.See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.********************************************************************************!!self.initialize_options() running egg_infowriting groundingdino.egg-info\PKG-INFOwriting dependency_links to groundingdino.egg-info\dependency_links.txtwriting requirements to groundingdino.egg-info\requires.txt     writing top-level names to groundingdino.egg-info\top_level.txt E:\GroundingDINO-main\.venv\lib\site-packages\torch\utils\cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.warnings.warn(msg.format('we could not find ninja.'))reading manifest file 'groundingdino.egg-info\SOURCES.txt'      adding license file 'LICENSE'writing manifest file 'groundingdino.egg-info\SOURCES.txt'      running build_extE:\GroundingDINO-main\.venv\lib\site-packages\torch\utils\cpp_extension.py:346: UserWarning: Error checking compiler version for cl: [WinError 2] 系统找不到指定的文件。    warnings.warn(f'Error checking compiler version for {compiler}: {error}')error: [WinError 2] 系统找不到指定的文件。[end of output]note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [44 lines of output]fatal: not a git repository (or any of the parent directories): .gitBuilding wheel groundingdino-0.1.0Compiling with CUDArunning developE:\GroundingDINO-main\.venv\lib\site-packages\setuptools\command\develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.!!********************************************************************************Please avoid running ``setup.py`` and ``easy_install``.     Instead, use pypa/build, pypa/installer or otherstandards-based tools.  See https://github.com/pypa/setuptools/issues/917 for details.********************************************************************************!!easy_install.initialize_options(self)E:\GroundingDINO-main\.venv\lib\site-packages\setuptools\_distutils\cmd.py:90: SetuptoolsDeprecationWarning: setup.py install is deprecated.!!********************************************************************************Please avoid running ``setup.py`` directly.Instead, use pypa/build, pypa/installer or otherstandards-based tools.  See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.********************************************************************************!!self.initialize_options()     running egg_infowriting groundingdino.egg-info\PKG-INFOwriting dependency_links to groundingdino.egg-info\dependency_links.txtwriting requirements to groundingdino.egg-info\requires.txtwriting top-level names to groundingdino.egg-info\top_level.txt     E:\GroundingDINO-main\.venv\lib\site-packages\torch\utils\cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.warnings.warn(msg.format('we could not find ninja.'))reading manifest file 'groundingdino.egg-info\SOURCES.txt'adding license file 'LICENSE'   writing manifest file 'groundingdino.egg-info\SOURCES.txt'running build_extE:\GroundingDINO-main\.venv\lib\site-packages\torch\utils\cpp_extension.py:346: UserWarning: Error checking compiler version for cl: [WinError 2] 系统找不到指定的文件。        warnings.warn(f'Error checking compiler version for {compiler}: {error}')error: [WinError 2] 系统找不到指定的文件。[end of output]note: This error originates from a subprocess, and is likely not a problem with pip.

x64 Native Tools Command Prompt

受不了了，我要上wsl了

要系统变量，环境变量vscode不认 然后重启vscode

12.5没镜像源

第一处修改位置（前向传播函数）
在 ms_deform_attn_cuda_forward 函数中，大约第65行附近：修改前：cpp
AT_DISPATCH_FLOATING_TYPES(value, "ms_deform_attn_forward_cuda", ([&] {ms_deformable_im2col_cuda(at::cuda::getCurrentCUDAStream(),value.data<scalar_t>() + n * im2col_step_ * per_value_size,spatial_shapes.data<int64_t>(),level_start_index.data<int64_t>(),sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size,batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,columns.data<scalar_t>());
}));
修改后：cpp
AT_DISPATCH_FLOATING_TYPES(value.scalar_type(), "ms_deform_attn_forward_cuda", ([&] {ms_deformable_im2col_cuda(at::cuda::getCurrentCUDAStream(),value.data<scalar_t>() + n * im2col_step_ * per_value_size,spatial_shapes.data<int64_t>(),level_start_index.data<int64_t>(),sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size,batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,columns.data<scalar_t>());
}));
第二处修改位置（反向传播函数）
在 ms_deform_attn_cuda_backward 函数中，大约第135行附近：修改前：cpp
AT_DISPATCH_FLOATING_TYPES(value, "ms_deform_attn_backward_cuda", ([&] {ms_deformable_col2im_cuda(at::cuda::getCurrentCUDAStream(),grad_output_g.data<scalar_t>(),value.data<scalar_t>() + n * im2col_step_ * per_value_size,spatial_shapes.data<int64_t>(),level_start_index.data<int64_t>(),sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size,batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,grad_value.data<scalar_t>() +  n * im2col_step_ * per_value_size,grad_sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,grad_attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size);
}));
修改后：cpp
AT_DISPATCH_FLOATING_TYPES(value.scalar_type(), "ms_deform_attn_backward_cuda", ([&] {ms_deformable_col2im_cuda(at::cuda::getCurrentCUDAStream(),grad_output_g.data<scalar_t>(),value.data<scalar_t>() + n * im2col_step_ * per_value_size,spatial_shapes.data<int64_t>(),level_start_index.data<int64_t>(),sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size,batch_n, spatial_size, num_heads, channels, num_levels, num_query, num_point,grad_value.data<scalar_t>() +  n * im2col_step_ * per_value_size,grad_sampling_loc.data<scalar_t>() + n * im2col_step_ * per_sample_loc_size,grad_attn_weight.data<scalar_t>() + n * im2col_step_ * per_attn_weight_size);
}));
主要修改点：将 AT_DISPATCH_FLOATING_TYPES(value, ...) 改为 AT_DISPATCH_FLOATING_TYPES(value.scalar_type(), ...)
这样可以直接使用 at::Tensor 对象的 scalar_type() 方法获取标量类型，而不需要通过旧的类型推导方式

安装成功

很难安装吗，我不觉得有什么难的

直接下载

https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth

直接运行

python groundingdino_server.py

还行

from groundingdino.util.inference import load_model, load_image, predict, annotate
import cv2
import os
# 修改模型路径为本地 weights 目录下的文件
model = load_model("groundingdino/config/GroundingDINO_SwinT_OGC.py", "weights/groundingdino_swint_ogc.pth")IMAGE_PATH = ".asset/shaft_and_sleeve_box.png"
TEXT_PROMPT = " cylindrical shaft . cylindrical sleeve . rectangular box ."# 检查文件是否存在
if not os.path.exists(IMAGE_PATH):print(f"Image file not found: {IMAGE_PATH}")print("Available files in asset directory:")if os.path.exists("./asset"):for file in os.listdir("./asset"):print(f"  {file}")exit(1)BOX_TRESHOLD = 0.35
TEXT_TRESHOLD = 0.25image_source, image = load_image(IMAGE_PATH)boxes, logits, phrases = predict(model=model,image=image,caption=TEXT_PROMPT,box_threshold=BOX_TRESHOLD,text_threshold=TEXT_TRESHOLD
)
print(f"Prediction phrases result: {phrases}")
print(f"Prediction boxes result: {boxes}")
print(f"Prediction logits result: {logits}")
annotated_frame = annotate(image_source=image_source, boxes=boxes, logits=logits, phrases=phrases)
cv2.imwrite("annotated_image.jpg", annotated_frame)