“Grounding DINO：Marrying DINO with Grounded Pre-Training for Open-Set Object Detection”的官方 PyTorch 實(shí)現(xiàn)：SoTA 開放集對(duì)象檢測(cè)器。
一、Helpful Tutorial
論文地址：
https://arxiv.org/abs/2303.05499
在 YouTube 上觀看介紹視頻：
https://www.youtube.com/watch?v=wxWDt5UiwY8&feature=youtu.be
Try the Colab Demo：
https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/zero-shot-object-detection-with-grounding-dino.ipynb
Try Official Huggingface Demo：
https://huggingface.co/spaces/ShilongLiu/Grounding_DINO_demo
二、相關(guān)的論文工作
2.1 相關(guān)的論文整理
- Grounded-SAM: Marrying Grounding DINO with Segment Anything
- Grounding DINO with Stable Diffusion
- Grounding DINO with GLIGEN for Controllable Image Editing
- OpenSeeD: A Simple and Strong Openset Segmentation Model
- SEEM: Segment Everything Everywhere All at Once
- X-GPT: Conversational Visual Agent supported by X-Decoder
- GLIGEN: Open-Set Grounded Text-to-Image Generation
- LLaVA: Large Language and Vision Assistant
2.2 論文的亮點(diǎn)
本工作的亮點(diǎn)：
- Open-Set Detection. Detect everything with language!
- High Performancce. COCO zero-shot 52.5 AP (training without COCO data!). COCO fine-tune 63.0 AP.
- Flexible. Collaboration with Stable Diffusion for Image Editting.
2.3 論文介紹
2.4 Marrying Grounding DINO and GLIGEN
2.5 輸入和輸出的說(shuō)明 / 提示
- Grounding DINO accepts an (image, text) pair as inputs.
- It outputs 900 (by default) object boxes. Each box has similarity scores across all input words. (as shown in Figures below.)
- We defaultly choose the boxes whose highest similarities are higher than a box_threshold.
- We extract the words whose similarities are higher than the text_threshold as predicted labels.
- If you want to obtain objects of specific phrases, like the dogs in the sentence two dogs with a stick., you can select the boxes with highest text similarities with dogs as final outputs.
- Note that each word can be split to more than one tokens with differetn tokenlizers. The number of words in a sentence may not equal to the number of text tokens.
- We suggest separating different category names with . for Grounding DINO.
三、環(huán)境配置過(guò)程
3.1 我的環(huán)境
系統(tǒng)：最新的ubuntu系統(tǒng)
顯卡：3090
CUDA：11.3
如果您有 CUDA 環(huán)境，請(qǐng)確保設(shè)置了環(huán)境變量 CUDA_HOME。 如果沒(méi)有可用的 CUDA，它將在 CPU-only 模式下編譯。
3.2 配置過(guò)程
3.2.1 Clone the GroundingDINO repository from GitHub
git clone https://github.com/IDEA-Research/GroundingDINO.git
下載后即可找到對(duì)應(yīng)的文件夾：
3.2.2 Change the current directory to the GroundingDINO folder
cd GroundingDINO/
3.2.3 Install the required dependencies in the current directory
pip3 install -q -e .
不知道為什么，我這個(gè)下載一直報(bào)錯(cuò)！換一個(gè)新的下載方式：
python setup.py install
但是也會(huì)飄紅！
這個(gè)時(shí)候不要害怕，遇到錯(cuò)誤的包，直接使用 pip 下載即可，耐得住性子，最后再運(yùn)行上面的安裝命令，即可順利成功！
3.2.4 Create a new directory called “weights” to store the model weights
mkdir weights
Change the current directory to the “weights” folder：
cd weights
Download the model weights file：
wget -q https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
四、測(cè)試
Check your GPU ID (only if you’re using a GPU)：
nvidia-smi
Replace {GPU ID}, image_you_want_to_detect.jpg, and “dir you want to save the output” with appropriate values in the following command：
CUDA_VISIBLE_DEVICES={GPU ID} python demo/inference_on_a_image.py \
-c /GroundingDINO/groundingdino/config/GroundingDINO_SwinT_OGC.py \
-p /GroundingDINO/weights/groundingdino_swint_ogc.pth \
-i image_you_want_to_detect.jpg \
-o "dir you want to save the output" \
-t "chair"
[--cpu-only] # open it for cpu mode
當(dāng)然了，我們也可以使用 Python 進(jìn)行測(cè)試：
from groundingdino.util.inference import load_model, load_image, predict, annotate
import cv2
model = load_model("./GroundingDINO/groundingdino/config/GroundingDINO_SwinT_OGC.py", "./GroundingDINO/weights/groundingdino_swint_ogc.pth")
IMAGE_PATH = "./GroundingDINO/weights/1.png"
TEXT_PROMPT = "person . bike . bottle ."
BOX_TRESHOLD = 0.35
TEXT_TRESHOLD = 0.25
image_source, image = load_image(IMAGE_PATH)
boxes, logits, phrases = predict(
model=model,
image=image,
caption=TEXT_PROMPT,
box_threshold=BOX_TRESHOLD,
text_threshold=TEXT_TRESHOLD
)
annotated_frame = annotate(image_source=image_source, boxes=boxes, logits=logits, phrases=phrases)
cv2.imwrite("./GroundingDINO/weights/annotated_image.jpg", annotated_frame)
我們的測(cè)試原圖片為：
測(cè)試后的圖片為：文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-467624.html
文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-467624.html
到了這里，關(guān)于【計(jì)算機(jī)視覺(jué) | 目標(biāo)檢測(cè)】Grounding DINO 深度學(xué)習(xí)環(huán)境的配置（含案例）的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
