一、前言
圖像配準(zhǔn)是一種圖像處理技術(shù)，用于將多個(gè)場(chǎng)景對(duì)齊到單個(gè)集成圖像中。在這篇文章中，我將討論如何在可見(jiàn)光及其相應(yīng)的熱圖像上應(yīng)用圖像配準(zhǔn)。在繼續(xù)該過(guò)程之前，讓我們看看什么是熱圖像及其屬性。
二、熱紅外數(shù)據(jù)介紹
熱圖像本質(zhì)上通常是灰度圖像：黑色物體是冷的，白色物體是熱的，灰色的深度表示兩者之間的差異。 然而，一些熱像儀會(huì)為圖像添加顏色，以幫助用戶識(shí)別不同溫度下的物體。
圖1 左圖為可見(jiàn)光；有圖為熱紅外圖像
上面兩個(gè)圖像是可見(jiàn)的，它是對(duì)應(yīng)的熱圖像，你可以看到熱圖像有點(diǎn)被裁剪掉了。 這是因?yàn)樵跓釄D像中并沒(méi)有捕獲整個(gè)場(chǎng)景，而是將額外的細(xì)節(jié)作為元數(shù)據(jù)存儲(chǔ)在熱圖像中。
因此，為了執(zhí)行配準(zhǔn)，我們要做的是找出可見(jiàn)圖像的哪一部分出現(xiàn)在熱圖像中，然后對(duì)圖像的該部分應(yīng)用配準(zhǔn)。
圖2 .與熱圖像匹配后裁剪的可見(jiàn)圖像
為了執(zhí)行上述操作，基本上包含兩張圖像，一張參考圖像和另一張要匹配的圖像。 因此，下面的算法會(huì)找出參考圖像的哪一部分出現(xiàn)在第二張圖像中，并為您提供匹配圖像部分的位置。
現(xiàn)在我們知道熱圖像中存在可見(jiàn)圖像的哪一部分，我們可以裁剪可見(jiàn)圖像，然后對(duì)生成的圖像進(jìn)行配準(zhǔn)。
三、配準(zhǔn)過(guò)程
為了執(zhí)行配準(zhǔn)，我們要做的是找出將像素從可見(jiàn)圖像映射到熱圖像的特征點(diǎn)，這在本文中進(jìn)行了解釋?zhuān)坏┪覀儷@得了一定數(shù)量的像素，我們就會(huì)停止并開(kāi)始映射這些像素，從而完成配準(zhǔn)過(guò)程完成了。
圖3 熱成像到可見(jiàn)光圖像配準(zhǔn)
一旦我們執(zhí)行了配準(zhǔn)，如果匹配正確，我們將獲得具有配準(zhǔn)圖像的輸出，如下圖所示。
圖4 最終輸出結(jié)果
我對(duì) 400 張圖像的數(shù)據(jù)集執(zhí)行了此操作，獲得的結(jié)果非常好。 錯(cuò)誤數(shù)量很少，請(qǐng)參考下面的代碼，看看一切是如何完成的。
from __future__ import print_function
import numpy as np
import argparse
import glob
import cv2
import os
MAX_FEATURES = 500
GOOD_MATCH_PERCENT = 0.15
#function to align the thermal and visible image, it returns the homography matrix
def alignImages(im1, im2,filename):
# Convert images to grayscale
im1Gray = cv2.cvtColor(im1, cv2.COLOR_BGR2GRAY)
im2Gray = cv2.cvtColor(im2, cv2.COLOR_BGR2GRAY)
# Detect ORB features and compute descriptors.
orb = cv2.ORB_create(MAX_FEATURES)
keypoints1, descriptors1 = orb.detectAndCompute(im1Gray, None)
keypoints2, descriptors2 = orb.detectAndCompute(im2Gray, None)
# Match features.
matcher = cv2.DescriptorMatcher_create(cv2.DESCRIPTOR_MATCHER_BRUTEFORCE_HAMMING)
matches = matcher.match(descriptors1, descriptors2, None)
# Sort matches by score
matches.sort(key=lambda x: x.distance, reverse=False)
# Remove not so good matches
numGoodMatches = int(len(matches) * GOOD_MATCH_PERCENT)
matches = matches[:numGoodMatches]
# Draw top matches
imMatches = cv2.drawMatches(im1, keypoints1, im2, keypoints2, matches, None)
if os.path.exists(os.path.join(args["output"],"registration")):
pass
else:
os.mkdir(os.path.join(args["output"],"registration"))
cv2.imwrite(os.path.join(args["output"],"registration",filename), imMatches)
# Extract location of good matches
points1 = np.zeros((len(matches), 2), dtype=np.float32)
points2 = np.zeros((len(matches), 2), dtype=np.float32)
for i, match in enumerate(matches):
points1[i, :] = keypoints1[match.queryIdx].pt
points2[i, :] = keypoints2[match.trainIdx].pt
# Find homography
h, mask = cv2.findHomography(points1, points2, cv2.RANSAC)
# Use homography
height, width, channels = im2.shape
im1Reg = cv2.warpPerspective(im1, h, (width, height))
return im1Reg, h
# construct the argument parser and parse the arguments
# run the file with python registration.py --image filename
ap = argparse.ArgumentParser()
# ap.add_argument("-t", "--template", required=True, help="Path to template image")
ap.add_argument("-i", "--image", required=False,default=r"熱紅外圖像的路徑",
help="Path to images where thermal template will be matched")
ap.add_argument("-v", "--visualize",required=False,default=r"真彩色影像的路徑")
ap.add_argument("-o", "--output",required=False,default=r"保存路徑")
args = vars(ap.parse_args())
# put the thermal image in a folder named thermal and the visible image in a folder named visible with the same name
# load the image image, convert it to grayscale, and detect edges
template = cv2.imread(args["image"])
template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)
template = cv2.Canny(template, 50, 200)
(tH, tW) = template.shape[:2]
cv2.imshow("Template", template)
#cv2.waitKey(0)
# loop over the images to find the template in
# load the image, convert it to grayscale, and initialize the
# bookkeeping variable to keep track of the matched region
image = cv2.imread(args["visualize"])
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
found = None
# loop over the scales of the image
for scale in np.linspace(0.2, 1.0, 20)[::-1]:
# resize the image according to the scale, and keep track
# of the ratio of the resizing
resized = cv2.resize(gray, (int(gray.shape[1] * scale),int(gray.shape[0] * scale)))
r = gray.shape[1] / float(resized.shape[1])
# if the resized image is smaller than the template, then break
# from the loop
if resized.shape[0] < tH or resized.shape[1] < tW:
break
# detect edges in the resized, grayscale image and apply template
# matching to find the template in the image
edged = cv2.Canny(resized, 50, 200)
result = cv2.matchTemplate(edged, template, cv2.TM_CCOEFF)
(_, maxVal, _, maxLoc) = cv2.minMaxLoc(result)
# check to see if the iteration should be visualized
if True:
# draw a bounding box around the detected region
clone = np.dstack([edged, edged, edged])
cv2.rectangle(clone, (maxLoc[0], maxLoc[1]),
(maxLoc[0] + tW, maxLoc[1] + tH), (0, 0, 255), 2)
cv2.imshow("Visualize", clone)
#cv2.waitKey(0)
# if we have found a new maximum correlation value, then update
# the bookkeeping variable
if found is None or maxVal > found[0]:
found = (maxVal, maxLoc, r)
# unpack the bookkeeping variable and compute the (x, y) coordinates
# of the bounding box based on the resized ratio
(_, maxLoc, r) = found
(startX, startY) = (int(maxLoc[0] * r), int(maxLoc[1] * r))
(endX, endY) = (int((maxLoc[0] + tW) * r), int((maxLoc[1] + tH) * r))
# draw a bounding box around the detected result and display the image
cv2.rectangle(image, (startX, startY), (endX, endY), (0, 0, 255), 2)
crop_img = image[startY:endY, startX:endX]
#cv2.imshow("Image", image)
cv2.imshow("Crop Image", crop_img)
#cv2.waitKey(0)
#name = r"E:\temp\data5/thermal/"+args["image"]+'.JPG'
thermal_image = cv2.imread(args["image"], cv2.IMREAD_COLOR)
#cropping out the matched part of the thermal image
crop_img = cv2.resize(crop_img, (thermal_image.shape[1], thermal_image.shape[0]))
#cropped image will be saved in a folder named output
if os.path.exists(os.path.join(args["output"],"process")):
pass
else:
os.mkdir(os.path.join(args["output"],"process"))
cv2.imwrite(os.path.join(args["output"],"process", os.path.basename(args["visualize"])),crop_img)
#both images are concatenated and saved in a folder named results
final = np.concatenate((crop_img, thermal_image), axis = 1)
if os.path.exists(os.path.join(args["output"],"results")):
pass
else:
os.mkdir(os.path.join(args["output"],"results"))
cv2.imwrite(os.path.join(args["output"],"results", os.path.basename(args["visualize"])),final)
#cv2.waitKey(0)
# Registration
# Read reference image
refFilename = args["image"]
print("Reading reference image : ", refFilename)
imReference = cv2.imread(refFilename, cv2.IMREAD_COLOR)
# Read image to be aligned
imFilename = os.path.join(args["output"],"process", os.path.basename(args["visualize"]))
print("Reading image to align : ", imFilename);
im = cv2.imread(imFilename, cv2.IMREAD_COLOR)
file_name=os.path.basename(args["image"])+'_registration.JPG'
imReg, h = alignImages(im,imReference,file_name)
cv2.imwrite(os.path.join(args["output"],"results", os.path.basename(args["image"])+'_result.JPG'),imReg)
print("Estimated homography : \n", h)我們已經(jīng)成功地進(jìn)行了熱到可見(jiàn)圖像配準(zhǔn)。你可以用你的數(shù)據(jù)集來(lái)嘗試一下，然后看看結(jié)果。
后續(xù)：
????????因opencv版本問(wèn)題做了修改，最終結(jié)果可以在registration和result保存路徑下查看，其中opencv原因需要英文路徑，調(diào)用使用方法如下：文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-469999.html
python .\main.py -i “熱紅外影像路徑” -v “真彩色影像路徑” -o “保存路徑”文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-469999.html
到了這里，關(guān)于熱紅外相機(jī)圖片與可見(jiàn)光圖片配準(zhǔn)教程的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
