目錄
前言
一、Opencv采集數(shù)字圖像
二、標(biāo)記圖像數(shù)字位置
三、yolov4-tiny機(jī)器學(xué)習(xí)訓(xùn)練
四、jetson nano識(shí)別數(shù)字
五、識(shí)別效果
單個(gè)數(shù)字識(shí)別
????
兩個(gè)數(shù)字識(shí)別
?四個(gè)數(shù)字識(shí)別
命令行結(jié)果顯示
小車OLED屏幕顯示
總結(jié)+數(shù)據(jù)集權(quán)重文件+小車程序
前言
啊！四天三夜的電賽終于結(jié)束了，我們組做了兩輛送藥小車，先上作品圖！
兩輛車都能完成基礎(chǔ)任務(wù)，配合后發(fā)揮任務(wù)一和二也都能完成，國(guó)賽初評(píng)結(jié)果是國(guó)二，勉勉強(qiáng)強(qiáng)吧，現(xiàn)在我來(lái)分享一下完成任務(wù)的過(guò)程，訓(xùn)練識(shí)別這里可以參考我之前寫的第七屆全國(guó)大學(xué)生工程訓(xùn)練大賽智能+賽道生活垃圾分類垃圾訓(xùn)練步驟（win10+yolov4-tiny）
一、Opencv采集數(shù)字圖像
我用Python編寫了一個(gè)OpenCV拍照腳本，代碼如下：
import cv2
import os
print("=============================================")
print("= 熱鍵(請(qǐng)?jiān)跀z像頭的窗口使用)： =")
print("= z: 更改存儲(chǔ)目錄 =")
print("= x: 拍攝圖片 =")
print("= q: 退出 =")
print("=============================================")
print()
class_name = input("請(qǐng)輸入存儲(chǔ)目錄：")
while os.path.exists(class_name):
class_name = input("目錄已存在！請(qǐng)輸入存儲(chǔ)目錄：")
os.mkdir(class_name)
index = 1
cap = cv2.VideoCapture(0)
width = 640
height = 480
w = 360
cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
crop_w_start = (width-w)//2
crop_h_start = (height-w)//2
print(width, height)
while True:
# get a frame
ret, frame = cap.read()
# show a frame
#frame = frame[crop_h_start:crop_h_start+w, 400]
#frame = cv2.flip(frame,1,dst=None)
cv2.imshow("capture", frame)
input = cv2.waitKey(1) & 0xFF
if input == ord('z'):
class_name = input("請(qǐng)輸入存儲(chǔ)目錄：")
while os.path.exists(class_name):
class_name = input("目錄已存在！請(qǐng)輸入存儲(chǔ)目錄：")
os.mkdir(class_name)
elif input == ord('x'):
cv2.imwrite("%s/%d.jpg" % (class_name, index),
cv2.resize(frame, (224, 224), interpolation=cv2.INTER_AREA))
print("%s: %d 張圖片" % (class_name, index))
index += 1
if input == ord('q'):
break
首先將USB攝像頭固定在車上之后將車放在場(chǎng)地上進(jìn)行拍照，每一個(gè)數(shù)字拍攝100-200張224*224的圖像，然后保存在文件夾中。
二、標(biāo)記圖像數(shù)字位置
使用標(biāo)記軟件labelImg進(jìn)行，生成就能保存數(shù)字在圖像中的坐標(biāo)和對(duì)象名稱的xml標(biāo)簽文件：
?
三、yolov4-tiny機(jī)器學(xué)習(xí)訓(xùn)練
訓(xùn)練框架采用輕量的yolov4-tiny，使用win10電腦進(jìn)行訓(xùn)練，將OpenCV攝像頭采集到的數(shù)字圖像新建文件夾JPEGImages保存，然后將標(biāo)記好數(shù)字的xml標(biāo)簽文件新建一個(gè)Annotations文件夾保存。
然后編寫Python腳本txt.py隨機(jī)選擇訓(xùn)練樣本圖片和對(duì)照樣本圖片，代碼如下：
import os
import random
trainval_percent = 0.5
train_percent = 0.5
xmlfilepath = 'VOCdevkit/VOC2021/Annotations'
txtsavepath = 'VOCdevkit/VOC2021/ImageSets/Main'
total_xml = os.listdir(xmlfilepath)
num = len(total_xml)
list = range(num)
tv = int(num * trainval_percent)
ptr = int(tv * train_percent)
trainval = random.sample(list, tv)
train = random.sample(trainval, ptr)
ftrainval = open(txtsavepath + '/trainval.txt', 'w')
ftest = open(txtsavepath + '/test.txt', 'w')
ftrain = open(txtsavepath + '/train.txt', 'w')
fval = open(txtsavepath + '/val.txt', 'w')
for i in list:
name = total_xml[i][:-4] + '\n'
if i in trainval:
ftrainval.write(name)
if i in train:
ftrain.write(name)
else:
fval.write(name)
else:
ftest.write(name)
ftrainval.close()
ftrain.close()
fval.close()
ftest.close()
?然后編寫Python腳本voc_label.py將標(biāo)簽文件中的名字和坐標(biāo)位置提取出來(lái)生成一個(gè)txt文件，代碼如下：
import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir, getcwd
from os.path import join
sets=[('2021', 'train'), ('2021', 'val'), ('2021', 'test')]
classes = ["one","two","three","four","five","six","seven","eight"]
def convert(size, box):
dw = 1./(size[0])
dh = 1./(size[1])
x = (box[0] + box[1])/2.0 - 1
y = (box[2] + box[3])/2.0 - 1
w = box[1] - box[0]
h = box[3] - box[2]
x = x*dw
w = w*dw
y = y*dh
h = h*dh
return (x,y,w,h)
def convert_annotation(year, image_id):
in_file = open('VOCdevkit/VOC%s/Annotations/%s.xml'%(year, image_id))
out_file = open('VOCdevkit/VOC%s/labels/%s.txt'%(year, image_id), 'w')
tree=ET.parse(in_file)
root = tree.getroot()
size = root.find('size')
w = int(size.find('width').text)
h = int(size.find('height').text)
for obj in root.iter('object'):
difficult = obj.find('difficult').text
cls = obj.find('name').text
if cls not in classes or int(difficult)==1:
continue
cls_id = classes.index(cls)
xmlbox = obj.find('bndbox')
b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text), float(xmlbox.find('ymax').text))
bb = convert((w,h), b)
out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb])+'\n')
wd = getcwd()
for year, image_set in sets:
if not os.path.exists('VOCdevkit/VOC%s/labels/'%(year)):
os.makedirs('VOCdevkit/VOC%s/labels/'%(year))
image_ids = open('VOCdevkit/VOC%s/ImageSets/Main/%s.txt'%(year, image_set)).read().strip().split()
list_file = open('%s_%s.txt'%(year, image_set), 'w')
for image_id in image_ids:
list_file.write('%s/VOCdevkit/VOC%s/JPEGImages/%s.jpg\n'%(wd, year, image_id))
convert_annotation(year, image_id)
list_file.close()
os.system("cat 2021_train.txt 2021_val.txt 2021_train.txt 2021_val.txt > train.txt")
os.system("cat 2021_train.txt 2021_val.txt 2021_test.txt 2021_train.txt 2021_val.txt > train.all.txt")
新建coco.name文件記錄識(shí)別類別名字：
新建coco.data文件記錄類別數(shù)量，訓(xùn)練圖像文件位置，測(cè)試圖像文件位置，類別名字對(duì)象文件位置，生成權(quán)重文件保存位置：
新建yolov4-tiny-train文件記錄訓(xùn)練圖像大小，訓(xùn)練迭代最大次數(shù)等：
?
?一切準(zhǔn)備就緒后使用darknet.exe程序輸入指令：darknet detector train object/coco.data object/yolov4-tiny-train.cfg yolov4-conv.29開(kāi)始訓(xùn)練。
?
?迭代6500次后loss損失度已經(jīng)降低到0.1左右后停止訓(xùn)練，得到權(quán)重文件：
我會(huì)把訓(xùn)練的圖片數(shù)據(jù)集還有生成的權(quán)重文件放到最后的總結(jié)部分，需要的同學(xué)可以自行下載哈。
四、jetson nano識(shí)別數(shù)字
編寫Python腳本testimage.py加載權(quán)重文件識(shí)別數(shù)字，并將識(shí)別到的結(jié)果進(jìn)行處理后將結(jié)果通過(guò)串口反饋給STM32F4單片機(jī)，識(shí)別流程：將第一次識(shí)別的數(shù)字作為目標(biāo)變量target保存，在第二次識(shí)別到這個(gè)數(shù)字判斷數(shù)字所在位置（假如在左側(cè)），如果是近端或中端則通過(guò)串口發(fā)送字符i或r給單片機(jī)，i代表left在左側(cè)，r代表right在右側(cè)，如果是遠(yuǎn)端則等待第三次識(shí)別target數(shù)字然后通過(guò)串口發(fā)送字符i或r給單片機(jī)。代碼如下：
def camera_thread():
network, class_names, class_colors = darknet.load_network('object2/yolov4-tiny-test.cfg', 'object2/coco.data',
'object2/backup/yolov4-tiny4.weights', batch_size=1)
cap = cv2.VideoCapture(0)
width = 640
height = 480
cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
w = 480
crop_w_start = (width - w) // 2
crop_h_start = (height - w) // 2
while True:
numbers = ['zero', 'one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight']
number = [['number', 0], ['number', 0], ['number', 0], ['number', 0], ['number', 0], ['number', 0],
['number', 0], ['number', 0]]
x = [0.0, 0.0, 0.0, 0.0]
y = [0.0, 0.0, 0.0, 0.0]
worth = 0.0
s = 1
flag = 0
counts = 0
print("start")
while True:
input = cv2.waitKey(1) & 0xFF
if input == ord('x'):
print("stop")
break
ret, frame = cap.read()
# frame = frame[crop_h_start:crop_h_start+w+60,crop_w_start:crop_w_start+w+60]
image, detections = image_detection(frame, network, class_names, class_colors, 0.25)
# darknet.print_detections(detections, True)
cv2.imshow('Inference', image)
counts = len(detections)
# print("檢測(cè)到",counts,"個(gè)數(shù)字")
for i in range(0, counts, 1):
# print(detections[i][0], detections[i][1], detections[i][2][0])
# str,str,float,float,float,float
# print(type(detections[i][0]),type(detections[i][1]),type(detections[i][2][0]),type(detections[i][2][1]),type(detections[i][2][2]),type(detections[i][2][3]))
number[i][0] = detections[i][0]
number[i][1] = int(detections[i][2][0])
# print(numbers.index(number[i][0]))
s = int(s + (detections[i][2][2] * detections[i][2][3]))
worth = worth + float(detections[i][1])
if s > counts and counts != 0:
# serial_port.write(('d'+(str(s/counts))).encode())
# print('distence:',s/counts)
s = 0
else:
s = 0
if worth > counts and counts >= 0:
if worth / counts >= 70:
number_sort = sorted(number, key=lambda x: x[1])
# serial_port.write(('n'+(str(counts))).encode())
for i in range(0, counts, 1):
print(numbers.index(number_sort[7 - i][0]), number_sort[7 - i][1])
# serial_port.write((str(numbers.index(number_sort[7-i][0]))).encode())
if flag == 1 and target == numbers.index(number_sort[7 - i][0]):
if (counts == 4 and (i == 0 or i == 1)) or (counts == 2 and (i == 0)):
GPIO.output(13, GPIO.LOW)
print("right")
serial_port.write(str(target).encode())
serial_port.write('r'.encode())
while True:
if serial_port.inWaiting() > 0:
data = serial_port.read()
# print(data)
if data == 'o':
flag = 2
break
input = cv2.waitKey(1) & 0xFF
if input == ord('q'):
break
# time.sleep(5)
elif (counts == 4 and (i == 2 or i == 3)) or (counts == 2 and (i == 1)):
GPIO.output(15, GPIO.LOW)
print("left")
serial_port.write('l'.encode())
while True:
if serial_port.inWaiting() > 0:
data = serial_port.read()
# print(data)
if data == 'o':
flag = 2
elif data == 'k':
flag = 0
break
input = cv2.waitKey(1) & 0xFF
if input == ord('q'):
break
print(i)
elif flag == 2 and target == numbers.index(number_sort[7 - i][0]):
if (counts == 4 and (i == 0 or i == 1)) or (counts == 2 and (i == 0)):
GPIO.output(13, GPIO.LOW)
print("right")
serial_port.write('r'.encode())
while True:
if serial_port.inWaiting() > 0:
data = serial_port.read()
# print(data)
if data == 'o':
flag = 3
elif data == 'k':
flag = 0
break
input = cv2.waitKey(1) & 0xFF
if input == ord('q'):
break
# time.sleep(5)
elif (counts == 4 and (i == 2 or i == 3)) or (counts == 2 and (i == 1)):
GPIO.output(15, GPIO.LOW)
print("left")
serial_port.write('l'.encode())
while True:
if serial_port.inWaiting() > 0:
data = serial_port.read()
# print(data)
if data == 'o':
flag = 3
break
input = cv2.waitKey(1) & 0xFF
if input == ord('q'):
break
print(i)
elif flag == 0 and counts == 1:
target = numbers.index(number[0][0])
print("target:", target)
if target == 1:
serial_port.write('1'.encode())
while True:
if serial_port.inWaiting() > 0:
data = serial_port.read()
# print(data)
if data == 'o':
flag = 3
break
input = cv2.waitKey(1) & 0xFF
if input == ord('q'):
break
elif target == 2:
serial_port.write('2'.encode())
while True:
if serial_port.inWaiting() > 0:
data = serial_port.read()
# print(data)
if data == 'o':
flag = 3
break
input = cv2.waitKey(1) & 0xFF
if input == ord('q'):
break
else:
serial_port.write('t'.encode())
serial_port.write('t'.encode())
serial_port.write('t'.encode())
# time.sleep(1)p
flag = 1
# serial_port.write('o'.encode())
print("counts:", counts)
number = [['number', 0], ['number', 0], ['number', 0], ['number', 0], ['number', 0], ['number', 0],
['number', 0], ['number', 0]]
worth = 0
else:
worth = 0
k = cv2.waitKey(1)
time.sleep(0.05)五、識(shí)別效果
單個(gè)數(shù)字識(shí)別
?
?
?
兩個(gè)數(shù)字識(shí)別
?
?四個(gè)數(shù)字識(shí)別
?
命令行結(jié)果顯示
?
小車OLED屏幕顯示
?
總結(jié)+數(shù)據(jù)集權(quán)重文件+小車程序
總結(jié)起來(lái)四天三夜做出來(lái)兩輛車還是很困難的，更別說(shuō)還需要搭建場(chǎng)地還有調(diào)試，中途還有很多問(wèn)題，整個(gè)團(tuán)隊(duì)都是每天睡覺(jué)不到4小時(shí)，還好最后的結(jié)果還算令人滿意，題目一開(kāi)始看著不難，但是做的過(guò)程中就有很多莫名其妙的問(wèn)題，在有限的時(shí)間內(nèi)解決也是考驗(yàn)選手的一點(diǎn)。
最后附上數(shù)字識(shí)別的數(shù)據(jù)集還有權(quán)重文件：
數(shù)字識(shí)別訓(xùn)練數(shù)據(jù)集和權(quán)重文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-621623.html
電賽F題送藥小車程序.zip文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-621623.html
到了這里，關(guān)于21年電賽F 題jetson nano+32F4識(shí)別數(shù)字（附源碼和數(shù)據(jù)集）的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
