在點(diǎn)云的3D感知算法中，常用
RandomFlip3D和GlobalRotScaleTrans的數(shù)據(jù)增強(qiáng)方式，這兩個(gè)可以有效地增強(qiáng)模型的魯棒性，提升模型的性能。
transforms=[
dict(
type='RandomFlip3D',
sync_2d=False,
flip_ratio_bev_horizontal=0.5,
flip_ratio_bev_vertical=0.5),
dict(
type='GlobalRotScaleTrans',
rot_range=[-0.78539816, 0.78539816],
scale_ratio_range=[0.95, 1.05]),
而本文的出發(fā)點(diǎn)在于當(dāng)我們同時(shí)對(duì)相同的
data使用了多種不同的隨機(jī)增強(qiáng)方式時(shí)，當(dāng)我們需要在模型中將不同增強(qiáng)方法下的Feature map或者GT 3D Box進(jìn)行對(duì)齊時(shí)，就需要涉及到逆變換。
尤其適用于半監(jiān)督等同時(shí)有l(wèi)abel和unlabel data的代碼。
簡(jiǎn)單粗暴，我們直接上代碼，注釋直接寫在代碼中：（代碼可以直接copy使用）
def align_aug_dataV2(self, pts_feats, # 可以是feature map, shape為 [B,C,H,W]
gt_bboxes_3d,# Nuscenes默認(rèn)的LidarBox格式, shape為 [B,N,x], x=7或9,有無(wú)速度的區(qū)別
img_metas,
return_tensor=True, # 如果想返回的pts_feats是tensor形式，默認(rèn)True
interploate_mode='bilinear'): # grid_sample的插值方式，如果是feat map，建議bilinear
'''
目前的box.tensor的格式為[x, y, z, x_size, y_size, z_size, yaw, vx, vy]，shape為[N, 9]
所以對(duì)gt_box做flip需要分別處理x,y,yaw和vx,vy
'''
# feature_map: [B,C,H,W]
def horizontal_flip(feature_map):# 水平翻轉(zhuǎn)
return torch.flip(feature_map, [2])
def vertical_flip(feature_map):# 垂直翻轉(zhuǎn)
return torch.flip(feature_map, [1])
def box_flip(boxes, bev_direction='horizontal'):
assert bev_direction in ('horizontal', 'vertical')
if bev_direction == 'horizontal':
boxes.tensor[:, 1] = -boxes.tensor[:, 1] # y
boxes.tensor[:, 6] = -boxes.tensor[:, 6] + np.pi
boxes.tensor[:, 7] = boxes.tensor[:, 7] # vx
boxes.tensor[:, 8] = -boxes.tensor[:, 8] # vy
elif bev_direction == 'vertical':
boxes.tensor[:, 0] = -boxes.tensor[:, 0] # x
boxes.tensor[:, 6] = -boxes.tensor[:, 6]
boxes.tensor[:, 7] = -boxes.tensor[:, 7] # vx
boxes.tensor[:, 8] = boxes.tensor[:, 8] # vy
return boxes
aligned_pts_feats = []
aligned_gt_bboxes_3d = deepcopy(gt_bboxes_3d)
for idx, (pts_feat, boxes_3d, img_meta) in enumerate(
zip(pts_feats, aligned_gt_bboxes_3d, img_metas)):
# 數(shù)據(jù)增強(qiáng)Pipline處理時(shí)aug順序： RandomFlip3D, GlobalRotScaleTrans(rot, scale, trans)
# 逆變換align順序需要倒過來(lái)
# ----------------------------------------------------
pts_feat = pts_feat.unsqueeze(0)
tgt_size = pts_feat.shape
dev = pts_feat.device
# GlobalRotScaleTrans
if 'pcd_trans' in img_meta:
if not (img_meta['pcd_trans'] == 0.).all():
# assert (img_meta['pcd_trans'] == 0.).all(), \
# "Translation is not allowed in `GlobalRotScaleTrans`, [0.,0.,0.] only."
if not (img_meta['pcd_trans'] == 0.).all():
# 1. feat map
Trans = torch.zeros_like(img_meta['pcd_rotation'].T) # [3,3]
Trans[0,0], Trans[1,1] = 1, 1
# 需要?dú)w一化`pcd_trans`,默認(rèn)的`pcd_trans`為pc_range范圍的參數(shù)
w, h = self.bbox_coder.pc_range
w, h = abs(w*2), abs(h*2)
Trans[0,2], Trans[1,2] = img_meta['pcd_trans'][0]/w, \
img_meta['pcd_trans'][1]/h
Trans = Trans[:2,:].unsqueeze(0) # [B,2,3]
grid = F.affine_grid(Trans, tgt_size).to(dev) # # 仿射變換矩陣
pts_feat = F.grid_sample(pts_feat, # 輸入tensor，shape為[B,C,W,H]
grid, # 上一步輸出的gird,shape為[B,C,W,H]
mode=interploate_mode)
# 2. gt_boxes
boxes_3d.translate(-img_meta['pcd_trans'])
if 'pcd_scale_factor' in img_meta:
if img_meta['pcd_scale_factor'] != 1.:
# 1. feat map
Scl = torch.zeros_like(img_meta['pcd_rotation'].T) # [3,3]
Scl[0,0], Scl[1,1] = 1/img_meta['pcd_scale_factor'], 1/img_meta['pcd_scale_factor']
Scl = Scl[:2,:].unsqueeze(0) # [B,2,3]
grid = F.affine_grid(Scl, tgt_size).to(dev) # # 仿射變換矩陣
pts_feat = F.grid_sample(pts_feat, # 輸入tensor，shape為[B,C,W,H]
grid, # 上一步輸出的gird,shape為[B,C,W,H]
mode=interploate_mode)
# 2. gt_boxes
boxes_3d.scale(1/img_meta['pcd_scale_factor'])
if 'pcd_rotation' in img_meta:
if img_meta['pcd_rotation'][0,0] != 1.:
# 1. feat map
Rot = img_meta['pcd_rotation'].T # [3,3]
Rot = Rot[:2,:].unsqueeze(0) # [B,2,3]
grid = F.affine_grid(Rot, tgt_size).to(dev) # # 仿射變換矩陣
pts_feat = F.grid_sample(pts_feat, # 輸入tensor，shape為[B,C,W,H]
grid, # 上一步輸出的gird,shape為[B,C,W,H]
mode=interploate_mode)
# 2. gt_boxes
boxes_3d.rotate(img_meta['pcd_rotation'].T)
pts_feat = pts_feat[0]
# ----------------------------------------------------
# RandomFlip3D
if 'pcd_vertical_flip' in img_meta:
if img_meta['pcd_vertical_flip']:
# 1. feat map
pts_feat = vertical_flip(pts_feat)
# 2. gt_boxes
boxes_3d = box_flip(boxes_3d, bev_direction='vertical')
if 'pcd_horizontal_flip' in img_meta:
if img_meta['pcd_horizontal_flip']:
# 1. feat map
pts_feat = horizontal_flip(pts_feat)
# 2. gt_boxes
# boxes_3d.flip(bev_direction='horizontal')
boxes_3d = box_flip(boxes_3d, bev_direction='horizontal')
aligned_pts_feats.append(pts_feat)
aligned_gt_bboxes_3d[idx] = boxes_3d
if return_tensor:
aligned_pts_feats = torch.stack(aligned_pts_feats)
return aligned_pts_feats, aligned_gt_bboxes_3d
有幾點(diǎn)需要注意：
- feat map在經(jīng)過Flip之后也不全是一一對(duì)應(yīng)，因?yàn)樵诰矸e過程中，翻轉(zhuǎn)后對(duì)應(yīng)的位置不同，所以feat在經(jīng)過翻轉(zhuǎn)變換之后略有不同是正?，F(xiàn)象；
- yaw角差360°（2*3.14）是正常情況，因?yàn)檗D(zhuǎn)換成cos和sin后數(shù)值相同。
測(cè)試效果
測(cè)試一下把有aug的feat map和GT都 對(duì)齊到?jīng)]有aug的情況下：
# 測(cè)試代碼： (1-3應(yīng)當(dāng)相同)
# 1.aligned_aug aligned_gt_bboxes_3d[0].tensor[0]
# 2.aligned_aug aligned_gt_bboxes_3d[4].tensor[0]
# 3.無(wú)aug gt_bboxes_3d[4].tensor[0]
# 4.有aug gt_bboxes_3d[0].tensor[0]
-
加上
RandomFlip3D：（標(biāo)紅的為有aug的原始值）
-
加上
RandomFlip3D和Scale：
-
加上
RandomFlip3D和Rotate：
-
只有
Translation的結(jié)果：
-
加上
RandomFlip3D和GlobalRotScaleTrans所有aug的對(duì)齊結(jié)果：
文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-472501.html
參考文章：文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-472501.html
- Pytorch——實(shí)現(xiàn)Tensor矩陣的任意角度旋轉(zhuǎn)、平移操作
- Pytorch中實(shí)現(xiàn)矩陣的的仿射變換，平移、旋轉(zhuǎn)、放縮(affine_grid)
到了這里，關(guān)于MMDet3D——數(shù)據(jù)增強(qiáng)Pipline‘GlobalRotScaleTrans‘和‘RandomFlip3D‘的Pytorch逆變換實(shí)現(xiàn)的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
