前言

論文原文：https://arxiv.org/pdf/2303.15334.pdf
論文代碼：https://github.com/ifzhang/ByteTrack-V2

Title：ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box
Authors：Yifu Zhang, Xinggang Wang, Xiaoqing Y e, Wei Zhang, Jincheng Lu, Xiao T an, Errui Ding, Peize Sun, Jingdong Wang

在此僅做翻譯（經(jīng)過(guò)個(gè)人調(diào)整，有基礎(chǔ)的話應(yīng)該不難理解），有時(shí)間會(huì)有詳細(xì)精讀筆記。

Abstract

多目標(biāo)跟蹤(MOT)旨在估計(jì)視頻幀內(nèi)物體的邊界框和身份。檢測(cè)框是二維和三維MOT的基礎(chǔ)。檢測(cè)分?jǐn)?shù)不可避免的變化會(huì)導(dǎo)致跟蹤后的目標(biāo)缺失。我們提出了一種分層的數(shù)據(jù)關(guān)聯(lián)策略來(lái)挖掘低分檢測(cè)框中的真實(shí)目標(biāo)，緩解了目標(biāo)缺失和軌跡碎片化的問(wèn)題。簡(jiǎn)單而通用的數(shù)據(jù)關(guān)聯(lián)策略在2D和3D設(shè)置下都顯示了有效性。在3D場(chǎng)景中，跟蹤器更容易預(yù)測(cè)世界坐標(biāo)中的物體速度。我們提出了一種補(bǔ)充的運(yùn)動(dòng)預(yù)測(cè)策略，將檢測(cè)到的速度與卡爾曼濾波器結(jié)合起來(lái)，以解決突然運(yùn)動(dòng)和短期消失的問(wèn)題。ByteT rackV2在nuScenes 3D MOT排行榜上領(lǐng)先于相機(jī)(56.4% AMOT A)和激光雷達(dá)(70.1% AMOT A)模式。此外，它是非參數(shù)的，可以與各種檢測(cè)器集成，使其在實(shí)際應(yīng)用中具有吸引力。源代碼發(fā)布于https://github.com/ifzhang/ByteTrack-V2。

1 INTRODUCTION

這項(xiàng)工作解決了2D和3D多目標(biāo)跟蹤(MOT)問(wèn)題。無(wú)論是2D還是3D多目標(biāo)跟蹤都是計(jì)算機(jī)視覺(jué)領(lǐng)域長(zhǎng)期存在的任務(wù)。MOT的目標(biāo)是在2D圖像平面或3D世界坐標(biāo)中判斷感興趣的物體的軌跡。成功解決這一問(wèn)題將有利于自動(dòng)駕駛和智能交通等許多應(yīng)用。

2D多目標(biāo)跟蹤和3D多目標(biāo)跟蹤本質(zhì)上是交織在一起的。這兩項(xiàng)任務(wù)都必須定位對(duì)象并獲得跨不同幀的對(duì)象對(duì)應(yīng)關(guān)系。然而，由于輸入數(shù)據(jù)來(lái)自不同的模態(tài)，來(lái)自不同領(lǐng)域的研究人員已經(jīng)獨(dú)立地解決了這些問(wèn)題。2D-MOT是在圖像平面上進(jìn)行的，圖像信息是物體對(duì)應(yīng)關(guān)系的重要線索?；谕庥^的跟蹤器從圖像中提取物體外觀特征，然后計(jì)算特征距離作為對(duì)應(yīng)關(guān)系。3D-MOT通常在包含深度信息的世界系統(tǒng)中進(jìn)行。通過(guò)空間相似性(如3D Intersection over Union (IoU)，或點(diǎn)距離)更容易區(qū)分不同的對(duì)象。二維MOT和三維MOT可視化如圖1所示。

PS：這個(gè)圖2和ByteTrackV1里面的圖一模一樣

2 RELATED WORK

在本節(jié)中，我們簡(jiǎn)要回顧了與我們的主題相關(guān)的現(xiàn)有工作，包括 2D 目標(biāo)檢測(cè)、3D 目標(biāo)檢測(cè)、2D 多目標(biāo)跟蹤和 3D 多目標(biāo)跟蹤。我們還討論了這些任務(wù)之間的關(guān)系。

2.1 2D Object Detection

2D目標(biāo)檢測(cè)旨在從圖像輸入中預(yù)測(cè)邊界框。它是計(jì)算機(jī)視覺(jué)中最活躍的課題之一，是多目標(biāo)跟蹤的基礎(chǔ)。隨著目標(biāo)檢測(cè)的快速發(fā)展，越來(lái)越多的多目標(biāo)跟蹤方法開(kāi)始利用更強(qiáng)大的檢測(cè)器以獲得更高的跟蹤性能。單階段目標(biāo)檢測(cè)器 RetinaNet 開(kāi)始被多種方法采用，例如 Retinatrack、Chained-tracker。anchor-free檢測(cè)器 CenterNet 是大多數(shù)方法Fairmot、CenterTrack、TraDeS、SOTMOT、GSDT、Learning to track with
object permanence、CorrTracker 最常用的檢測(cè)器，因?yàn)樗?jiǎn)單高效. YOLO系列檢測(cè)器YOLOV3、YOLOV4、YOLOX也被大量方法采用，如Towards real-time
multi-object tracking、CSTracker、CSTrackerV2、Transmot、Unicorn、Robust multi-object tracking by marginal inference，來(lái)實(shí)現(xiàn)準(zhǔn)確性和速度的平衡。最近，基于transformer的檢測(cè)器DETR、Deformable detr、Conditional detr for fast training convergence 被一些跟蹤器Transtrack、TrackFormer、MOTR 用于其優(yōu)雅的端到端框架。我們采用 YOLOX 作為我們的高效二維物體檢測(cè)器。

MOT17 數(shù)據(jù)集提供了 DPM、Faster R-CNN 和 SDP 等流行檢測(cè)器獲得的檢測(cè)結(jié)果。大量的多目標(biāo)跟蹤方法[49]、[50]、[51]、[52]、[53]、[54]、[55]側(cè)重于基于這些給定的檢測(cè)結(jié)果來(lái)提高跟蹤性能。我們還在這個(gè)公共檢測(cè)設(shè)置下評(píng)估我們的跟蹤算法。

[49] J. Xu, Y . Cao, Z. Zhang, and H. Hu, “Spatial-temporal relation networks for multi-object tracking,” in ICCV, 2019, pp. 3988–3998.
[50] P . Chu and H. Ling, “Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking,” in ICCV, 2019, pp. 6172–6181.
[51] P . Bergmann, T. Meinhardt, and L. Leal-Taixe, “Tracking without bells and whistles,” in ICCV, 2019, pp. 941–951.
[52] L. Chen, H. Ai, Z. Zhuang, and C. Shang, “Real-time multiple people tracking with deeply learned candidate selection and person reidentification,” in ICME. IEEE, 2018, pp. 1–6.
[53] J. Zhu, H. Yang, N. Liu, M. Kim, W. Zhang, and M.-H. Yang, “Online multi-object tracking with dual matching attention networks,” in Proceedings of the ECCV (ECCV), 2018, pp. 366–382.
[54] G. Brasó and L. Leal-Taixé, “Learning a neural solver for multiple object tracking,” in CVPR, 2020, pp. 6247–6257.
[55] A. Hornakova, R. Henschel, B. Rosenhahn, and P . Swoboda, “Lifted disjoint paths with application in multiple object tracking,” in International Conference on Machine Learning. PMLR, 2020, pp. 4364–4375.

2.2 3D Object Detection

3D 目標(biāo)檢測(cè)旨在從圖像或 LiDAR 輸入中預(yù)測(cè)三維旋轉(zhuǎn)邊界框。它是 3D 多目標(biāo)跟蹤不可或缺的組成部分，因?yàn)轭A(yù)測(cè)的 3D 邊界框的質(zhì)量在跟蹤性能中起著重要作用。
基于 LiDAR 的 3D 目標(biāo)檢測(cè)方法 [11]、[56]、[57]、[58]、[59]、[60]、[61] 取得了令人印象深刻的性能，因?yàn)閺?LiDAR 傳感器檢索的點(diǎn)云包含準(zhǔn)確的 3D 結(jié)構(gòu)信息.然而，激光雷達(dá)的高成本限制了其廣泛應(yīng)用。

[11] T. Yin, X. Zhou, and P . Krahenbuhl, “Center-based 3d object detection and tracking,” in CVPR, 2021, pp. 11 784–11 793.
[56] Y . Zhou and O. Tuzel, “V oxelnet: End-to-end learning for point cloud based 3d object detection,” in CVPR, 2018, pp. 4490–4499.
[57] Y . Yan, Y . Mao, and B. Li, “Second: Sparsely embedded convolutional detection,” Sensors, vol. 18, no. 10, p. 3337, 2018.
[58] A. H. Lang, S. V ora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in CVPR, 2019, pp. 12 697–12 705.
[59] S. Shi, X. Wang, and H. Li, “Pointrcnn: 3d object proposal generation and detection from point cloud,” in CVPR, 2019, pp. 770–779.
[60] L. Du, X. Ye, X. Tan, E. Johns, B. Chen, E. Ding, X. Xue, and J. Feng, “Ago-net: Association-guided 3d point cloud object detection network,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[61] J. Liu, Y . Chen, X. Ye, Z. Tian, X. Tan, and X. Qi, “Spatial pruned sparse convolution for efficient 3d object detection,” arXiv preprint arXiv:2209.14201, 2022.

或者，基于相機(jī)的方法的最新進(jìn)展使其低成本且移動(dòng)性廣泛可用，因此基于相機(jī)的方法由于其低成本和豐富的上下文信息而受到越來(lái)越多的關(guān)注。由于缺乏準(zhǔn)確的深度，單目 3D 對(duì)象檢測(cè)方法 [62]、[63]、[64]、[65]、[66]、[67] 直接通過(guò)深度神經(jīng)網(wǎng)絡(luò)推斷幾何知識(shí)或逐像素研究深度估計(jì)分布將圖像轉(zhuǎn)換為偽 LiDAR 點(diǎn) [68]、[69]、[70]、[71]、[72]、[73]。

[62] X. Chen, K. Kundu, Y . Zhu, A. G. Berneshawi, H. Ma, S. Fidler, and R. Urtasun, “3d object proposals for accurate object class detection,” Advances in neural information processing systems, vol. 28, 2015.
[63] X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun, “Monocular 3d object detection for autonomous driving,” in CVPR, 2016, pp. 2147–2156.
[64] T. Wang, X. Zhu, J. Pang, and D. Lin, “Fcos3d: Fully convolutional one-stage monocular 3d object detection,” in ICCV, 2021, pp. 913–922.
[65] Z. Zou, X. Ye, L. Du, X. Cheng, X. Tan, L. Zhang, J. Feng, X. Xue, and E. Ding, “The devil is in the task: Exploiting reciprocal appearancelocalization features for monocular 3d object detection,” in ICCV, 2021, pp. 2713–2722.
[66] Y . Zhang, J. Lu, and J. Zhou, “Objects are different: Flexible monocular 3d object detection,” in CVPR, 2021, pp. 3289–3298.
[67] X. Ye, M. Shu, H. Li, Y . Shi, Y . Li, G. Wang, X. Tan, and E. Ding, “Rope3d: The roadside perception dataset for autonomous driving and monocular 3d object detection task,” in CVPR, 2022, pp. 21 341–21 350.
[68] C. Reading, A. Harakeh, J. Chae, and S. L. Waslander, “Categorical depth distribution network for monocular 3d object detection,” in CVPR, 2021, pp. 8555–8564.
[69] X. Ye, L. Du, Y . Shi, Y . Li, X. Tan, J. Feng, E. Ding, and S. Wen, “Monocular 3d object detection via feature domain adaptation,” in ECCV. Springer, 2020, pp. 17–34.
[70] Y . Wang, W.-L. Chao, D. Garg, B. Hariharan, M. Campbell, and K. Q. Weinberger, “Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving,” in CVPR, 2019, pp.8445–8453.
[71] X. Weng and K. Kitani, “Monocular 3d object detection with pseudolidar point cloud,” in CVPRW, 2019, pp. 0–0.
[72] Y . Y ou, Y . Wang, W.-L. Chao, D. Garg, G. Pleiss, B. Hariharan, M. Campbell, and K. Q. Weinberger, “Pseudo-lidar++: Accurate depth for 3d object detection in autonomous driving,” arXiv preprint arXiv:1906.06310, 2019.
[73] J. M. U. Vianney, S. Aich, and B. Liu, “Refinedmpl: Refined monocular pseudolidar for 3d object detection in autonomous driving,” arXiv preprint arXiv:1911.09712, 2019.

多攝像頭 3D 對(duì)象檢測(cè) [74]、[75]、[76]、[77]、[78]、[79] 通過(guò)學(xué)習(xí)鳥(niǎo)瞰圖 (BEV) 中的強(qiáng)大表示，正在成為趨勢(shì)并引起廣泛關(guān)注，這由于其統(tǒng)一表示和對(duì)未來(lái)預(yù)測(cè)和規(guī)劃等下游任務(wù)的輕松適應(yīng)，因此非常簡(jiǎn)單。因此，以視覺(jué)為中心的多視圖 BEV 感知方法顯著縮小了基于相機(jī)和基于 LiDAR 的方法之間的性能差距。

[74] J. Philion and S. Fidler, “Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,” in ECCV.Springer, 2020, pp. 194–210.
[75] Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Y u, and J. Dai, “Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,” arXiv preprint arXiv:2203.17270, 2022.
[76] Y . Wang, V . C. Guizilini, T. Zhang, Y . Wang, H. Zhao, and J. Solomon, “Detr3d: 3d object detection from multi-view images via 3d-to-2d queries,” in Conference on Robot Learning. PMLR, 2022, pp. 180–191.
[77] Y . Liu, T. Wang, X. Zhang, and J. Sun, “Petr: Position embedding transformation for multi-view 3d object detection,” arXiv preprint arXiv:2203.05625, 2022.
[78] Y . Li, Z. Ge, G. Y u, J. Yang, Z. Wang, Y . Shi, J. Sun, and Z. Li, “Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,” arXiv preprint arXiv:2206.10092, 2022.
[79] K. Xiong, S. Gong, X. Ye, X. Tan, J. Wan, E. Ding, J. Wang, and X. Bai, “Cape: Camera view position embedding for multi-view 3d object detection,” arXiv preprint arXiv:2303.10209, 2023.

基于 LiDAR 的檢測(cè)器是 3D MOT 的熱門(mén)選擇。PointRCNN和 CenterPoint 因其簡(jiǎn)單性和有效性而被許多 3D MOT 方法 [7]、[11]、[80]、[81]、[82] 采用。最近，基于圖像的 3D 對(duì)象檢測(cè)器 [76]、[78]、[83] 開(kāi)始被一些 3D 跟蹤器采用，例如 [84]、[85]、[86]，因?yàn)閳D像信息可以為跟蹤提供外觀線索。我們的跟蹤框架與模式無(wú)關(guān)，因此可以輕松地與各種 3D 對(duì)象檢測(cè)器結(jié)合使用。

[7] X. Weng, J. Wang, D. Held, and K. Kitani, “3d multi-object tracking: A baseline and new evaluation metrics,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 10 359–10 366.
[11] T. Yin, X. Zhou, and P . Krahenbuhl, “Center-based 3d object detection and tracking,” in CVPR, 2021, pp. 11 784–11 793.
[80] N. Benbarka, J. Schr?der, and A. Zell, “Score refinement for confidencebased 3d multi-object tracking,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021, pp.8083–8090.
[81] Z. Pang, Z. Li, and N. Wang, “Simpletrack: Understanding and rethinking 3d multi-object tracking,” arXiv preprint arXiv:2111.09621, 2021.
[82] Q. Wang, Y . Chen, Z. Pang, N. Wang, and Z. Zhang, “Immortal tracker: Tracklet never dies,” arXiv preprint arXiv:2111.13672, 2021.
[83] A. Kundu, Y . Li, and J. M. Rehg, “3d-rcnn: Instance-level 3d object reconstruction via render-and-compare,” in CVPR, 2018, pp. 3559– 3568.
[84] H.-N. Hu, Y .-H. Yang, T. Fischer, T. Darrell, F. Y u, and M. Sun, “Monocular quasi-dense 3d object tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[85] T. Zhang, X. Chen, Y . Wang, Y . Wang, and H. Zhao, “Mutr3d: A multicamera tracking framework via 3d-to-2d queries,” in CVPR, 2022, pp.4537–4546.
[86] J. Yang, E. Y u, Z. Li, X. Li, and W. Tao, “Quality matters: Embracing quality clues for robust 3d multi-object tracking,” arXiv preprint arXiv:2208.10976, 2022.

2.3 2D Multi-Object Tracking

數(shù)據(jù)關(guān)聯(lián)是多目標(biāo)跟蹤的核心，它首先計(jì)算tracklets和檢測(cè)框之間的相似度，并根據(jù)相似度利用不同的策略來(lái)匹配它們。
在 2D MOT 中，圖像信息起著計(jì)算相似性的基本作用。位置、運(yùn)動(dòng)和外觀是數(shù)據(jù)關(guān)聯(lián)的有用線索。 SORT以非常簡(jiǎn)單的方式結(jié)合了位置和運(yùn)動(dòng)提示。它首先采用卡爾曼濾波器來(lái)預(yù)測(cè)新幀中軌跡的位置，然后計(jì)算檢測(cè)框與預(yù)測(cè)框之間的 IoU 作為相似度。其他方法 [26]、[27]、[43]、[87] 設(shè)計(jì)網(wǎng)絡(luò)來(lái)學(xué)習(xí)對(duì)象運(yùn)動(dòng)，并在大相機(jī)運(yùn)動(dòng)或低幀率的情況下獲得更穩(wěn)健的結(jié)果。位置和運(yùn)動(dòng)相似性在短期關(guān)聯(lián)中都是準(zhǔn)確的，而外觀相似性有助于長(zhǎng)期關(guān)聯(lián)。長(zhǎng)時(shí)間遮擋后，可以使用外觀相似度重新識(shí)別對(duì)象。外觀相似度可以通過(guò) Re-ID 特征的余弦相似度來(lái)衡量。 DeepSORT 采用獨(dú)立的 Re-ID 模型從檢測(cè)框中提取外觀特征。最近，聯(lián)合檢測(cè)和 Re-ID 模型 [8]、[9]、[23]、[34]、[88]、[89]、[90] 因其簡(jiǎn)單和高效而變得越來(lái)越流行。

[8] Y . Zhang, C. Wang, X. Wang, W. Zeng, and W. Liu, “Fairmot: On the fairness of detection and re-identification in multiple object tracking,” International Journal of Computer Vision, vol. 129, no. 11, pp. 3069– 3087, 2021.
[9] J. Pang, L. Qiu, X. Li, H. Chen, Q. Li, T. Darrell, and F. Y u, “Quasidense similarity learning for multiple object tracking,” in CVPR, 2021, pp. 164–173.
[23] Z. Lu, V . Rathod, R. V otel, and J. Huang, “Retinatrack: Online single stage joint detection and tracking,” in CVPR, 2020, pp. 14 668–14 678.
[34] Z. Wang, L. Zheng, Y . Liu, Y . Li, and S. Wang, “Towards real-time multi-object tracking,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16. Springer, 2020, pp. 107–122.
[26] X. Zhou, V . Koltun, and P . Kr?henbühl, “Tracking objects as points,” in ECCV. Springer, 2020, pp. 474–490.
[27] J. Wu, J. Cao, L. Song, Y . Wang, M. Yang, and J. Y uan, “Track to detect and segment: An online multi-object tracker,” in CVPR, 2021, pp. 12 352–12 361.
[43] P . Sun, Y . Jiang, R. Zhang, E. Xie, J. Cao, X. Hu, T. Kong, Z. Y uan, C. Wang, and P . Luo, “Transtrack: Multiple-object tracking with transformer,” arXiv preprint arXiv:2012.15460, 2020.
[87] B. Shuai, A. Berneshawi, X. Li, D. Modolo, and J. Tighe, “Siammot: Siamese multi-object tracking,” in CVPR, 2021, pp. 12 372–12 382.
[88] Y . Zhang, C. Wang, X. Wang, W. Liu, and W. Zeng, “V oxeltrack: Multi-person 3d human pose estimation and tracking in the wild,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[89] X. Zhou, T. Yin, V . Koltun, and P . Kr?henbühl, “Global tracking transformers,” in CVPR, 2022, pp. 8771–8780.
[90] Z. Xu, W. Yang, W. Zhang, X. Tan, H. Huang, and L. Huang, “Segment as points for efficient and effective online multi-object tracking and segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6424–6437, 2021.

在相似度計(jì)算之后，匹配策略為對(duì)象分配身份。這可以通過(guò)匈牙利算法或貪心分配來(lái)完成。 SORT 通過(guò)匹配一次將檢測(cè)框與軌跡匹配。 DeepSORT 提出了一種級(jí)聯(lián)匹配策略，首先將檢測(cè)框與最近的軌跡匹配，然后再與丟失的軌跡匹配。MOTDT 首先利用外觀相似性進(jìn)行匹配，然后利用 IoU 相似性來(lái)匹配未匹配的軌跡。QDTrack 通過(guò)雙向 softmax 操作將外觀相似度轉(zhuǎn)化為概率，并采用最近鄰搜索來(lái)完成匹配。注意力機(jī)制可以直接在幀之間傳播框并隱式執(zhí)行關(guān)聯(lián)。最近的方法，如 [44]、[45]、[93] 提出跟蹤查詢來(lái)預(yù)測(cè)跟蹤對(duì)象在后續(xù)幀中的位置。在不使用匈牙利算法的情況下，匹配是在注意交互過(guò)程中隱式執(zhí)行的。

[44] T. Meinhardt, A. Kirillov, L. Leal-Taixe, and C. Feichtenhofer, “Trackformer: Multi-object tracking with transformers,” in CVPR, 2022, pp.8844–8854.
[45] F. Zeng, B. Dong, T. Wang, C. Chen, X. Zhang, and Y . Wei, “Motr: End-to-end multiple-object tracking with transformer,” arXiv preprint arXiv:2105.03247, 2021.
[93] Z. Zhao, Z. Wu, Y . Zhuang, B. Li, and J. Jia, “Tracking objects as pixel-wise distributions,” in ECCV. Springer, 2022, pp. 76–94.

大多數(shù) 2D MOT 方法側(cè)重于如何設(shè)計(jì)更好的關(guān)聯(lián)策略。然而，我們認(rèn)為檢測(cè)框的使用方式?jīng)Q定了數(shù)據(jù)關(guān)聯(lián)的上限。在通過(guò)各種檢測(cè)器獲得檢測(cè)框后，大多數(shù)方法[8]、[9]、[23]、[34]只將高分框保持一個(gè)閾值，即0.5，并將這些框作為數(shù)據(jù)關(guān)聯(lián)的輸入。這是因?yàn)榈头挚虬S多損害跟蹤性能的背景。然而，我們觀察到許多被遮擋的物體可以被正確檢測(cè)到但得分較低。為了減少缺失檢測(cè)并保持軌跡的持久性，我們保留所有檢測(cè)框并關(guān)聯(lián)它們中的每一個(gè)。我們專注于如何在關(guān)聯(lián)過(guò)程中充分利用從高分到低分的檢測(cè)框。

2.4 3D Multi-Object Tracking

3D MOT 與 2D MOT 有許多共同點(diǎn)，即數(shù)據(jù)關(guān)聯(lián)。大多數(shù) 3D MOT 方法遵循檢測(cè)跟蹤范例，該范例首先檢測(cè)對(duì)象，然后跨時(shí)間關(guān)聯(lián)它們。與 2D MOT 相比，3D MOT 中使用的位置和運(yùn)動(dòng)線索更加準(zhǔn)確可靠，因?yàn)樗鼈儼疃刃畔?。例如，?dāng)兩個(gè)行人相遇時(shí)，從圖像平面獲得的 2D IoU 很大，很難通過(guò) 2D 位置區(qū)分他們。在 3D 場(chǎng)景中，可以根據(jù)深度的不同輕松將兩個(gè)行人分開(kāi)。
基于 LiDAR 的 3D 跟蹤器傾向于利用位置和運(yùn)動(dòng)線索來(lái)計(jì)算相似性。與 SORT 類(lèi)似，AB3DMOT 為 3D MOT 提供了一個(gè)簡(jiǎn)單的基線和一個(gè)新的評(píng)估指標(biāo)，它采用卡爾曼濾波器作為運(yùn)動(dòng)模型，并使用檢測(cè)和軌跡之間的 3D IoU 進(jìn)行關(guān)聯(lián)。CenterPoint 將 CenterTrack 中基于中心的跟蹤范例擴(kuò)展到 3D。它利用預(yù)測(cè)的物體速度作為恒速運(yùn)動(dòng)模型，并在突然運(yùn)動(dòng)下顯示出有效性。以下工作 [80]、[81]、[82]、[94] 側(cè)重于關(guān)聯(lián)度量和生命周期管理的改進(jìn)。 QTrack 估計(jì)預(yù)測(cè)對(duì)象屬性的質(zhì)量，并提出質(zhì)量感知關(guān)聯(lián)策略以獲得更穩(wěn)健的關(guān)聯(lián)。

[80] N. Benbarka, J. Schr?der, and A. Zell, “Score refinement for confidencebased 3d multi-object tracking,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021, pp.8083–8090.
[81] Z. Pang, Z. Li, and N. Wang, “Simpletrack: Understanding and rethinking 3d multi-object tracking,” arXiv preprint arXiv:2111.09621, 2021.
[82] Q. Wang, Y . Chen, Z. Pang, N. Wang, and Z. Zhang, “Immortal tracker: Tracklet never dies,” arXiv preprint arXiv:2111.13672, 2021.
[94] H.-K. Chiu, J. Li, R. Ambrus ?, and J. Bohg, “Probabilistic 3d multi-
modal, multi-object tracking for autonomous driving,” in ICRA, 2021, pp. 14 227–14 233.

從圖像中提取的視覺(jué)外觀特征可以進(jìn)一步增強(qiáng) 3D MOT 中的長(zhǎng)期關(guān)聯(lián)。 QD3DT 在 2D MOT 中提出了一種遵循 QDTrack 的外觀特征的準(zhǔn)密集相似性學(xué)習(xí)來(lái)處理對(duì)象重現(xiàn)問(wèn)題。 GNN3DMOT 集成了運(yùn)動(dòng)和外觀特征，并通過(guò)圖神經(jīng)網(wǎng)絡(luò)學(xué)習(xí)它們之間的交互。 TripletTrack 提出局部對(duì)象特征嵌入來(lái)編碼有關(guān)視覺(jué)外觀和單眼 3D 對(duì)象特征的信息，以在遮擋和缺失檢測(cè)的情況下實(shí)現(xiàn)更穩(wěn)健的性能。 Transformer 的最新進(jìn)展使其在 3D MOT 中具有吸引力。繼 2D MOT 中的 MOTR 之后，基于相機(jī)的 3D 跟蹤器 MUTR3D 利用 Transformer 學(xué)習(xí)具有 2D 視覺(jué)信息的 3D 表示，并以端到端的方式跨時(shí)間傳播 3D 邊界框。

我們只利用運(yùn)動(dòng)線索來(lái)執(zhí)行數(shù)據(jù)關(guān)聯(lián)，以更簡(jiǎn)單地統(tǒng)一 2D 和 3D MOT。我們提出了一種基于檢測(cè)速度和卡爾曼濾波器的綜合運(yùn)動(dòng)預(yù)測(cè)策略。采用卡爾曼濾波預(yù)測(cè)的平滑位置進(jìn)行長(zhǎng)期關(guān)聯(lián)，起到類(lèi)似外觀特征處理重現(xiàn)問(wèn)題的作用。

3 BYTETRACK V2

我們?cè)谝粋€(gè)簡(jiǎn)單統(tǒng)一的框架中解決了 2D 和 3D MOT 的問(wèn)題。它包含三個(gè)部分：對(duì)象檢測(cè)、運(yùn)動(dòng)預(yù)測(cè)和數(shù)據(jù)關(guān)聯(lián)。我們首先在第3.1節(jié)中介紹了如何表述2D MOT和3D MOT的問(wèn)題。然后，我們?cè)诘?.2節(jié)中列出了一些2D和3D檢測(cè)器以及基本的卡爾曼濾波器運(yùn)動(dòng)模型。在第3.3節(jié)中，我們介紹了專門(mén)為3D MOT提出的互補(bǔ)3D運(yùn)動(dòng)預(yù)測(cè)策略。最后，我們?cè)诘?.4節(jié)中詳細(xì)闡述了所提出的檢測(cè)驅(qū)動(dòng)的分層數(shù)據(jù)關(guān)聯(lián)策略的核心步驟，以及它如何緩解對(duì)象丟失和軌跡碎片的問(wèn)題。跟蹤框架的概述如圖3所示。

3.1 Problem Formulation（問(wèn)題表述）

多目標(biāo)跟蹤。多目標(biāo)跟蹤的目標(biāo)是估計(jì)視頻中的目標(biāo)軌跡。假設(shè)我們要在視頻中獲得 $L$ 個(gè)軌跡 $\mathbb{S}=s^1,s^2,...,s^L$。每條軌跡$s^i=b_{t1}^i,b_{t1+1}^i,...,b_{t2}^i$包含一個(gè)對(duì)象在一個(gè)時(shí)間段內(nèi)的位置信息，即從$t_1$幀到$t_2$幀，該對(duì)象出現(xiàn)的位置。在 2D MOT 中，對(duì)象 $i$ 在幀 $t$ 的位置可以表示為 $b_{t2d}^i=[x_1,y_1,x_2,y_2]\in {\mathbb{R}}^4$，其中 $(x_1,y_1),(x_2,y_2)$是圖像平面中二維對(duì)象邊界框的左上和右下坐標(biāo)。在 3D MOT 中，跟蹤過(guò)程通常在 3D 世界坐標(biāo)中執(zhí)行。對(duì)象 $i$ 在幀 $t$ 的 3D 位置可以表示為 $b_{t3d}^i=[x,y,z,\theta ,l,w,h]\in {\mathbb{R}}^7$，其中 $(x,y,z)$ 是對(duì)象中心的 3D 世界位置, $\theta$ 是物體方向，$(l,w,h)$ 是物體尺寸。


數(shù)據(jù)關(guān)聯(lián)。我們遵循多目標(biāo)跟蹤中流行的tracking-by-detection范例，該范例首先檢測(cè)單個(gè)視頻幀中的對(duì)象，然后將它們關(guān)聯(lián)到幀之間并隨時(shí)間形成軌跡。假設(shè)我們?cè)诘?$t$ 幀有 $M$ 個(gè)檢測(cè)和 $N$ 個(gè)歷史軌跡，我們的目標(biāo)是將每個(gè)檢測(cè)分配給其中一個(gè)軌跡，該軌跡在整個(gè)視頻中具有相同的身份。令 $\mathbb{A}$ 表示由所有可能的關(guān)聯(lián)（或匹配）組成的空間。在多目標(biāo)跟蹤的設(shè)置下，每個(gè)檢測(cè)最多匹配一個(gè)軌跡，每個(gè)軌跡最多匹配一個(gè)檢測(cè)。我們定義空間 $\mathbb{A}$ 如下：

其中 M = { 1 , 2 , . . . , M } ， N = { 1 , 2 , . . . , N } \mathbb{M} = \left\{ 1, 2, ..., M \right\} ，\mathbb{N} = \left\{ 1, 2, ..., N \right\} M={1,2,...,M}，N={1,2,...,N}，$A$ 是整個(gè) $M$ 個(gè)檢測(cè)和 $N$ 個(gè)軌跡的一個(gè)可能匹配。當(dāng)?shù)?$i$ 個(gè)檢測(cè)與第 $j $個(gè)軌跡匹配時(shí)，則 $m_{ij}=1$。設(shè) $d_t^1,...,d_t^M$ 和$h_t^1,...,h_t^N$ 分別是幀$t$ 處所有 $M$ 個(gè)檢測(cè)和 $N$ 個(gè)軌跡的位置。我們計(jì)算所有檢測(cè)和軌跡之間的相似度矩陣 $S_t\in {\mathbb{R}}^{M\times N}$，如下所示：

其中相似性可以通過(guò)檢測(cè)和軌跡之間的一些空間距離來(lái)計(jì)算，例如 IoU 或 L2 距離。我們的目標(biāo)是獲得最佳匹配 $A?$，其中匹配檢測(cè)和軌跡之間的總相似度（或得分）最高：

3.2 Preliminary

二維物體檢測(cè)器。我們采用 YOLOX 作為我們的 2D 物體檢測(cè)器。 YOLOX 是一種無(wú)錨檢測(cè)器，配備了先進(jìn)的檢測(cè)技術(shù)，即解耦頭，以及源自 OTA 的領(lǐng)先標(biāo)簽分配策略 SimOTA。它還采用強(qiáng)大的數(shù)據(jù)增強(qiáng)，如mosaic[32]和mixup[98]，以進(jìn)一步提高檢測(cè)性能。與其他現(xiàn)代檢測(cè)器 [99]、[100] 相比，YOLOX 在速度和精度之間取得了極好的平衡，并且在實(shí)際應(yīng)用中很有吸引力。

[99] H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. M. Ni, and H.-Y .Shum, “Dino: Detr with improved denoising anchor boxes for end-toend object detection,” arXiv preprint arXiv:2203.03605, 2022.
[100] Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” arXiv preprint arXiv:2103.14030, 2021.

基于相機(jī)的 3D 對(duì)象檢測(cè)器。我們遵循多攝像頭 3D 對(duì)象檢測(cè)設(shè)置，通過(guò)學(xué)習(xí)鳥(niǎo)瞰圖 (BEV) 中強(qiáng)大且統(tǒng)一的表示，顯示出優(yōu)于單目方法的優(yōu)勢(shì)。我們利用 PETRv2 [101] 作為我們基于相機(jī)的 3D 物體檢測(cè)器。它建立在 PETR [77] 之上，通過(guò)將 3D 坐標(biāo)的位置信息編碼為圖像特征，將基于變換器的 2D 對(duì)象檢測(cè)器 DETR [40] 擴(kuò)展到多視圖 3D 設(shè)置。 PETRv2 利用先前幀的時(shí)間信息來(lái)提高檢測(cè)性能。
基于 LiDAR 的 3D 物體檢測(cè)器。我們采用 CenterPoint [11] 和 TransFusion-L [102] 作為我們基于 LiDAR 的 3D 物體檢測(cè)器。 CenterPoint 利用關(guān)鍵點(diǎn)檢測(cè)器找到對(duì)象的中心，并簡(jiǎn)單地回歸到其他 3D 屬性。它還在第二階段使用對(duì)象上的附加點(diǎn)特征來(lái)優(yōu)化這些 3D 屬性。 TransFusion-L 由卷積主干和基于變壓器解碼器的檢測(cè)頭組成。 它使用一組稀疏的對(duì)象查詢從 LiDAR 點(diǎn)云預(yù)測(cè) 3D 邊界框。
基本運(yùn)動(dòng)模型。我們利用卡爾曼濾波器 [12] 作為 2D 和 3D MOT 的基本運(yùn)動(dòng)模型。與[4]類(lèi)似，我們?cè)诙S跟蹤場(chǎng)景中定義了一個(gè)八維狀態(tài)空間$（u,v,a,b,\dot{u},\dot{v},\dot{a},\dot{b}）$，其中$P^{2d}=(u,v,a,b)$是 2D 邊界框中心位置、寬高比（寬度/高度）和邊界框高度。 $V^{2d}=(\dot{u},\dot{v},\dot{a},\dot{b})$ 是圖像平面中各自的速度。在 3D 跟蹤場(chǎng)景中，我們按照 [7] 定義一個(gè)十維狀態(tài)空間$(x,y,z,\theta ,l,w,h,\dot{x},\dot{y},\dot{z})$，其中 $P^{3d}=(x,y,z)$是 3D 邊界框中心位置，$(l,w,h)$是物體的大小，$\theta$是物體方向，$V^{3d}=(\dot{x},\dot{y},\dot{z})$ 是 3D 空間中的相應(yīng)速度。與 [7] 不同，我們?cè)?3D 世界坐標(biāo)中定義狀態(tài)空間以消除自我運(yùn)動(dòng)的影響。我們直接采用標(biāo)準(zhǔn)的卡爾曼濾波器，具有等速運(yùn)動(dòng)和線性觀測(cè)模型。 2D和3D跟蹤場(chǎng)景中第$t+1$幀的運(yùn)動(dòng)預(yù)測(cè)過(guò)程可以表示如下：
$$\begin{gathered} P_{t+1}^{2d}=P_t^{2d}+V_t^{2d} \\ {P}_{t+1}^{3d}=P_t^{3d}+V_t^{3d} \end{gathered}$$

[7] X. Weng, J. Wang, D. Held, and K. Kitani, “3d multi-object tracking: A baseline and new evaluation metrics,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 10 359–10 366.

每個(gè)軌跡的更新?tīng)顟B(tài)是軌跡和匹配檢測(cè)（或觀察）的加權(quán)平均值。權(quán)重由軌跡和匹配檢測(cè)遵循貝葉斯規(guī)則的不確定性決定。

3.3 Complementary 3D Motion Prediction(互補(bǔ)的 3D 運(yùn)動(dòng)預(yù)測(cè))

我們提出了一種互補(bǔ)的 3D 運(yùn)動(dòng)預(yù)測(cè)策略來(lái)解決駕駛場(chǎng)景中的突然運(yùn)動(dòng)和短期物體消失問(wèn)題。具體來(lái)說(shuō)，我們采用短期關(guān)聯(lián)的檢測(cè)速度和長(zhǎng)期關(guān)聯(lián)的卡爾曼濾波器。

在 3D 場(chǎng)景中，現(xiàn)代檢測(cè)器 [11]、[75]、[101] 能夠通過(guò)時(shí)間建模預(yù)測(cè)準(zhǔn)確的短期速度?？柭鼮V波器通過(guò)基于歷史信息的狀態(tài)更新對(duì)平滑的長(zhǎng)期速度進(jìn)行建模。我們通過(guò)雙邊預(yù)測(cè)策略最大化兩種運(yùn)動(dòng)模型的優(yōu)勢(shì)。我們采用卡爾曼濾波器進(jìn)行前向預(yù)測(cè)，采用檢測(cè)到的物體速度進(jìn)行后向預(yù)測(cè)。后向預(yù)測(cè)負(fù)責(zé)活動(dòng)軌跡的短期關(guān)聯(lián)，而前向預(yù)測(cè)用于丟失軌跡的長(zhǎng)期關(guān)聯(lián)。圖 4 說(shuō)明了互補(bǔ)運(yùn)動(dòng)預(yù)測(cè)策略。

[11] T. Yin, X. Zhou, and P . Krahenbuhl, “Center-based 3d object detection and tracking,” in CVPR, 2021, pp. 11 784–11 793.
[75] Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Y u, and J. Dai, “Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,” arXiv preprint arXiv:2203.17270, 2022.
[101] Y . Liu, J. Yan, F. Jia, S. Li, Q. Gao, T. Wang, X. Zhang, and J. Sun, “Petrv2: A unified framework for 3d perception from multi-camera images,” arXiv preprint arXiv:2206.01256, 2022.

在雙邊預(yù)測(cè)之后，我們使用了第3.4節(jié)中引入的統(tǒng)一2D和3D數(shù)據(jù)關(guān)聯(lián)策略。在第一個(gè)關(guān)聯(lián)中，在來(lái)自反向預(yù)測(cè)的檢測(cè)結(jié)果$D^{t?1}$和軌跡$T^{t?1}$之間計(jì)算相似度$S^t\in {\mathbb{R}}^{M\times N}$，如下所示：
$$S_t(i,j)=GIoU(D_i^{t?1},T_j^{t?1})(9)$$
我們采用 3D GIoU [103] 作為相似性度量來(lái)解決檢測(cè)框和軌跡框之間偶爾出現(xiàn)的非重疊問(wèn)題。我們使用匈牙利算法[91]完成基于$S_t$的身份分配。關(guān)聯(lián)后，匹配的檢測(cè)$D_{match}^t$用于按照標(biāo)準(zhǔn)卡爾??曼濾波器更新規(guī)則更新匹配的tracklets $T_{match}^t$。在式8中的前向預(yù)測(cè)策略，當(dāng)軌跡丟失時(shí)起著重要作用，即沒(méi)有匹配的檢測(cè)。當(dāng)丟失的對(duì)象在后續(xù)幀中再次出現(xiàn)時(shí)，可以通過(guò)與預(yù)測(cè)位置的相似度重新關(guān)聯(lián)，也稱為軌跡重生。算法 1 中的第二個(gè)關(guān)聯(lián)遵循與第一個(gè)關(guān)聯(lián)相同的過(guò)程。

我們采用檢測(cè)分?jǐn)?shù)通過(guò)自適應(yīng)更新幀$t$ 軌跡 $j$ 的卡爾曼濾波器中的測(cè)量不確定性矩陣 ${\hat{R}}_t^j$ 來(lái)進(jìn)一步增強(qiáng)運(yùn)動(dòng)預(yù)測(cè)，如下所示：
$${\hat{R}}_t^j=\alpha (1?s_t^j{)}^2{R}_t^j(10)$$
其中 $s_t^j$ 是軌跡 $j$ 在幀 $t$ 的檢測(cè)分?jǐn)?shù)，$\alpha$ 是控制不確定性大小的超參數(shù)。通過(guò)將檢測(cè)分?jǐn)?shù)插入不確定性矩陣，我們使卡爾曼濾波器對(duì)不同質(zhì)量的檢測(cè)更加穩(wěn)健。

3.4 Unified 2D and 3D Data Association

我們?yōu)?2D 和 3D MOT 提出了一種簡(jiǎn)單、有效且統(tǒng)一的數(shù)據(jù)關(guān)聯(lián)方法。與以往的方法[7]、[8]、[9]、[11]、[34]只保留高分檢測(cè)框不同，我們保留每個(gè)檢測(cè)框并將它們分為高分和低分檢測(cè)框。得分。我們檢測(cè)驅(qū)動(dòng)的分層數(shù)據(jù)關(guān)聯(lián)策略的整個(gè)流程如圖 3 所示。

概述。在視頻的第一幀中，我們將所有檢測(cè)框初始化為 tracklet。在接下來(lái)的幀中，我們首先將高分檢測(cè)框與軌跡關(guān)聯(lián)起來(lái)。一些軌跡不匹配是因?yàn)樗鼈儾黄ヅ溥m當(dāng)?shù)母叻謾z測(cè)框，這通常發(fā)生在發(fā)生遮擋、運(yùn)動(dòng)模糊或大小變化時(shí)。然后，我們將低分檢測(cè)框與這些不匹配的軌跡關(guān)聯(lián)起來(lái)，以恢復(fù)低分檢測(cè)框中的對(duì)象并同時(shí)過(guò)濾掉背景。 ByteTrackV2 的偽代碼如算法 1 所示。

輸入。 ByteTrackV2 的輸入是一個(gè)視頻序列$V$，以及一個(gè)對(duì)象檢測(cè)器 Det。我們還設(shè)置了檢測(cè)分?jǐn)?shù)閾值 $\tau$。輸出是視頻的軌道 $T$，每個(gè)軌道包含每個(gè)幀中對(duì)象的邊界框和標(biāo)識(shí)。

檢測(cè)箱。對(duì)于視頻中的每一幀，我們使用檢測(cè)器 Det 預(yù)測(cè)檢測(cè)框和分?jǐn)?shù)。我們根據(jù)檢測(cè)分?jǐn)?shù)閾值 $\tau$ 將所有檢測(cè)框分為 $D_{high}$ 和 $D_{low}$ 兩部分。對(duì)于分?jǐn)?shù)高于$\tau$的檢測(cè)框，我們將它們放入高分檢測(cè)框$D_{high}$中。對(duì)于分?jǐn)?shù)低于 $\tau$ 的檢測(cè)框，我們將它們放入低分檢測(cè)框 $D_{low}$（算法 1 中的第 3 至 13 行）。

運(yùn)動(dòng)預(yù)測(cè)。在分離低分檢測(cè)框和高分檢測(cè)框后，我們預(yù)測(cè)$T$中每個(gè)軌道在當(dāng)前幀中的新位置（算法1中的第14行至第16行）。對(duì)于2D MOT，我們直接采用卡爾曼濾波器進(jìn)行運(yùn)動(dòng)預(yù)測(cè)。對(duì)于3D MOT，我們使用第3.3節(jié)中介紹的互補(bǔ)運(yùn)動(dòng)預(yù)測(cè)策略。

高分框關(guān)聯(lián)。在高分檢測(cè)框$D_{high}$和所有軌道$T$（包括丟失軌道$T_{lost}$）之間執(zhí)行第一關(guān)聯(lián)。Similarity #1可以通過(guò)檢測(cè)框$D_{high}$和軌道$T$的預(yù)測(cè)框之間的空間距離（例如IoU）來(lái)計(jì)算。然后，我們采用匈牙利算法[91]來(lái)完成基于相似度的匹配。我們保留了$D_{remain}$中的不匹配檢測(cè)和$T_{remain}$中的非匹配軌道（算法1中的第17行至第19行）。整個(gè)管道非常靈活，可以兼容其他不同的關(guān)聯(lián)方法。例如，當(dāng)它與FairMOT[8]相結(jié)合時(shí)，Re-ID特征被添加到算法1中的*first association*中，其他都是一樣的。在2D MOT的實(shí)驗(yàn)中，我們將關(guān)聯(lián)方法應(yīng)用于9種不同的最先進(jìn)的跟蹤器，并在幾乎所有的指標(biāo)上都取得了顯著的改進(jìn)。

低分框關(guān)聯(lián)。在低得分檢測(cè)框$D_{low}$和第一次關(guān)聯(lián)之后的剩余軌道$T_{remain}$之間執(zhí)行第二關(guān)聯(lián)。我們保留$T_{re?remain}$中不匹配的軌道，并刪除所有不匹配的低分檢測(cè)框，因?yàn)槲覀儗⑺鼈円暈楸尘?。（算?中的第20至21行）。我們發(fā)現(xiàn)單獨(dú)使用IoU作為第二關(guān)聯(lián)中的Similarity#2很重要，因?yàn)榈头謹(jǐn)?shù)檢測(cè)框通常包含嚴(yán)重的遮擋或運(yùn)動(dòng)模糊，并且外觀特征不可靠。因此，當(dāng)應(yīng)用于其他基于Re-ID的跟蹤器[8]、[9]、[34]時(shí)，我們?cè)诘诙侮P(guān)聯(lián)中不采用外觀相似性。

追蹤重生。關(guān)聯(lián)后，未匹配的軌跡將從軌跡集中刪除。為了簡(jiǎn)單起見(jiàn)，我們沒(méi)有在算法1中列出軌道再生[4]、[26]、[52]的過(guò)程。事實(shí)上，對(duì)于長(zhǎng)期的關(guān)聯(lián)來(lái)說(shuō)，保留軌跡的身份是必要的。對(duì)于第二次關(guān)聯(lián)后保留的不匹配軌道$T_{re?remain}$，我們將它們放入$T_{lost}$。對(duì)于$T_{lost}$中的每個(gè)軌道，只有當(dāng)它存在超過(guò)一定數(shù)量的幀（即30幀）時(shí)，我們才會(huì)將其從軌道$T$中刪除。否則，我們保留$T$中丟失的軌跡$T_{lost}$（算法1中的第22行）。最后，我們?cè)诘谝淮侮P(guān)聯(lián)之后從不匹配的高分檢測(cè)框$D_{remain}$中初始化新的軌跡（算法1中的第23到27行）。每個(gè)單獨(dú)幀的輸出是當(dāng)前幀中軌道$T$的邊界框和標(biāo)識(shí)。

討論。我們根據(jù)經(jīng)驗(yàn)發(fā)現(xiàn)，當(dāng)遮擋率增加時(shí)，檢測(cè)分?jǐn)?shù)下降。當(dāng)發(fā)生遮擋情況時(shí)，分?jǐn)?shù)先降低后增加，因?yàn)樾腥讼缺徽趽?，然后再次出現(xiàn)。這激發(fā)了我們首先將高分框與tracklets聯(lián)系起來(lái)。如果軌跡集與任何高分框都不匹配，則很有可能被遮擋，檢測(cè)分?jǐn)?shù)下降。然后，我們將其與低分?jǐn)?shù)框相關(guān)聯(lián)，以跟蹤被遮擋的目標(biāo)。對(duì)于那些低分框中的假陽(yáng)性，沒(méi)有任何tracklet能夠與之匹配。因此，我們將它們?nèi)拥?。這是我們的數(shù)據(jù)關(guān)聯(lián)算法工作的關(guān)鍵點(diǎn)。

4 DATASETS AND METRICS

4.1 Datasets

MOT17數(shù)據(jù)集[46]具有由移動(dòng)和固定攝像機(jī)以不同幀速率從不同視點(diǎn)拍攝的視頻。它包含7個(gè)訓(xùn)練視頻和7個(gè)測(cè)試視頻。我們使用MOT17訓(xùn)練集中每個(gè)視頻的前半部分進(jìn)行訓(xùn)練，后半部分進(jìn)行驗(yàn)證[26]。MOT17同時(shí)提供“公共檢測(cè)”和“私人檢測(cè)”協(xié)議。公共探測(cè)器包括DPM[47]、Faster R-CNN[15]和SDP[48]。對(duì)于私人檢測(cè)設(shè)置，我們?cè)谙谘芯恐凶裱璠26]、[27]、[43]、[45]，在CrowdHuman數(shù)據(jù)集[104]和MOT17半訓(xùn)練集的組合上進(jìn)行訓(xùn)練。在MOT17的測(cè)試集上進(jìn)行測(cè)試時(shí)，我們添加了Cityperson[105]和ETHZ[106]用于[8]、[34]、[35]之后的培訓(xùn)。

MOT20數(shù)據(jù)集[107]在非常擁擠的場(chǎng)景中捕捉視頻，因此會(huì)發(fā)生很多遮擋。車(chē)架內(nèi)的平均行人比MOT17大得多（139人對(duì)33人）。MOT20包含4個(gè)訓(xùn)練視頻和4個(gè)較長(zhǎng)視頻長(zhǎng)度的測(cè)試視頻。它還提供了FasterRCNN的公共檢測(cè)結(jié)果。我們只使用CrowdHuman數(shù)據(jù)集和MOT20的訓(xùn)練集在私人檢測(cè)設(shè)置下進(jìn)行訓(xùn)練。

HiEve數(shù)據(jù)集[108]是一個(gè)以人為中心的大規(guī)模數(shù)據(jù)集，專注于擁擠和復(fù)雜的事件。它包含更長(zhǎng)的平均軌跡長(zhǎng)度，給人類(lèi)跟蹤任務(wù)帶來(lái)了更大的難度。HiEve拍攝了30多種不同場(chǎng)景的視頻，包括地鐵站、街道和餐廳，這使得跟蹤問(wèn)題成為一項(xiàng)更具挑戰(zhàn)性的任務(wù)。它包含19個(gè)培訓(xùn)視頻和13個(gè)測(cè)試視頻。我們將CrowdHuman和HiEve的訓(xùn)練集結(jié)合起來(lái)進(jìn)行訓(xùn)練。

BDD100K數(shù)據(jù)集[109]是最大的2D駕駛視頻數(shù)據(jù)集，2D MOT任務(wù)的數(shù)據(jù)集拆分為1400個(gè)用于訓(xùn)練的視頻、200個(gè)用于驗(yàn)證的視頻和400個(gè)用于測(cè)試的視頻。它需要跟蹤8類(lèi)對(duì)象，并包含大型相機(jī)運(yùn)動(dòng)的情況。我們將檢測(cè)任務(wù)和2D MOT任務(wù)的訓(xùn)練集結(jié)合起來(lái)進(jìn)行訓(xùn)練。

nuScenes數(shù)據(jù)集[110]是一個(gè)大規(guī)模的3D對(duì)象檢測(cè)和跟蹤數(shù)據(jù)集。這是第一個(gè)搭載全自動(dòng)駕駛汽車(chē)傳感器套件的數(shù)據(jù)集：6個(gè)攝像頭、5個(gè)雷達(dá)和1個(gè)激光雷達(dá)，均具有全360度視場(chǎng)。跟蹤任務(wù)包含具有7個(gè)對(duì)象類(lèi)的三維標(biāo)注。nuScenes包括1000個(gè)場(chǎng)景，包括700個(gè)訓(xùn)練視頻、150個(gè)驗(yàn)證視頻和150個(gè)測(cè)試視頻。對(duì)于每個(gè)序列，僅對(duì)以2 FPS采樣的關(guān)鍵幀進(jìn)行注釋。在一個(gè)序列中，每個(gè)相機(jī)大約有40個(gè)關(guān)鍵幀。我們只使用每個(gè)序列中的關(guān)鍵幀進(jìn)行跟蹤。

4.2 Metrics

2D MOT。我們使用CLEAR度量[111]，包括MOTA、FP、FN、ID等，IDF1[112]和HOTA[113]來(lái)評(píng)估跟蹤性能的不同方面。MOTA是根據(jù)FP、FN和ID計(jì)算的，如下所示：

其中GT代表地面實(shí)況對(duì)象的數(shù)量。考慮到FP和FN的數(shù)量大于ID，MOTA更關(guān)注檢測(cè)性能。IDF1評(píng)估身份保存能力，并更多地關(guān)注關(guān)聯(lián)性能。HOTA是最近提出的一種度量，它明確地平衡了執(zhí)行精確檢測(cè)、關(guān)聯(lián)和定位的效果。對(duì)于BDD100K數(shù)據(jù)集，有一些多類(lèi)度量，如mMOTA和mIDF1。mMOTA/mIDF1是通過(guò)對(duì)所有類(lèi)別的MOTA/IDF1進(jìn)行平均來(lái)計(jì)算的。

3D MOT. nuScenes跟蹤基準(zhǔn)采用平均多目標(biāo)跟蹤精度（AMOTA）[7]作為主要指標(biāo)，對(duì)不同召回閾值上的MOTA指標(biāo)進(jìn)行平均，以減少檢測(cè)置信閾值的影響。sMOTAr[7]將MOTA增加一個(gè)術(shù)語(yǔ)，以針對(duì)相應(yīng)的召回進(jìn)行調(diào)整，并保證sMOTAr值的范圍從0.0到1.0：

然后，利用40點(diǎn)插值來(lái)計(jì)算AMOTA度量

然而，我們?cè)趯?shí)驗(yàn)中發(fā)現(xiàn)，它仍然需要選擇一個(gè)合適的檢測(cè)分?jǐn)?shù)閾值，因?yàn)榧訇?yáng)性可能會(huì)誤導(dǎo)關(guān)聯(lián)結(jié)果

5 EXPERIMENTS

5.1 Implementation Details

2D MOT。檢測(cè)器是YOLOX[33]，YOLOX-X作為主干，COCO預(yù)訓(xùn)練模型[114]作為初始化權(quán)重。對(duì)于MOT17，訓(xùn)練計(jì)劃是MOT17、CrowdHuman、Cityperson和ETHZ組合的80個(gè)epochs。對(duì)于MOT20和HiEve，我們只添加CrowdHuman作為額外的訓(xùn)練數(shù)據(jù)。對(duì)于BDD100K，我們不使用額外的訓(xùn)練數(shù)據(jù)，只訓(xùn)練50個(gè)epochs。在多尺度訓(xùn)練過(guò)程中，輸入圖像大小為1440×800，最短邊的范圍為576到1024。數(shù)據(jù)擴(kuò)充包括Mosaic[32]和Mixup[98]。該模型在8個(gè)NVIDIA特斯拉V100 GPU上進(jìn)行訓(xùn)練，批量大小為48。優(yōu)化器是SGD，重量衰減為5×10?4，動(dòng)量為0.9。初始學(xué)習(xí)率為10?3，有1個(gè)epochs預(yù)熱和余弦退火計(jì)劃。總訓(xùn)練時(shí)間約為12小時(shí)。根據(jù)[33]，在單個(gè)GPU上以FP16精度[115]和1的批量大小測(cè)量FPS。

對(duì)于跟蹤部分，除非另有規(guī)定，否則默認(rèn)檢測(cè)分?jǐn)?shù)閾值τ為0.6。對(duì)于MOT17、MOT20和HiEve的基準(zhǔn)評(píng)估，我們只使用IoU作為相似性度量。在線性分配步驟中，如果檢測(cè)框和軌跡框之間的IoU小于0.2，則匹配將被拒絕。對(duì)于丟失的軌跡，我們將其保留30幀，以防再次出現(xiàn)。對(duì)于BDD100K，我們使用UniTrack[116]作為Re-ID模型。在消融研究中，我們使用FastReID[117]來(lái)提取MOT17的Re-ID特征。

3D MOT. 對(duì)于相機(jī)模態(tài)設(shè)置，我們采用PETRv2[101]，VoVNetV2主干[118]作為檢測(cè)器。輸入圖像大小為1600×640。檢測(cè)查詢編號(hào)設(shè)置為1500。它在驗(yàn)證數(shù)據(jù)集上使用預(yù)訓(xùn)練模型FCOS3D[64]進(jìn)行訓(xùn)練，在測(cè)試數(shù)據(jù)集上用DD3D[119]預(yù)訓(xùn)練模型進(jìn)行初始化。優(yōu)化器是AdamW[120]，其重量衰減為0.01。學(xué)習(xí)率用2.0×10?4初始化，并用余弦退火策略衰減。該模型在8個(gè)特斯拉A100 GPU上進(jìn)行了24個(gè)時(shí)期的批量8個(gè)GPU的訓(xùn)練。對(duì)于激光雷達(dá)模態(tài)設(shè)置，我們使用具有VoxelNet[56]3D主干的TransFusion-L作為nuScenes測(cè)試集的檢測(cè)器。該模型以激光雷達(dá)點(diǎn)云作為輸入，訓(xùn)練了20個(gè)時(shí)期。對(duì)于nuScenes驗(yàn)證集的結(jié)果，我們采用了以V oxelNet主干作為檢測(cè)器的CenterPoint[11]。相同的檢測(cè)結(jié)果用于與其他3D MOT方法[7]、[11]、[80]、[94]進(jìn)行公平比較。

對(duì)于跟蹤部分，PETRv2和CenterPoint的檢測(cè)得分閾值τ為0.2。TransFusionL的檢測(cè)分?jǐn)?shù)低于其他檢測(cè)器，因?yàn)樗挥?jì)算為熱圖分?jǐn)?shù)和分類(lèi)分?jǐn)?shù)的幾何平均值，所以我們將TransFusionL.的閾值τ設(shè)置為0.01。我們?yōu)椴煌膶?duì)象類(lèi)別設(shè)置不同的3D GIoU閾值，因?yàn)樗鼈兙哂胁煌拇笮『退俣?。具體地說(shuō)，我們?yōu)樽孕熊?chē)設(shè)置了-0.7，為公共汽車(chē)設(shè)置了-0.2，為汽車(chē)設(shè)置了-0.1，為摩托車(chē)設(shè)置了-0.5，為行人設(shè)置了-0.0，為拖車(chē)設(shè)置了-0.4，為卡車(chē)設(shè)置了-0.1。與2D MOT類(lèi)似，我們將丟失的軌跡保持30幀，以防再次出現(xiàn)。用于更新方程中測(cè)量不確定度矩陣的超參數(shù)α。10對(duì)于基于相機(jī)的方法為100，對(duì)于基于激光雷達(dá)的方法為10。

實(shí)驗(yàn)部分就略過(guò)了，詳細(xì)的可以去原論文看，在此不多贅述

6 CONCLUSION

我們介紹了ByteTrackV2，這是一個(gè)簡(jiǎn)單而統(tǒng)一的跟蹤框架，旨在解決2D和3D MOT的問(wèn)題。ByteTrackV2融合了對(duì)象檢測(cè)、運(yùn)動(dòng)預(yù)測(cè)和檢測(cè)驅(qū)動(dòng)的分層數(shù)據(jù)關(guān)聯(lián)，使其成為MOT的全面解決方案。分層數(shù)據(jù)關(guān)聯(lián)策略利用檢測(cè)分?jǐn)?shù)作為強(qiáng)大的先驗(yàn)，在低分?jǐn)?shù)檢測(cè)中識(shí)別正確的對(duì)象，減少了遺漏檢測(cè)和碎片軌跡的問(wèn)題。此外，我們針對(duì)3D MOT的集成運(yùn)動(dòng)預(yù)測(cè)策略有效地解決了突然運(yùn)動(dòng)和物體丟失的問(wèn)題。ByteTrackV2在2D和3D MOT基準(zhǔn)測(cè)試上都實(shí)現(xiàn)了最先進(jìn)的性能。此外，它具有強(qiáng)大的泛化能力，可以很容易地與不同的2D和3D檢測(cè)器組合，而無(wú)需任何可學(xué)習(xí)的參數(shù)。我們相信，這個(gè)簡(jiǎn)單統(tǒng)一的跟蹤框架將在現(xiàn)實(shí)世界的應(yīng)用中發(fā)揮作用。

REFERENCES

[1] A. Milan, S. Roth, and K. Schindler, “Continuous energy minimization for multitarget tracking,” IEEE transactions on pattern analysis and machine intelligence, vol. 36, no. 1, pp. 58–72, 2013.

[2] S.-H. Bae and K.-J. Y oon, “Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning,” in CVPR, 2014, pp. 1218–1225.

[3] A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple online and realtime tracking,” in ICIP. IEEE, 2016, pp. 3464–3468.

[4] N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” in ICIP. IEEE, 2017, pp.3645–3649.

[5] W. Luo, B. Yang, and R. Urtasun, “Fast and furious: Real time endto-end 3d detection, tracking and motion forecasting with a single convolutional net,” in CVPR, 2018, pp. 3569–3577.

[6] E. Baser, V . Balasubramanian, P . Bhattacharyya, and K. Czarnecki, “Fantrack: 3d multi-object tracking with feature association network,” in 2019 IEEE Intelligent V ehicles Symposium (IV). IEEE, 2019, pp.1426–1433.

[7] X. Weng, J. Wang, D. Held, and K. Kitani, “3d multi-object tracking: A baseline and new evaluation metrics,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 10 359–10 366.

[8] Y . Zhang, C. Wang, X. Wang, W. Zeng, and W. Liu, “Fairmot: On the fairness of detection and re-identification in multiple object tracking,” International Journal of Computer Vision, vol. 129, no. 11, pp. 3069– 3087, 2021.

[9] J. Pang, L. Qiu, X. Li, H. Chen, Q. Li, T. Darrell, and F. Y u, “Quasidense similarity learning for multiple object tracking,” in CVPR, 2021, pp. 164–173.

[10] X. Weng, J. Wang, D. Held, and K. Kitani, “Ab3dmot: A baseline for 3d multi-object tracking and new evaluation metrics,” arXiv preprint arXiv:2008.08063, 2020.

[11] T. Yin, X. Zhou, and P . Krahenbuhl, “Center-based 3d object detection and tracking,” in CVPR, 2021, pp. 11 784–11 793.

[12] R. E. Kalman, “A new approach to linear filtering and prediction problems,” J. Fluids Eng., vol. 82, no. 1, pp. 35–45, 1960.

[13] S. Chen, X. Wang, T. Cheng, Q. Zhang, C. Huang, and W. Liu, “Polar parametrization for vision-based surround-view 3d detection,” arXiv preprint arXiv:2206.10965, 2022.

[14] Y . Zhang, P . Sun, Y . Jiang, D. Y u, F. Weng, Z. Y uan, P . Luo, W. Liu, and X. Wang, “Bytetrack: Multi-object tracking by associating every detection box,” in ECCV. Springer, 2022, pp. 1–21.

[15] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in neural information processing systems, 2015, pp. 91–99.

[16] K. He, G. Gkioxari, P . Dollár, and R. Girshick, “Mask r-cnn,” in ICCV, 2017, pp. 2961–2969.

[17] J. Redmon and A. Farhadi, “Y olov3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.

[18] T.-Y . Lin, P . Goyal, R. Girshick, K. He, and P . Dollár, “Focal loss for dense object detection,” in ICCV, 2017, pp. 2980–2988.

[19] Z. Cai and N. V asconcelos, “Cascade r-cnn: Delving into high quality object detection,” in CVPR, 2018, pp. 6154–6162.

[20] P . Sun, R. Zhang, Y . Jiang, T. Kong, C. Xu, W. Zhan, M. Tomizuka, L. Li, Z. Y uan, C. Wang et al, “Sparse r-cnn: End-to-end object detection with learnable proposals,” in CVPR, 2021, pp. 14 454–14 463.

[21] P . Sun, Y . Jiang, E. Xie, W. Shao, Z. Y uan, C. Wang, and P . Luo, “What makes for end-to-end object detection?” in Proceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 139. PMLR, 2021, pp. 9934–9944.

[22] C.-Y . Wang, A. Bochkovskiy, and H.-Y . M. Liao, “Y olov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” arXiv preprint arXiv:2207.02696, 2022.

[23] Z. Lu, V . Rathod, R. V otel, and J. Huang, “Retinatrack: Online single stage joint detection and tracking,” in CVPR, 2020, pp. 14 668–14 678.

[24] J. Peng, C. Wang, F. Wan, Y . Wu, Y . Wang, Y . Tai, C. Wang, J. Li, F. Huang, and Y . Fu, “Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking,” in ECCV. Springer, 2020, pp. 145–161.

[25] X. Zhou, D. Wang, and P . Kr?henbühl, “Objects as points,” arXiv preprint arXiv:1904.07850, 2019.

[26] X. Zhou, V . Koltun, and P . Kr?henbühl, “Tracking objects as points,” in ECCV. Springer, 2020, pp. 474–490.

[27] J. Wu, J. Cao, L. Song, Y . Wang, M. Yang, and J. Y uan, “Track to detect and segment: An online multi-object tracker,” in CVPR, 2021, pp. 12 352–12 361.

[28] L. Zheng, M. Tang, Y . Chen, G. Zhu, J. Wang, and H. Lu, “Improving multiple object tracking with single object tracking,” in CVPR, 2021, pp. 2453–2462.

[29] Y . Wang, K. Kitani, and X. Weng, “Joint object detection and multi-object tracking with graph neural networks,” arXiv preprint arXiv:2006.13164, 2020.

[30] P . Tokmakov, J. Li, W. Burgard, and A. Gaidon, “Learning to track with object permanence,” arXiv preprint arXiv:2103.14258, 2021.

[31] Q. Wang, Y . Zheng, P . Pan, and Y . Xu, “Multiple object tracking with correlation learning,” in CVPR, 2021, pp. 3876–3886.

[32] A. Bochkovskiy, C.-Y . Wang, and H.-Y . M. Liao, “Y olov4: Optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020.

[33] Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “Y olox: Exceeding yolo series in 2021,” arXiv preprint arXiv:2107.08430, 2021.

[34] Z. Wang, L. Zheng, Y . Liu, Y . Li, and S. Wang, “Towards real-time multi-object tracking,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16. Springer, 2020, pp. 107–122.

[35] C. Liang, Z. Zhang, Y . Lu, X. Zhou, B. Li, X. Ye, and J. Zou, “Rethinking the competition between detection and reid in multi-object tracking,” arXiv preprint arXiv:2010.12138, 2020.

[36] C. Liang, Z. Zhang, X. Zhou, B. Li, Y . Lu, and W. Hu, “One more check: Making” fake background” be tracked again,” arXiv preprint arXiv:2104.09441, 2021.

[37] P . Chu, J. Wang, Q. Y ou, H. Ling, and Z. Liu, “Transmot: Spatialtemporal graph transformer for multiple object tracking,” arXiv preprint arXiv:2104.00194, 2021.

[38] B. Yan, Y . Jiang, P . Sun, D. Wang, Z. Y uan, P . Luo, and H. Lu, “Towards grand unification of object tracking,” arXiv preprint arXiv:2207.07078, 2022.

[39] Y . Zhang, C. Wang, X. Wang, W. Zeng, and W. Liu, “Robust multiobject tracking by marginal inference,” in ECCV. Springer, 2022, pp.22–40.

[40] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in ECCV. Springer, 2020, pp. 213–229.

[41] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,” arXiv preprint arXiv:2010.04159, 2020.

[42] D. Meng, X. Chen, Z. Fan, G. Zeng, H. Li, Y . Y uan, L. Sun, and J. Wang, “Conditional detr for fast training convergence,” in ICCV, 2021, pp.3651–3660.

[43] P . Sun, Y . Jiang, R. Zhang, E. Xie, J. Cao, X. Hu, T. Kong, Z. Y uan, C. Wang, and P . Luo, “Transtrack: Multiple-object tracking with transformer,” arXiv preprint arXiv:2012.15460, 2020.

[44] T. Meinhardt, A. Kirillov, L. Leal-Taixe, and C. Feichtenhofer, “Trackformer: Multi-object tracking with transformers,” in CVPR, 2022, pp.8844–8854.

[45] F. Zeng, B. Dong, T. Wang, C. Chen, X. Zhang, and Y . Wei, “Motr: End-to-end multiple-object tracking with transformer,” arXiv preprint arXiv:2105.03247, 2021.

[46] A. Milan, L. Leal-Taixé, I. Reid, S. Roth, and K. Schindler, “Mot16: A benchmark for multi-object tracking,” arXiv preprint arXiv:1603.00831, 2016.

[47] P . Felzenszwalb, D. McAllester, and D. Ramanan, “A discriminatively trained, multiscale, deformable part model,” in CVPR. IEEE, 2008, pp. 1–8.

[48] F. Yang, W. Choi, and Y . Lin, “Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers,” in CVPR, 2016, pp. 2129–2137.

[49] J. Xu, Y . Cao, Z. Zhang, and H. Hu, “Spatial-temporal relation networks for multi-object tracking,” in ICCV, 2019, pp. 3988–3998.

[50] P . Chu and H. Ling, “Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking,” in ICCV, 2019, pp. 6172–6181.

[51] P . Bergmann, T. Meinhardt, and L. Leal-Taixe, “Tracking without bells and whistles,” in ICCV, 2019, pp. 941–951.

[52] L. Chen, H. Ai, Z. Zhuang, and C. Shang, “Real-time multiple people tracking with deeply learned candidate selection and person reidentification,” in ICME. IEEE, 2018, pp. 1–6.

[53] J. Zhu, H. Yang, N. Liu, M. Kim, W. Zhang, and M.-H. Yang, “Online multi-object tracking with dual matching attention networks,” in Proceedings of the ECCV (ECCV), 2018, pp. 366–382.

[54] G. Brasó and L. Leal-Taixé, “Learning a neural solver for multiple object tracking,” in CVPR, 2020, pp. 6247–6257.

[55] A. Hornakova, R. Henschel, B. Rosenhahn, and P . Swoboda, “Lifted disjoint paths with application in multiple object tracking,” in International Conference on Machine Learning. PMLR, 2020, pp. 4364–4375.

[56] Y . Zhou and O. Tuzel, “V oxelnet: End-to-end learning for point cloud based 3d object detection,” in CVPR, 2018, pp. 4490–4499.

[57] Y . Yan, Y . Mao, and B. Li, “Second: Sparsely embedded convolutional detection,” Sensors, vol. 18, no. 10, p. 3337, 2018.

[58] A. H. Lang, S. V ora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in CVPR, 2019, pp. 12 697–12 705.

[59] S. Shi, X. Wang, and H. Li, “Pointrcnn: 3d object proposal generation and detection from point cloud,” in CVPR, 2019, pp. 770–779.

[60] L. Du, X. Ye, X. Tan, E. Johns, B. Chen, E. Ding, X. Xue, and J. Feng, “Ago-net: Association-guided 3d point cloud object detection network,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.

[61] J. Liu, Y . Chen, X. Ye, Z. Tian, X. Tan, and X. Qi, “Spatial pruned sparse convolution for efficient 3d object detection,” arXiv preprint arXiv:2209.14201, 2022.

[62] X. Chen, K. Kundu, Y . Zhu, A. G. Berneshawi, H. Ma, S. Fidler, and R. Urtasun, “3d object proposals for accurate object class detection,” Advances in neural information processing systems, vol. 28, 2015.

[63] X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun, “Monocular 3d object detection for autonomous driving,” in CVPR, 2016, pp. 2147–2156.

[64] T. Wang, X. Zhu, J. Pang, and D. Lin, “Fcos3d: Fully convolutional one-stage monocular 3d object detection,” in ICCV, 2021, pp. 913–922.

[65] Z. Zou, X. Ye, L. Du, X. Cheng, X. Tan, L. Zhang, J. Feng, X. Xue, and E. Ding, “The devil is in the task: Exploiting reciprocal appearancelocalization features for monocular 3d object detection,” in ICCV, 2021, pp. 2713–2722.

[66] Y . Zhang, J. Lu, and J. Zhou, “Objects are different: Flexible monocular 3d object detection,” in CVPR, 2021, pp. 3289–3298.

[67] X. Ye, M. Shu, H. Li, Y . Shi, Y . Li, G. Wang, X. Tan, and E. Ding, “Rope3d: The roadside perception dataset for autonomous driving and monocular 3d object detection task,” in CVPR, 2022, pp. 21 341–21 350.

[68] C. Reading, A. Harakeh, J. Chae, and S. L. Waslander, “Categorical depth distribution network for monocular 3d object detection,” in CVPR, 2021, pp. 8555–8564.

[69] X. Ye, L. Du, Y . Shi, Y . Li, X. Tan, J. Feng, E. Ding, and S. Wen, “Monocular 3d object detection via feature domain adaptation,” in ECCV. Springer, 2020, pp. 17–34.

[70] Y . Wang, W.-L. Chao, D. Garg, B. Hariharan, M. Campbell, and K. Q.

Weinberger, “Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving,” in CVPR, 2019, pp.8445–8453.

[71] X. Weng and K. Kitani, “Monocular 3d object detection with pseudolidar point cloud,” in CVPRW, 2019, pp. 0–0.

[72] Y . Y ou, Y . Wang, W.-L. Chao, D. Garg, G. Pleiss, B. Hariharan, M. Campbell, and K. Q. Weinberger, “Pseudo-lidar++: Accurate depth for 3d object detection in autonomous driving,” arXiv preprint arXiv:1906.06310, 2019.

[73] J. M. U. Vianney, S. Aich, and B. Liu, “Refinedmpl: Refined monocular pseudolidar for 3d object detection in autonomous driving,” arXiv preprint arXiv:1911.09712, 2019.

[74] J. Philion and S. Fidler, “Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,” in ECCV.Springer, 2020, pp. 194–210.

[75] Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Y u, and J. Dai, “Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,” arXiv preprint arXiv:2203.17270, 2022.

[76] Y . Wang, V . C. Guizilini, T. Zhang, Y . Wang, H. Zhao, and J. Solomon, “Detr3d: 3d object detection from multi-view images via 3d-to-2d queries,” in Conference on Robot Learning. PMLR, 2022, pp. 180–191.

[77] Y . Liu, T. Wang, X. Zhang, and J. Sun, “Petr: Position embedding transformation for multi-view 3d object detection,” arXiv preprint arXiv:2203.05625, 2022.

[78] Y . Li, Z. Ge, G. Y u, J. Yang, Z. Wang, Y . Shi, J. Sun, and Z. Li, “Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,” arXiv preprint arXiv:2206.10092, 2022.

[79] K. Xiong, S. Gong, X. Ye, X. Tan, J. Wan, E. Ding, J. Wang, and X. Bai, “Cape: Camera view position embedding for multi-view 3d object detection,” arXiv preprint arXiv:2303.10209, 2023.

[80] N. Benbarka, J. Schr?der, and A. Zell, “Score refinement for confidencebased 3d multi-object tracking,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021, pp.8083–8090.

[81] Z. Pang, Z. Li, and N. Wang, “Simpletrack: Understanding and rethinking 3d multi-object tracking,” arXiv preprint arXiv:2111.09621, 2021.

[82] Q. Wang, Y . Chen, Z. Pang, N. Wang, and Z. Zhang, “Immortal tracker: Tracklet never dies,” arXiv preprint arXiv:2111.13672, 2021.

[83] A. Kundu, Y . Li, and J. M. Rehg, “3d-rcnn: Instance-level 3d object reconstruction via render-and-compare,” in CVPR, 2018, pp. 3559– 3568.

[84] H.-N. Hu, Y .-H. Yang, T. Fischer, T. Darrell, F. Y u, and M. Sun, “Monocular quasi-dense 3d object tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.

[85] T. Zhang, X. Chen, Y . Wang, Y . Wang, and H. Zhao, “Mutr3d: A multicamera tracking framework via 3d-to-2d queries,” in CVPR, 2022, pp.4537–4546.

[86] J. Yang, E. Y u, Z. Li, X. Li, and W. Tao, “Quality matters: Embracing quality clues for robust 3d multi-object tracking,” arXiv preprint arXiv:2208.10976, 2022.

[87] B. Shuai, A. Berneshawi, X. Li, D. Modolo, and J. Tighe, “Siammot: Siamese multi-object tracking,” in CVPR, 2021, pp. 12 372–12 382.

[88] Y . Zhang, C. Wang, X. Wang, W. Liu, and W. Zeng, “V oxeltrack: Multi-person 3d human pose estimation and tracking in the wild,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.

[89] X. Zhou, T. Yin, V . Koltun, and P . Kr?henbühl, “Global tracking transformers,” in CVPR, 2022, pp. 8771–8780.

[90] Z. Xu, W. Yang, W. Zhang, X. Tan, H. Huang, and L. Huang, “Segment as points for efficient and effective online multi-object tracking and segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6424–6437, 2021.

[91] H. W. Kuhn, “The hungarian method for the assignment problem,” Naval research logistics quarterly, vol. 2, no. 1-2, pp. 83–97, 1955.

[92] A. V aswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, ?. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems, 2017, pp. 5998–6008.

[93] Z. Zhao, Z. Wu, Y . Zhuang, B. Li, and J. Jia, “Tracking objects as pixel-wise distributions,” in ECCV. Springer, 2022, pp. 76–94.

[94] H.-K. Chiu, J. Li, R. Ambrus ?, and J. Bohg, “Probabilistic 3d multimodal, multi-object tracking for autonomous driving,” in ICRA, 2021, pp. 14 227–14 233.

[95] X. Weng, Y . Wang, Y . Man, and K. M. Kitani, “Gnn3dmot: Graph neural network for 3d multi-object tracking with 2d-3d multi-feature learning,” in CVPR, 2020, pp. 6499–6508.

[96] N. Marinello, M. Proesmans, and L. V an Gool, “Triplettrack: 3d object tracking using triplet embeddings and lstm,” in CVPR, 2022, pp. 4500– 4510.

[97] Z. Ge, S. Liu, Z. Li, O. Y oshie, and J. Sun, “Ota: Optimal transport assignment for object detection,” in CVPR, 2021, pp. 303–312.

[98] H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” arXiv preprint arXiv:1710.09412, 2017.

[99] H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. M. Ni, and H.-Y .Shum, “Dino: Detr with improved denoising anchor boxes for end-toend object detection,” arXiv preprint arXiv:2203.03605, 2022.

[100] Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” arXiv preprint arXiv:2103.14030, 2021.

[101] Y . Liu, J. Yan, F. Jia, S. Li, Q. Gao, T. Wang, X. Zhang, and J. Sun, “Petrv2: A unified framework for 3d perception from multi-camera images,” arXiv preprint arXiv:2206.01256, 2022.

[102] X. Bai, Z. Hu, X. Zhu, Q. Huang, Y . Chen, H. Fu, and C.-L. Tai, “Transfusion: Robust lidar-camera fusion for 3d object detection with transformers,” in CVPR, 2022, pp. 1090–1099.

[103] H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” in CVPR, 2019, pp. 658–666.

[104] S. Shao, Z. Zhao, B. Li, T. Xiao, G. Y u, X. Zhang, and J. Sun, “Crowdhuman: A benchmark for detecting human in a crowd,” arXiv preprint arXiv:1805.00123, 2018.

[105] S. Zhang, R. Benenson, and B. Schiele, “Citypersons: A diverse dataset for pedestrian detection,” in CVPR, 2017, pp. 3213–3221.

[106] A. Ess, B. Leibe, K. Schindler, and L. V an Gool, “A mobile vision system for robust multi-person tracking,” in CVPR. IEEE, 2008, pp.1–8.

[107] P . Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taixé, “Mot20: A benchmark for multi object tracking in crowded scenes,” arXiv preprint arXiv:2003.09003, 2020.

[108] W. Lin, H. Liu, S. Liu, Y . Li, R. Qian, T. Wang, N. Xu, H. Xiong, G.-J. Qi, and N. Sebe, “Human in events: A large-scale benchmark for human-centric video analysis in complex events,” arXiv preprint arXiv:2005.04490, 2020.

[109] F. Y u, H. Chen, X. Wang, W. Xian, Y . Chen, F. Liu, V . Madhavan, and T. Darrell, “Bdd100k: A diverse driving dataset for heterogeneous multitask learning,” in CVPR, 2020, pp. 2636–2645.

[110] H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in CVPR, 2020, pp. 11 621–11 631.

[111] K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: the clear mot metrics,” EURASIP Journal on Image and Video Processing, vol. 2008, pp. 1–10, 2008.

[112] E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Performance measures and a data set for multi-target, multi-camera tracking,” in ECCV. Springer, 2016, pp. 17–35.

[113] J. Luiten, A. Osep, P . Dendorfer, P . Torr, A. Geiger, L. Leal-Taixé, and B. Leibe, “Hota: A higher order metric for evaluating multi-object tracking,” International journal of computer vision, vol. 129, no. 2, pp.548–578, 2021.

[114] T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P . Perona, D. Ramanan, P . Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in ECCV. Springer, 2014, pp. 740–755.

[115] P . Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. V enkatesh et al, “Mixed precision training,” arXiv preprint arXiv:1710.03740, 2017.

[116] Z. Wang, H. Zhao, Y .-L. Li, S. Wang, P . H. Torr, and L. Bertinetto, “Do different tracking tasks require different appearance models?” arXiv preprint arXiv:2107.02156, 2021.

[117] L. He, X. Liao, W. Liu, X. Liu, P . Cheng, and T. Mei, “Fastreid: A pytorch toolbox for general instance re-identification,” arXiv preprint arXiv:2006.02631, 2020.

[118] Y . Lee and J. Park, “Centermask: Real-time anchor-free instance segmentation,” in CVPR, 2020, pp. 13 906–13 915.

[119] D. Park, R. Ambrus, V . Guizilini, J. Li, and A. Gaidon, “Is pseudolidar needed for monocular 3d object detection?” in ICCV, 2021, pp.3142–3152.

[120] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017.

[121] B. Pang, Y . Li, Y . Zhang, M. Li, and C. Lu, “Tubetk: Adopting tubes to track multi-object in a one-step training model,” in CVPR, 2020, pp.6308–6318.

[122] Y . Xu, Y . Ban, G. Delorme, C. Gan, D. Rus, and X. AlamedaPineda, “Transcenter: Transformers with dense queries for multipleobject tracking,” arXiv preprint arXiv:2103.15145, 2021.

[123] F. Yang, X. Chang, S. Sakti, Y . Wu, and S. Nakamura, “Remot: A model-agnostic refinement for multiple object tracking,” Image and Vision Computing, vol. 106, p. 104091, 2021.

[124] E. Bochinski, V . Eiselein, and T. Sikora, “High-speed tracking-bydetection without using image information,” in A VSS. IEEE, 2017, pp. 1–6.

[125] M. Chaabane, P . Zhang, J. R. Beveridge, and S. O’Hara, “Deft: Detection embeddings for tracking,” arXiv preprint arXiv:2102.02267, 2021.

[126] P . Li and J. Jin, “Time3d: End-to-end joint monocular 3d object detection and tracking for autonomous driving,” in CVPR, 2022, pp.3885–3894.

[127] Y . Shi, J. Shen, Y . Sun, Y . Wang, J. Li, S. Sun, K. Jiang, and D. Yang, “Srcn3d: Sparse r-cnn 3d surround-view camera object detection and tracking for autonomous driving,” arXiv preprint arXiv:2206.14451, 2022.

[128] Y . Li, Y . Chen, X. Qi, Z. Li, J. Sun, and J. Jia, “Unifying voxel-based representation with transformer for 3d object detection,” arXiv preprint arXiv:2206.00630, 2022.

[129] J.-N. Zaech, A. Liniger, D. Dai, M. Danelljan, and L. V an Gool, “Learnable online graph representations for 3d multi-object tracking,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 5103–5110, 2022.

[130] A. Kim, G. Brasó, A. Oˇsep, and L. Leal-Taixé, “Polarmot: How far can geometric relations take us in 3d multi-object tracking?” in ECCV.Springer, 2022, pp. 41–58.

[131] F. Meyer, T. Kropfreiter, J. L. Williams, R. Lau, F. Hlawatsch, P . Braca, and M. Z. Win, “Message passing algorithms for scalable multitarget tracking,” Proceedings of the IEEE, vol. 106, no. 2, pp. 221–259, 2018.

[132] J. Liu, L. Bai, Y . Xia, T. Huang, B. Zhu, and Q.-L. Han, “Gnn-pmb: A simple but effective online 3d multi-object tracker without bells and whistles,” arXiv preprint arXiv:2206.10255, 2022.文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-744997.html

到了這里，關(guān)于深度學(xué)習(xí)論文分享（一）ByteTrackV2: 2D and 3D Multi-Object T racking by Associating Every Detection Box的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！