PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstruction

Abstract

We propose a novel 3D spatial representation for data fusion and scene reconstruction. Probabilistic Signed Distance Function (Probabilistic SDF, PSDF) is proposed to depict uncertainties in the 3D space. It is modeled by a joint distribution describing SDF value and its inlier probability, reflecting input data quality and surface geometry. A hybrid data structure involving voxel, surfel, and mesh is designed to fully exploit the advantages of various prevalent 3D representations. Connected by PSDF, these components reasonably cooperate in a consistent framework. Given sequential depth measurements, PSDF can be incrementally refined with less ad hoc parametric Bayesian updating. Supported by PSDF and the efficient 3D data representation, high-quality surfaces can be extracted on-the-fly, and in return contribute to reliable data fusion using the geometry information. Experiments demonstrate that our system reconstructs scenes with higher model quality and lower redundancy, and runs faster than existing online mesh generation systems.

我們提出了一種用于數(shù)據(jù)融合和場景重建的新穎 3D 空間表示。概率符號(hào)距離函數(shù)（概率SDF，PSDF）被提出來描述3D空間中的不確定性。它通過描述 SDF 值及其內(nèi)點(diǎn)概率的聯(lián)合分布建模，反映輸入數(shù)據(jù)質(zhì)量和表面幾何形狀。涉及體素、面元和網(wǎng)格的混合數(shù)據(jù)結(jié)構(gòu)旨在充分利用各種流行 3D 表示的優(yōu)勢。這些組件通過 PSDF 連接，在一致的框架中合理地協(xié)作。給定連續(xù)的深度測量，PSDF 可以通過較少的臨時(shí)參數(shù)貝葉斯更新來逐步細(xì)化。在 PSDF 和高效 3D 數(shù)據(jù)表示的支持下，可以即時(shí)提取高質(zhì)量的表面，從而利用幾何信息實(shí)現(xiàn)可靠的數(shù)據(jù)融合。實(shí)驗(yàn)表明，我們的系統(tǒng)以更高的模型質(zhì)量和更低的冗余重建場景，并且比現(xiàn)有的在線網(wǎng)格生成系統(tǒng)運(yùn)行得更快。

1 Introduction

In recent years, we have witnessed the appearance of consumer-level depth sensors and the increasing demand of real-time 3D geometry information in nextgeneration applications. Therefore, online dense scene reconstruction has been a popular research topic. The essence of the problem is to fuse noisy depth data stream into a reliable 3D representation where clean models can be extracted. It is necessary to consider uncertainty in terms of sampling density, measurement accuracy, and surface complexity so as to better understand the 3D space.
Many representations built upon appropriate mathematical models are designed for robust data fusion in such a context. To handle uncertainties, surfel and point based approaches [29,13,15] adopt filtering-based probabilistic models that explicitly manipulate input data. Volume based methods [20,12,5,27], on the other hand, maximize spatial probabilistic distributions and output discretized 3D properties such as SDF and occupancy state. With fixed topologies, mesh based methods [35] may also involve parametric minimization of error functions.

近年來，我們見證了消費(fèi)級(jí)深度傳感器的出現(xiàn)以及下一代應(yīng)用中對實(shí)時(shí) 3D 幾何信息的需求不斷增長。因此，在線密集場景重建一直是熱門的研究課題。問題的本質(zhì)是將嘈雜的深度數(shù)據(jù)流融合到可靠的 3D 表示中，從而可以提取干凈的模型。有必要考慮采樣密度、測量精度和表面復(fù)雜性方面的不確定性，以便更好地理解 3D 空間。
許多基于數(shù)學(xué)模型的表示都是為了在這種情況下進(jìn)行穩(wěn)健的數(shù)據(jù)融合而設(shè)計(jì)的。為了處理不確定性，面元和基于點(diǎn)的方法[29,13,15]采用基于過濾的概率模型來顯式地操縱輸入數(shù)據(jù)。另一方面，基于體積的方法 [20,12,5,27] 最大化空間概率分布并輸出離散 3D 屬性，例如 SDF 和占用狀態(tài)。對于固定拓?fù)?，基于網(wǎng)格的方法[35]也可能涉及誤差函數(shù)的參數(shù)最小化。

While such representations have been proven effective by various applications, their underlying data structures endure more or less drawbacks. Surfels and points are often loosely managed without topology connections, requiring additional modules for efficient indexing and rendering [1], and is relatively prone to noisy input. Volumetric grids lack flexibility to some extent, hence corresponding data fusion can be either oversimplified using weighted average [20], or much time-consuming in order to maximize joint distributions [27]. In addition, ray-casting based volume rendering is also non-trivial. Usually storing vertices with strong topological constraints, mesh is similarly hard to manipulate and is less applicable to many situations. There have been studies incorporating aforementioned data structures [18,24,14,23], yet most of these pipelines are loosely organized without fully taking the advantages of each representation.

雖然這種表示形式已被各種應(yīng)用程序證明是有效的，但它們的底層數(shù)據(jù)結(jié)構(gòu)或多或少存在缺陷。面元和點(diǎn)通常是松散管理的，沒有拓?fù)溥B接，需要額外的模塊來進(jìn)行有效的索引和渲染[1]，并且相對容易產(chǎn)生噪聲輸入。體積網(wǎng)格在某種程度上缺乏靈活性，因此相應(yīng)的數(shù)據(jù)融合要么使用加權(quán)平均值過于簡化[20]，要么為了最大化聯(lián)合分布而耗費(fèi)大量時(shí)間[27]。此外，基于光線投射的體積渲染也很重要。網(wǎng)格通常存儲(chǔ)具有強(qiáng)拓?fù)浼s束的頂點(diǎn)，同樣難以操作并且不太適用于許多情況。已經(jīng)有研究結(jié)合了上述數(shù)據(jù)結(jié)構(gòu)[18,24,14,23]，但大多數(shù)這些管道都是松散組織的，沒有充分利用每種表示的優(yōu)點(diǎn)。

In this paper, we design a novel framework to fully exploit the power of existing 3D representations, supported by PSDF-based probabilistic computations. Our framework is able to perform reliable depth data fusion and reconstruct high-quality surfaces in real-time with more details and less noise, as depicted in Fig.1. Our contributions can be concluded as:
1. We present a novel hybrid data structure integrating voxel, surfel, and mesh;
2. The involved 3D representations are systematically incorporated in the consistent probabilistic framework linked by the proposed PSDF;
3. Incremental 3D data fusion is built upon less ad-hoc probabilistic computations in a parametric Bayesian updating fashion, contributes to online surface reconstruction, and benefits from iteratively recovered geometry in return.

在本文中，我們設(shè)計(jì)了一個(gè)新穎的框架，以充分利用現(xiàn)有 3D 表示的能力，并由基于 PSDF 的概率計(jì)算支持。我們的框架能夠執(zhí)行可靠的深度數(shù)據(jù)融合，并實(shí)時(shí)重建高質(zhì)量的表面，細(xì)節(jié)更多，噪聲更少，如圖 1 所示。我們的貢獻(xiàn)可以總結(jié)為：

1. 我們提出了一種集成體素、面元和網(wǎng)格的新型混合數(shù)據(jù)結(jié)構(gòu)；
2. 所涉及的 3D 表示被系統(tǒng)地納入所提出的 PSDF 鏈接的一致概率框架中；
3. 增量 3D 數(shù)據(jù)融合以參數(shù)貝葉斯更新方式建立在較少的臨時(shí)概率計(jì)算之上，有助于在線表面重建，并從迭代恢復(fù)的幾何圖形中受益。

圖 1.burghers的重建網(wǎng)格。熱圖表示 SDF 內(nèi)點(diǎn)比率（SDF 置信度）。注意細(xì)節(jié)被保留，異常值已被全部刪除，無需任何后處理。圖 4、5、6 中的內(nèi)點(diǎn)比熱圖也符合此顏色條。

2 Related Work

3 Overview

Our framework is based on the hybrid data structure involving three 3D representations linked by PSDF. The pipeline consists of iterative operations of data fusion and surface generation.

我們的框架基于混合數(shù)據(jù)結(jié)構(gòu)，涉及由 PSDF 鏈接的三個(gè) 3D 表示。該管道由數(shù)據(jù)融合和表面生成的迭代操作組成。

3.1 Hybrid Data Structure

We follow [22,14,6] and use a spatial hashing based structure to efficiently manage the space. A hash entry would point to a block, which is the smallest unit to allocate and free. A block is further divided into 8 × 8 × 8 small voxels. Following [6] we consider a voxel as a 3-edge structure instead of merely a cube, as depicted in Fig.2(a), which will avoid ambiguity when we refer to shared edges. PSDF values are stored at the corners of these structures. In addition, we maintain surfels on the volumetric grids by limiting their degree of freedom on the edges of voxels; within a voxel at most 3 surfels on edges could be allocated. This constraint would regularize the distribution of surfels, guarantee easier access, and avoid duplicate allocation. Triangles are loosely organized in the level of blocks, linking adjacent surfels. In the context of mesh, a surfel could be also interpreted as a triangle vertex.

我們遵循[22,14,6]并使用基于空間哈希的結(jié)構(gòu)來有效地管理空間。哈希條目將指向一個(gè)塊，這是分配和釋放的最小單元。一個(gè)塊被進(jìn)一步分為 8 × 8 × 8 個(gè)小體素。在[6]之后，我們將體素視為3邊結(jié)構(gòu)，而不僅僅是一個(gè)立方體，如圖2（a）所示，這將避免當(dāng)我們引用共享邊時(shí)產(chǎn)生歧義。 PSDF 值存儲(chǔ)在這些結(jié)構(gòu)的拐角處。此外，我們通過限制體素邊緣的自由度來維持體積網(wǎng)格上的面元；在一個(gè)體素內(nèi)最多可以分配邊緣上的 3 個(gè)面元。此約束將規(guī)范面元的分布，保證更容易訪問，并避免重復(fù)分配。三角形在塊的水平上松散地組織，連接相鄰的面元。在網(wǎng)格的上下文中，面元也可以解釋為三角形頂點(diǎn)。

圖 2. 基本混合單元和系統(tǒng)管道。

3.2 3D Representations

Voxel and PSDF.
In most volumetric reconstruction systems SDF or truncated SDF (TSDF) (denoted by D) of a voxel is updated when observed 3D points fall in its neighbor region. Projective signed distances from measurements, which could be explained as SDF observations, are integrated by computing weight average. Newcombe [19] suggests that it can be regarded as the solution of a maximum likelihood estimate of a joint Gaussian distribution taking SDF as a random variable. While Gaussian distribution could depict the uncertainty of data noise, it might fail to handle outlier inputs which are common in reconstruction tasks using consumer-level sensors. Moreover, SDF should depict the projective distance from a voxel to its closest surface. During integration, however, it is likely that non-closest surface points are taken into account, which should also be regarded as outlier SDF observations. In view of this, we introduce another random variable $\pi$ to denote the inlier ratio of SDF, initially used in [29] to model the inlier ratio of 3D points:

在大多數(shù)體積重建系統(tǒng)中，當(dāng)觀察到的 3D 點(diǎn)落入其鄰近區(qū)域時(shí)，SDF 或截?cái)?SDF (TSDF)（用 D 表示）就會(huì)更新。測量的投影有符號(hào)距離（可以解釋為 SDF 觀測值）通過計(jì)算權(quán)重平均值進(jìn)行積分。 Newcombe [19]認(rèn)為它可以被視為以 SDF 作為隨機(jī)變量的聯(lián)合高斯分布的最大似然估計(jì)的解。雖然高斯分布可以描述數(shù)據(jù)噪聲的不確定性，但它可能無法處理使用消費(fèi)者級(jí)傳感器的重建任務(wù)中常見的異常輸入。此外，SDF 應(yīng)該描述從體素到其最近表面的投影距離。然而，在積分過程中，很可能會(huì)考慮非最近的表面點(diǎn)，這也應(yīng)被視為離群 SDF 觀測值。鑒于此，我們引入另一個(gè)隨機(jī)變量 $\pi$ 來表示 SDF 的內(nèi)點(diǎn)比率，最初在[29]中用于建模 3D 點(diǎn)的內(nèi)點(diǎn)比率：

where $D_i^o$ reads an SDF observation computed with depth measurements, ${\tau }_i$ is the variance of the SDF observation, $N$ and $U$ are Gaussian and Uniform distributions.

其中 $D_i^o$ 讀取通過深度測量計(jì)算的 SDF 觀測值，${\tau }_i$ 是 SDF 觀測值的方差，$N$ 和 $U$ 是高斯分布和均勻分布。
Following [29], the posterior of PSDF can be parameterized by a Beta distribution multiplying a Gaussian distribution $B(a,b)N(\mu ;{\sigma }^2)$, given a series of observed input SDF measurements. The details will be discussed in §4.1. The parameters $a,b,\mu ,\sigma$ of the parameterized distribution are maintained per voxel.

根據(jù)[29]，在給定一系列觀察到的輸入 SDF 測量值的情況下，PSDF 的后驗(yàn)可以通過 Beta 分布乘以高斯分布 $B(a,b)N(\mu ;{\sigma }^2)$ 進(jìn)行參數(shù)化。詳細(xì)信息將在第 4.1 節(jié)中討論。每個(gè)體素都保留參數(shù)化分布的參數(shù) $a,b,\mu ,\sigma$ 。

Surfel.
A surfel in our pipeline is formally defined by a position $x$, a normal $n$, and a radius $r$. Since a certain surfel is constrained on an edge in the volume, $x$ is generally an interpolation of 2 adjacent voxel corners.

面元
我們管道中的面元由位置 $x$、法線 $n$ 和半徑 $r$ 定義。由于某個(gè)面元被約束在體積中的邊緣上，因此 $x$ 通常是 2 個(gè)相鄰體素角的插值。

圖 3. 左圖顯示具有多個(gè) SDF 觀測值的 PSDF 分布（$D_1^o$ 可能是內(nèi)值觀測值，$D_2^o$ 可能是離群值）。紅色曲線顯示每個(gè)觀察值的高斯分布。藍(lán)色曲線描繪了 Beta 分布，直觀地表明內(nèi)點(diǎn)觀測值應(yīng)該在 $D_1^o$ 附近。右圖是涉及觀測到的輸入 3D 點(diǎn)和已知面元的 SDF 內(nèi)點(diǎn)比率 ρ 的估計(jì)。

Triangle.
A triangle consists of 3 edges, each linking two adjacent surfels. These surfels can be located in different voxels, even different blocks. In our framework triangles are mainly extracted for rendering; the contained topology information may be further utilized in extensions.

三角形
三角形由 3 條邊組成，每條邊連接兩個(gè)相鄰的面元。這些面元可以位于不同的體素，甚至不同的塊中。在我們的框架中，主要提取三角形用于渲染；所包含的拓?fù)湫畔⒖梢赃M(jìn)一步在擴(kuò)展中使用。

Depth input.
We receive depth measurements from sensors as input, while sensor poses are assumed known. Each observed input depth zo is modeled as a random variable subject to a simple Gaussian distribution:

深度輸入
我們接收來自傳感器的深度測量值作為輸入，同時(shí)假設(shè)傳感器姿態(tài)已知。每個(gè)觀察到的輸入深度 $z^o$ 都被建模為服從簡單高斯分布的隨機(jī)變量：

where $\iota$ can be estimated from a precomputed sensor error model.

其中 $\iota$ 可以根據(jù)預(yù)先計(jì)算的傳感器誤差模型進(jìn)行估計(jì)。

3.3 Pipeline

In general, our system will first generate a set of surfels $S_{t?1}$ in the volumetric PSDF field $P{D}_{t?1}$. Meanwhile, mesh triangle set $T_t$ is also determined by linking reliable surfels in $S_{t?1}$. $S_{t?1}$ explicitly defines the surfaces of the scene, hence can be treated as a trustworthy geometry cue to estimate outlier ratio of the input depth data $z_t^o$ . $P{D}_{t?1}$ is then updated to $P{D}_t$ by fusing evaluated depth data distribution $z_t$ via Bayesian updating. The process will be performed iteratively every time input data come, as depicted in Fig.2(b). We assume the poses of the sensors are known and all the computations are in the world coordinate system.

一般來說，我們的系統(tǒng)將首先在體積 PSDF 場 $PDt?1$ 中生成一組面元 $S_{t?1}$。同時(shí)，網(wǎng)格三角形集合 $T_t$ 也是通過鏈接$S_{t?1}$中的可靠面元來確定的。$S_{t?1}$ 明確定義了場景的表面，因此可以被視為可靠的幾何線索來估計(jì)輸入深度數(shù)據(jù) $z_t^o$的異常值比率。然后通過貝葉斯更新融合評估的深度數(shù)據(jù)分布 $z_t$，將 $P{D}_{t?1}$ 更新為 $P{D}_t$。每次輸入數(shù)據(jù)到來時(shí)，該過程都會(huì)迭代執(zhí)行，如圖2（b）所示。我們假設(shè)傳感器的位姿已知，并且所有計(jì)算都在世界坐標(biāo)系中進(jìn)行。

4 PSDF Fusion and Surface Reconstruction

4.1 PSDF Fusion

Similar to [20], in order to get SDF observations of a voxel $V$ given input 3D points from depth images, we first project $V$ to the depth image to find the projective closest depth measurement $z_i$. Signed distance from V to the input 3D data is defined by

與[20]類似，為了獲得給定深度圖像輸入3D點(diǎn)的體素V的SDF觀測值，我們首先將 $V$ 投影到深度圖像以找到投影最接近的深度測量$z_i$。從 $V$ 到輸入 3D 數(shù)據(jù)的有符號(hào)距離定義為

where $z^V$ is a constant value, the projective depth of V along the scanning ray.

其中$z^V$是一個(gè)常數(shù)值，即$V$沿掃描射線的投影深度。

The observed $D_i^o$ is affected by the variance ${\iota }_i$ of $z_i$ in Eq.2 contributing to the Gaussian distribution component in Eq.1, provided $D_i$ is an inlier. Otherwise, $D_i$ would be counted in the uniform distribution part. Fig.3(a) illustrates the possible observations of SDF in one voxel.
Variance${\iota }_i$ can be directly estimated by pre-measured sensor priors such as proposed in [21]. In this case, due to the simple linear form in Eq.3, we can directly set $D_i^o=z_i^o?z^V$ and ${\tau }_i={\iota }_i$ in Eq.1.
Given a series of independent observations $D_i^o$, we can derive the posterior

假設(shè) $D_i$ 是內(nèi)點(diǎn)，觀測到的 $D_i^o$會(huì)受到方程 2 中 $z_i$ 的方差 ${\iota }_i$ 的影響，該方差對方程 1 中的高斯分布分量有貢獻(xiàn)。否則，$D_i$將被計(jì)入均勻分布部分。圖 3(a) 說明了一個(gè)體素中 SDF 的可能觀察結(jié)果。
方差 ${\iota }_i$可以通過預(yù)先測量的傳感器先驗(yàn)直接估計(jì)，如[21]中提出的。在這種情況下，由于方程3中的簡單線性形式，我們可以直接在方程1中設(shè)置$D_i^o=z_i^o?z^V$和 ${\tau }_i={\iota }_i$。
給定一系列獨(dú)立觀察 $D_i^o$ ，我們可以得出后驗(yàn)

where $p(D,\pi )$ is a prior and $p(D_i^o|D,{\tau }_i^2,\pi )$ is defined by Eq.1. It would be intractable to evaluate the production of such distributions with additions. Fortunately, [29] proved that the posterior could be approximated by a parametric joint distribution:

其中 $p(D,\pi )$ 是先驗(yàn)，$p(D_i^o|D,{\tau }_i^2,\pi )$ 由方程 1 定義。要評估這種有附加值的分布是很困難的。幸運(yùn)的是，[29]證明后驗(yàn)可以通過參數(shù)聯(lián)合分布來近似：

therefore the problem could be simplified as a parameter estimation in an incremental fashion:

因此，問題可以簡化為增量方式的參數(shù)估計(jì)：

In [29] by equating first and second moments of the random variables $\pi$ and $D$, the parameters could be easily updated, in our case evoking the change of SDF distribution and its inlier probability:

在 [29] 中，通過等效隨機(jī)變量 $\pi$ 和 $D$ 的第一矩和第二矩，可以很容易地更新參數(shù)，在我們的案例中，引起了 SDF 分布及其異常值概率的變化：：

the computation of $a$ and $b$ are the same as [29] hence ignored here. In our experiments we find that a truncated $D_i^o$ leads to better results, as it directly rejects distant outliers. SDF observations from non-closest surfaces are left to be handled by PSDF.

$a$ 和 $b$ 的計(jì)算與[29]相同，因此這里忽略。在我們的實(shí)驗(yàn)中，我們發(fā)現(xiàn)截?cái)嗟?$D_i^o$ 會(huì)帶來更好的結(jié)果，因?yàn)樗苯泳芙^遠(yuǎn)處的異常值。來自非最近表面的 SDF 觀測結(jié)果由 PSDF 處理。

4.2 Inlier Ratio Evaluation

In Eq.11-12, the expectation of $B(\pi |a,b)$ is used to update the coefficients, failing to make full use of known geometry properties in scenes. In our pipeline, available surface geometry is considered to evaluate the inlier ratio ${\rho }_n$ of $D_n^o$, replacing the simple $\frac{a_{n?1}}{a_{n?1}+b_{n?1}}$ . Note ${\rho }_n$ is computed per-frame in order to update $C_1,C_2$; $\pi$ is still parameterized by $a$ and $b$.

式11-12中，利用 $B(\pi |a,b)$ 的期望來更新系數(shù)，未能充分利用場景中已知的幾何特性。在我們的流程中，考慮可用的表面幾何形狀來評估 $D_n^o$ 的內(nèi)點(diǎn)比率 ${\rho }_n$ ，取代簡單的 $\frac{a_{n?1}}{a_{n?1}+b_{n?1}}$ 。注意 ${\rho }_n$ 是逐幀計(jì)算的，以便更新 $C_1、C_2$； $\pi$ 仍然由 $a$ 和 $b$ 參數(shù)化。

${\rho }_n$ can be determined by whether an input point $z$ is near the closest surface of a voxel and results in an inlier SDF observation. We first cast the scanning ray into the volume and collect the surfels maintained on the voxels hit by the ray. Given the surfels, 3 heuristics are used, as illustrated in Fig.3(b).

${\rho }_n$可以通過輸入點(diǎn) $z$ 是否靠近體素的最近表面以及是否會(huì)產(chǎn)生離群 SDF 觀察結(jié)果來確定。首先，我們將掃描射線投射到體量中，然后收集射線擊中的體素上的曲面。如圖 3(b)所示，我們使用了 3 種啟發(fā)式方法。

Projective distance.
This factor is used to measure whether a sampled point is close enough to a surfel which is assumed the nearest surface to the voxel:

投影距離
該因子用于測量采樣點(diǎn)是否足夠接近面元，該面元被假定為距離體素最近的表面：

where $v$ is the normalized direction of the ray in world coordinate system and $\theta$ is a preset parameter proportional to the voxel resolution.

其中 $v$ 是世界坐標(biāo)系中光線的歸一化方向，$\theta$ 是與體素分辨率成比例的預(yù)設(shè)參數(shù)。

Angle.
Apart from projective distance, we consider angle as another factor, delineating the possibility that a scanning ray will hit a surfel. We use the empirical angle weight in [15]:

角度
除了投影距離外，我們還將角度作為另一個(gè)因素來考慮，因?yàn)榻嵌葲Q定了掃描光線擊中曲面的可能性。我們使用 [15] 中的經(jīng)驗(yàn)角度權(quán)重：：

where $\alpha =〈n,v〉$, ${\alpha }_{max}$ is set to 80 deg and $w_{angle}^0$ assigned to 0.1.

其中 $\alpha =〈n,v〉$，${\alpha }_{max}$ 設(shè)置為 80 度，$w_{angle}^0$ 角度指定為 0.1。

Radius.
The area that surfels could influence vary, due to the local shape of the surface. The further a point is away from the center of a surfel, the less possible it would be supported. A sigmoid-like function is used to encourage a smooth transition of the weight:

半徑
由于曲面的局部形狀不同，面元所能影響的區(qū)域也不同。一個(gè)點(diǎn)離面元中心越遠(yuǎn)，它獲得支持的可能性就越小。為了使權(quán)重平穩(wěn)過渡，我們使用了一個(gè)類似于 sigmoid 的函數(shù)：

where parameter $\gamma \in [0,1)$ and is set to 0.5 in our case.
Putting all the factors together, we now have

其中參數(shù) $\gamma \in [0,1)$ 在我們的例子中設(shè)置為 0.5。
將所有因素放在一起，我們現(xiàn)在有

To compute the $\rho$ predicted by all the surfels, one may consider either summations or multiplications. However, we choose the highest $\rho$ instead – intuitively a depth measurement is a sample on a surface, corresponding to exactly one surfel. A more sophisticated selection might include a ray consistency evaluation [32,27,28] where occlusion is handled. When a new area is explored where no surfels have been extracted, we use a constant value ${\rho }_{pr}=0.1$ to represent a simple occupancy prior in space, hence we have

為了計(jì)算所有面元預(yù)測的 $\rho$，可以考慮求和或乘法。然而，我們選擇最高的 $\rho$ ——直觀上，深度測量是表面上的樣本，恰好對應(yīng)于一個(gè)面元。更復(fù)雜的選擇可能包括光線一致性評估[32,27,28]，其中會(huì)對遮擋進(jìn)行處理。當(dāng)探索未提取面元的新區(qū)域時(shí)，我們使用常數(shù)值 ${\rho }_{pr}=0.1$ 來表示空間中的簡單占用先驗(yàn)，因此我們有

4.3 Surface Extraction

PSDF implicitly defines zero crossing surfaces and decides whether they are true surfaces. The surface extraction is divided into two steps.

PSDF 隱式定義零交叉曲面并決定它們是否為真實(shí)曲面。表面提取分為兩個(gè)步驟。

Surfel generation.
In this stage we enumerate zero-crossing points upon 3 edges of each voxel and generate surfels when condition are satisfied,

面元生成
在此階段，我們枚舉每個(gè)體素 3 個(gè)邊緣上的零交叉點(diǎn)，并在以下條件滿足時(shí)生成面元

where $i1$ and $i2$ are indices of adjacent voxels and ${\pi }_{thr}$ is a confidence threshold. Supported by the reliable update of the PSDF, false surfaces could be rejected and duplicates could be removed. According to our experiments, our framework is not sensitive to ${\pi }_{thr}$; 0.4 would work for all the testing scenes. A surfel’s position $x$ would be the linear interpolation of corresponding voxels’ positions $x^i$ indexed by $i$, and the radius would be determined by $\sigma$ of adjacent voxels, simulating its affecting area. Normal is set to normalized gradient of the SDF field, as mentioned in [20].

其中 $i1$ 和 $i2$ 是相鄰體素的索引，${\pi }_{thr}$ 是置信度閾值。在 PSDF 可靠更新的支持下，可以抑制錯(cuò)誤表面并刪除重復(fù)表面。根據(jù)我們的實(shí)驗(yàn)，我們的框架對 ${\pi }_{thr}$ 不敏感； 0.4 適用于所有測試場景。面元的位置 $x$ 將是由 $i$ 索引的相應(yīng)體素位置 $x^i$ 的線性插值，半徑將由相鄰體素的 $\sigma$ 確定，模擬其影響區(qū)域。法線設(shè)置為 SDF 場的歸一化梯度，如[20]中所述。

Triangle generation.
Having sufficient geometry information within surfels, there is only one more step to go for rendering-ready mesh. The connections between adjacent surfels are determined by the classical MarchingCubes [17] method. As a simple modification, we reject edges in the voxel whose $\sigma$ is larger than a preset parameter ${\sigma }_{thr}$. This operation will improve the visual quality of reconstructed model while preserving surfels for the prediction stage.

三角形生成
有了曲面內(nèi)足夠的幾何信息，只需再進(jìn)行一個(gè)步驟就可以得到可渲染的網(wǎng)格了。相鄰曲面之間的連接由經(jīng)典的 MarchingCubes [17] 方法確定。作為一個(gè)簡單的修改，我們會(huì)剔除體素中 $\sigma$ 大于預(yù)設(shè)參數(shù) ${\sigma }_{thr}$ 的邊。這一操作將提高重建模型的視覺質(zhì)量，同時(shí)為預(yù)測階段保留曲面。

5 Experiments

We test our framework (denoted by PSDF) on three RGB-D datasets: TUM [25], ICL-NUIM [9], and dataset from Zhou and Koltun [34]. Our method is compared against [6] (denoted by TSDF ) which incrementally extracts mesh in spatial-hashed TSDF volumes. The sensors’ poses are assumed known for these datasets, therefore the results of TSDF should be similar to other stateof-the-art methods such as [22,23] where TSDF integration strategies are the same. We demonstrate that our method reconstructs high quality surfaces by both qualitative and quantitative results. Details are preserved while noise is removed in the output models. The running speed for online mesh extraction is also improved by avoiding computations on false surfel candidates.

我們在三個(gè) RGB-D 數(shù)據(jù)集上測試我們的框架（用 PSDF 表示）：TUM [25]、ICL-NUIM [9] 以及來自 Zhou 和 Koltun 的數(shù)據(jù)集 [34]。我們的方法與 [6]（由 TSDF 表示）進(jìn)行比較，后者增量地提取空間哈希 TSDF 體積中的網(wǎng)格。假設(shè)這些數(shù)據(jù)集的傳感器姿態(tài)已知，因此 TSDF 的結(jié)果應(yīng)該與其他最先進(jìn)的方法類似，例如 [22,23]，其中 TSDF 集成策略是相同的。我們證明我們的方法通過定性和定量結(jié)果重建高質(zhì)量的表面。保留細(xì)節(jié)，同時(shí)消除輸出模型中的噪聲。通過避免對虛假面元候選的計(jì)算，還提高了在線網(wǎng)格提取的運(yùn)行速度。

For [25,34] we choose a voxel size of 8mm and ${\sigma }_{thr}$= 16mm; for [9] voxel size is set to 12mm and ${\sigma }_{thr}$ = 48mm. The truncation distance is set to 3× voxel size plus 3 × $\tau$ ; with a smaller truncation distance we found strides and holes in meshes. Kinect’s error model [21] was adopted to get $\iota$ where the factor of angle was removed, which we think might cause double counting considering wangle in the inlier prediction stage. The program is written in C++/CUDA 8.0 and runs on a laptop with an Intel i7-6700 CPU and an NVIDIA 1070 graphics card.

對于[25,34]，我們選擇體素大小為 8mm，${\sigma }_{thr}$ = 16mm；對于[9]，體素大小設(shè)置為 12mm，${\sigma }_{thr}$ = 48mm。截?cái)嗑嚯x設(shè)置為 3× 體素大小加上 3 × $\tau$ ；使用較小的截?cái)嗑嚯x，我們在網(wǎng)格中發(fā)現(xiàn)了裂縫和孔洞。采用Kinect的誤差模型[21]來得到 $\iota$，其中去除了角度因素，我們認(rèn)為考慮到內(nèi)點(diǎn)預(yù)測階段的角度，這可能會(huì)導(dǎo)致重復(fù)計(jì)算。該程序采用 C++/CUDA 8.0 編寫，在配備 Intel i7-6700 CPU 和 NVIDIA 1070 顯卡的筆記本電腦上運(yùn)行。

圖4. frei3 long office 場景輸出網(wǎng)格對比。左，PSDF。右，TSDF。我們的方法生成對象的光滑表面和干凈的邊緣，特別是在藍(lán)色矩形中。

5.1 Qualitative Results

We first show that PSDF accompanied by the related mesh extraction algorithm produces higher quality surfaces than TSDF. Our results are displayed with shaded heatmap whose color indicates the inlier ratio of related SDF. Both geometry and probability properties can be viewed in such a representation.

我們首先證明 PSDF 結(jié)合相關(guān)的網(wǎng)格提取算法可以產(chǎn)生比 TSDF 更高質(zhì)量的表面。我們的結(jié)果以陰影熱圖顯示，其顏色表示相關(guān) SDF 的內(nèi)點(diǎn)比率。幾何和概率屬性都可以在這樣的表示中查看。

Fig.4 shows that PSDF outperforms TSDF by generating clean boundaries of small objects and rejecting noisy areas on the ground. In Fig.5, in addition to the results of TSDF, we also display the reconstructed mesh from offline methods provided by [34] as references. It appears that our method produces results very similar to [34]. While guaranteeing well-covered reconstruction of scenes, we filter outliers and preserve details. In copyroom, the wires are completely reconstructed, one of which above PC is smoothed out in [34] and only partially recovered by TSDF. In lounge, we can observe a complete shape of table, and de-noised details in the clothes.

圖 4 顯示 PSDF 通過生成小物體的清晰邊界并拒絕地面上的噪聲區(qū)域而優(yōu)于 TSDF。在圖5中，除了TSDF的結(jié)果之外，我們還顯示了[34]提供的離線方法的重建網(wǎng)格作為參考?？磥砦覀兊姆椒óa(chǎn)生的結(jié)果與[34]非常相似。在保證場景重建得到良好覆蓋的同時(shí)，我們過濾異常值并保留細(xì)節(jié)。在copyroom中，電線被完全重建，其中 PC 上方的電線在[34]中被平滑，并且僅由 TSDF 部分恢復(fù)。在lounge，我們可以觀察到桌子的完整形狀，以及衣服上的去噪細(xì)節(jié)。

圖 5. copyroom和lounge場景的輸出網(wǎng)格。從左到右，PSDF、TSDF、網(wǎng)格由[34]提供。放大區(qū)域表明我們的方法能夠過濾異常值，同時(shí)保持完整的模型并保留細(xì)節(jié)。最好以彩色形式觀看。

We also visualize the incremental update of $\pi$ by rendering $E(\beta (\pi |a,b))=a/(a+b)$ as the inlier ratio of reconstructed mesh in sequence using colored heatmap. Fig.6 shows the fluctuation of confidence around surfaces. The complex regions such as fingers and wrinkles on statues are more prone to noise, therefore apparent change of shape along with color can be observed.

我們還通過使用彩色熱圖按順序?qū)?$E(\beta (\pi |a,b))=a/(a+b)$ 渲染為重建網(wǎng)格的內(nèi)點(diǎn)比率來可視化 $\pi$ 的增量更新。圖 6 顯示了表面周圍置信度的波動(dòng)。雕像上的手指和皺紋等復(fù)雜區(qū)域更容易出現(xiàn)噪點(diǎn)，因此可以觀察到形狀和顏色的明顯變化。

圖 6. burghers 數(shù)據(jù)集的增量重建?？梢栽谌菀壮鲥e(cuò)的區(qū)域觀察到概率波動(dòng)，作為不確定性傳播的指示。

5.2 Quantitative Results

Reconstruction Accuracy.
We reconstruct mesh of the synthetic dataset livingroom2 with added noise whose error model is presented in [9]. Gaussian noise on inverse depth plus local offsets is too complicated for our error model, therefore we simplify it by assigning inverse sigma at certain inverse depths to ${\iota }_i$. The mesh vertices are compared with the ground truth point cloud using the free software CloudCompare [3].

重建精度
我們對合成數(shù)據(jù)集 livingroom2 的網(wǎng)格進(jìn)行了重建，并添加了噪聲，其誤差模型見 [9]。反演深度上的高斯噪聲加上局部偏移對于我們的誤差模型來說過于復(fù)雜，因此我們通過將特定反演深度上的反演 sigma 賦值給 ${\iota }_i$ 來簡化誤差模型。使用免費(fèi)軟件 CloudCompare [3]，將網(wǎng)格頂點(diǎn)與地面真實(shí)點(diǎn)云進(jìn)行比較。

Table.1 indicates that PSDF reconstructs better models than TSDF. Further details in Fig.7 suggest that less outliers appear in the model reconstructed by PSDF, leading to cleaner surfaces.

表 1 表明 PSDF 重建的模型比 TSDF 更好。圖 7 中的更多細(xì)節(jié)表明 PSDF 重建的模型中出現(xiàn)的異常值較少，表面更干凈。

表 1 重建模型頂點(diǎn)到地面真實(shí)點(diǎn)云的點(diǎn)對點(diǎn)距離統(tǒng)計(jì)。 PSDF 產(chǎn)生更好的重建精度。

圖 7. PSDF 和 TSDF 的輸出網(wǎng)格質(zhì)量比較。第一行：從重建模型到GT的點(diǎn)對點(diǎn)距離的熱圖。注意紅色矩形中的異常值。第二行，到地面實(shí)況的距離分布。將分布分為兩部分，以強(qiáng)調(diào) TSDF 重建過程中異常值的存在，而 PSDF 則不會(huì)面臨此類問題。最好以彩色形式觀看。

Mesh size.
Our method maintains the simplicity of mesh by reducing false surface candidates caused by noise and outliers. As shown in Fig.8 and Table 2, PSDF in most cases generates less vertices (% 20) than TSDF, most of which are outliers and boundaries with low confidence. Fig.8 shows that the vertex count remains approximately constant when a loop closure occurred in frei3 long office, while the increasing rate is strictly constrained in the lounge sequence where there is no loop closure.

網(wǎng)格尺寸
我們的方法通過減少由噪聲和異常值引起的虛假表面候選來保持網(wǎng)格的簡單性。如圖 8 和表 2 所示，大多數(shù)情況下 PSDF 生成的頂點(diǎn)數(shù)量比 TSDF 少（% 20），其中大部分是離群點(diǎn)和置信度較低的邊界。圖8表明，當(dāng) frei3 long office 中發(fā)生環(huán)路閉合時(shí)，頂點(diǎn)數(shù)保持近似恒定，而在沒有環(huán)路閉合的 lounge 序列中，增加率受到嚴(yán)格限制。

圖 8. (a)-(b)，網(wǎng)格頂點(diǎn)（面元）的增加趨勢。 (a)，frei3 long office 場景，其中在 1500 幀左右發(fā)生循環(huán)閉合。 (b)，沒有循環(huán)閉合的lounge 場景。 PSDF 生成的網(wǎng)格消耗了 TSDF 大約 80% 的內(nèi)存。 ( c)-(f)，我們的方法的運(yùn)行時(shí)間分析以及lounge 場景中的比較方法。由于額外的內(nèi)點(diǎn)預(yù)測階段，PSDF 在 GPU 上慢了 1 毫秒，但會(huì)節(jié)省更多表面提取時(shí)間，總體上速度更快。在線網(wǎng)格劃分的速度比光線投射慢，但相當(dāng)。

表 2. 各種數(shù)據(jù)集上的內(nèi)存和時(shí)間成本評估。 PSDF 通過拒絕虛假表面和噪聲來減少模型的冗余。 TSDF 的映射階段速度更快，但一般來說 PSDF 考慮映射和網(wǎng)格劃分階段花費(fèi)的時(shí)間較少。

Time.
To make a running time analysis, we take real world lounge as a typical sequence where noise is common while the camera trajectory fits scanning behavior of humans. As we have discussed, evaluation of inlier ratio was performed, increasing total time of the fusion stage. However we find on a GPU, even with a relatively high resolution, the average increased time is at the scale of ms (see Fig.8( c) and Table.2) and can be accepted. When we come to meshing, we find that by taking the advantage of PSDF fusion and inlier ratio evaluation, unnecessary computations can be avoided and PSDF method runs faster than TSDF, as plotted in Fig.8(d). The meshing stage is the runtime bottleneck of the approach, in general the saved time compensate for the cost in fusion stage, see Fig.8(e) and Table.2. We also compare the time of meshing to the widely used ray-casting that renders surfaces in real-time. According to Table 2, in some scenes where sensor is close to the surfaces performing scanning, less blocks are allocated in viewing frustum and the meshing speed could be comparative to ray-casting, as illustrated in Fig.8(f). As for other scenes requiring a large scanning range, especially frei3 long office where more blocks in frustum have to be processed, ray-casting shows its advantage. We argue that in applications that only require visualization, ray-casting can be adopted; otherwise meshing offers more information and is still preferable.

時(shí)間
為了進(jìn)行運(yùn)行時(shí)間分析，我們將現(xiàn)實(shí)世界的lounge 作為一個(gè)典型的序列，其中噪聲很常見，而相機(jī)軌跡符合人類的掃描行為。正如我們所討論的，進(jìn)行了內(nèi)點(diǎn)比率的評估，增加了融合階段的總時(shí)間。然而我們發(fā)現(xiàn)在GPU上，即使具有相對較高的分辨率，平均增加的時(shí)間也在ms的量級(jí)（見圖8（c）和表2）并且可以接受。
當(dāng)我們進(jìn)行網(wǎng)格劃分時(shí)，我們發(fā)現(xiàn)通過利用 PSDF 融合和內(nèi)點(diǎn)比率評估的優(yōu)勢，可以避免不必要的計(jì)算，并且 PSDF 方法比 TSDF 運(yùn)行得更快，如圖 8（d）所示。網(wǎng)格劃分階段是該方法的運(yùn)行時(shí)瓶頸，一般來說節(jié)省的時(shí)間補(bǔ)償了融合階段的成本，見圖8（e）和表2。
我們還將網(wǎng)格劃分時(shí)間與廣泛使用的實(shí)時(shí)渲染表面的光線投射時(shí)間進(jìn)行了比較。根據(jù)表2，在一些傳感器靠近執(zhí)行掃描的表面的場景中，在視錐體中分配較少的塊，并且網(wǎng)格劃分速度可以與射線投射相當(dāng)，如圖8（f）所示。對于其他需要大掃描范圍的場景，特別是 frei3 long office，需要處理更多的視錐體塊，光線投射就顯示出了它的優(yōu)勢。我們認(rèn)為，在只需要可視化的應(yīng)用中，可以采用光線投射；否則，網(wǎng)格劃分可以提供更多信息，并且仍然是更好的選擇。

6 Conclusions

We propose PSDF, a joint probabilistic distribution to model the spatial uncertainties and 3D geometries. With the help of Bayesian updating, parameters of such a distribution could be incrementally estimated by measuring the quality of input depth data. Built upon a hybrid data structure, our framework can iteratively generate surfaces from the volumetric PSDF field and update PSDF values through reliable probabilistic data fusion supported by reconstructed surfaces. As an output, high-quality mesh can be generated on-the-fly in real-time with duplicates removed and noise cleared.
In the future we intend to extend our framework upon the basis of PSDF. We plan to establish a hierarchical data structure so that the resolution of spatialhashed blocks could be adaptive to input data according to PSDF distributions. We also work on employing more priors to further enrich PSDF probability distributions. Localization modules using the proposed 3D representation are also planned to be integrated in our framework, in order to construct a complete probabilistic SLAM system.

我們提出 PSDF，一種聯(lián)合概率分布，用于對空間不??確定性和 3D 幾何進(jìn)行建模。借助貝葉斯更新，可以通過測量輸入深度數(shù)據(jù)的質(zhì)量來增量估計(jì)這種分布的參數(shù)。我們的框架建立在混合數(shù)據(jù)結(jié)構(gòu)的基礎(chǔ)上，可以從體積 PSDF 場迭代生成表面，并通過重建表面支持的可靠概率數(shù)據(jù)融合來更新 PSDF 值。作為輸出，可以實(shí)時(shí)生成高質(zhì)量網(wǎng)格，并刪除重復(fù)項(xiàng)和清除噪聲。

未來我們打算在 PSDF 的基礎(chǔ)上擴(kuò)展我們的框架。我們計(jì)劃建立一個(gè)分層數(shù)據(jù)結(jié)構(gòu)，以便空間散列塊的分辨率可以根據(jù) PSDF 分布自適應(yīng)輸入數(shù)據(jù)。我們還致力于采用更多先驗(yàn)來進(jìn)一步豐富 PSDF 概率分布。使用所提出的 3D 表示的定位模塊也計(jì)劃集成到我們的框架中，以構(gòu)建完整的概率 SLAM 系統(tǒng)。

拋磚：文章來源地址http://www.zghlxwxcb.cn/news/detail-775142.html

1. “PSDF，一種聯(lián)合概率分布，用于對空間不??確定性和 3D 幾何進(jìn)行建模?！薄?這正是目前各類3D重建方法想做到的，尤其是不確定評估這塊兒，很多現(xiàn)有方法都要通過GT數(shù)據(jù)進(jìn)行評估，當(dāng)然這是最可靠的評估手段。但是GT數(shù)據(jù)不總已知，且很多真實(shí)場景的GT數(shù)據(jù)也是通過激光掃描或其他手段獲取的，雖然可以視為GT，但真能做到GT嗎，顯然未必。
2. “使用基于空間哈希的結(jié)構(gòu)來有效地管理空間?！?/li>
3. “將體素視為3邊結(jié)構(gòu)，而不僅僅是一個(gè)立方體，這將避免當(dāng)我們引用共享邊時(shí)產(chǎn)生歧義?！?/li>
4. 是否可以通過深度學(xué)習(xí)的方式來構(gòu)建PSDF呢？目前似乎沒有看到相關(guān)工作。TSDF倒是有做了的。
5. 此文是2018年的工作(可惜沒有開源，不然還真想試試看效果如何)，應(yīng)該有不少相關(guān)跟進(jìn)工作，可以跟蹤跟蹤。

到了這里，關(guān)于【論文閱讀】PSDF Fusion：用于動(dòng)態(tài) 3D 數(shù)據(jù)融合和場景重建的概率符號(hào)距離函數(shù)的文章就介紹完了。如果您還想了解更多內(nèi)容，請?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！