Title?

題目

Combined Spiral Transformation and Model-Driven Multi-Modal Deep Learning Scheme for Automatic Prediction of TP53 Mutation in Pancreatic Cancer

螺旋變換與模型驅(qū)動的多模態(tài)深度學習方案相結(jié)合，用于自動預測胰腺癌中TP53突變

01

文獻速遞介紹

胰腺癌是最致命的惡性腫瘤之一，其特點是診斷晚、死亡率高、總生存率低?；颊叩奈迥晟媛实陀?.5%，其中75%的患者表現(xiàn)出TP53基因突變。作為一個腫瘤抑制基因，TP53用于編碼P53蛋白，該蛋白可以在許多細胞過程中抑制細胞增殖。突變的TP53基因促進腫瘤細胞的增殖、侵襲和存活。

特定的TP53突變與預后不良和對治療方式的抗性有關(guān)。針對胰腺癌的靶向治療正在研究中，而TP53可能是胰腺癌基因增強治療中一個有吸引力的靶點。因此，TP53的狀態(tài)可能在治療選擇中起著至關(guān)重要的作用。進行手術(shù)和活組織檢查以進行突變分析是標準的醫(yī)療程序；然而，由于臨床實踐中的侵入性操作，它們不能頻繁使用。因此，開發(fā)非侵入性方法來預測腫瘤基因突變是必要的。

近年來，醫(yī)學影像被用于非侵入性評估活體組織中的遺傳變化。組織的生物醫(yī)學成像結(jié)果反映了潛在的病理生理特征。特別是，影像中的定量參數(shù)，如腫瘤的強度、形狀、大小、體積和紋理，提供了關(guān)于腫瘤表型和微環(huán)境的重要信息。

放射組學方法在非侵入性評估遺傳變化方面取得了有希望的結(jié)果。例如，Eran等在CT影像和原發(fā)性人類肝癌的基因表達之間獲得了相關(guān)性；Coudray等使用肺癌的組織病理學影像提取直方圖并預測如TP53等基因突變。研究提取了部分放射組學特征，如形狀、紋理和密度，然后選定特征并建立機器學習模型來預測基因突變。

Abstract-Background?

摘要

Pancreatic cancer is a malignant form of cancer with one of the worst prognoses. The poor prog nosis and resistance to therapeutic modalities have been linked to TP53 mutation. Pathological examinations, such as biopsies, cannot be frequently performed in clinical practice; therefore, noninvasive and reproducible methods are desired. However, automatic prediction methods based on imaging have drawbacks such as poor 3D information utilization, small sample size, and ineffectiveness multi modal fusion. In this study, we proposed a model-driven multi-modal deep learning scheme to overcome these chal lenges. A spiral transformation algorithm was developed to obtain 2D images from 3D data, with the transformed image inheriting and retaining the spatial correlation of the?original texture and edge information. The spiral transfor mation could be used to effectively apply the 3D informa tion with less computational resources and conveniently augment the data size with high quality. Moreover, model driven items were designed to introduce prior knowledge in the deep learning framework for multi-modal fusion. The model-driven strategy and spiral transformation-based data augmentation can improve the performance of the small sample size. A bilinear pooling module was introduced to improve the performance of fine-grained prediction. The experimental results show that the proposed model gives the desired performance in predicting TP53 mutation in pancreatic cancer, providing a new approach for noninva sive gene prediction. The proposed methodologies of spiral?transformation and model-driven deep learning can also be used for the artificial intelligence community dealing with oncological applications.?

胰腺癌是一種惡性癌癥，預后極差。預后不良和對治療方式的抗性與TP53突變有關(guān)。如活組織檢查等病理學檢查不能在臨床實踐中頻繁進行；因此，需要非侵入性且可重復的方法。然而，基于影像的自動預測方法存在缺陷，如3D信息利用不足、樣本量小、多模態(tài)融合無效等。在本研究中，我們提出了一種模型驅(qū)動的多模態(tài)深度學習方案，以克服這些挑戰(zhàn)。開發(fā)了一種螺旋變換算法，從3D數(shù)據(jù)中獲得2D圖像，轉(zhuǎn)換后的圖像繼承并保留了原始紋理和邊緣信息的空間相關(guān)性。螺旋變換可以有效地利用3D信息，減少計算資源消耗，并方便地用高質(zhì)量增加數(shù)據(jù)大小。此外，設計了模型驅(qū)動項目，在深度學習框架中引入先驗知識進行多模態(tài)融合。模型驅(qū)動策略和基于螺旋變換的數(shù)據(jù)增強可以提高小樣本量的性能。引入了雙線性池化模塊，以提高細粒度預測的性能。實驗結(jié)果表明，所提出的模型在預測胰腺癌中TP53突變方面提供了期望的性能，為非侵入性基因預測提供了一種新方法。螺旋變換和模型驅(qū)動深度學習的提出方法也可以用于處理腫瘤應用的人工智能社區(qū)。

CONCLUSION

結(jié)論

In this study, we proposed a model-driven multi-modal deep learning model for the automated prediction of TP53 mutation in pancreatic cancer based on a small amount of data. A spiral transformation method was developed to apply 3D information with less computational resources and effectively augment the data. Model-driven items are also proposed to use the prior knowledge in the deep learning framework for fusing multi modal information and improve the performance of the small sample size. A bilinear module was introduced in the model?driven model to improve the performance of fine-grained prediction. Extensive experiments confirmed the performance and robustness of our proposed in predicting the TP53 muta tion in pancreatic cancer. The proposed methodologies for the utilization of 3D information with small sample size and effective multi-modal fusion are potential paradigms for medical imaging analyses.

在本研究中，我們提出了一種模型驅(qū)動的多模態(tài)深度學習模型，用于基于少量數(shù)據(jù)自動預測胰腺癌中TP53突變。開發(fā)了一種螺旋變換方法，以較少的計算資源應用3D信息并有效增強數(shù)據(jù)。還提出了使用模型驅(qū)動項目，在深度學習框架中利用先驗知識融合多模態(tài)信息，并改善小樣本量的性能。在模型驅(qū)動模型中引入了一個雙線性模塊，以提高細粒度預測的性能。廣泛的實驗確認了我們提出的在預測胰腺癌TP53突變方面的性能和魯棒性。提出的方法論，對于小樣本量下3D信息的利用和有效的多模態(tài)融合，是醫(yī)學影像分析的潛在范式。

Method

方法

A. Data Augmentation

The most common data augmentation method for

images is geometric transformation. Rajpurkar [18] and Valvano et al. [19] performed horizontal flip, rotation, scaling, and other transformations on 2D images to increase the amount of data. Zhao et al. [20] translated by small amounts, zoomed in on small-range multiples , and rotated original data while extracting 3D patches. In the training process, real time data augmentation methods have a strong regularization effect on the model. Guan et al.? performed a random selection without changing the total amount of data for the above-mentioned geometric transformation; they consequently improved the robustness of the network. Alex? proposed principal component analysis (PCA) jittering to perform fea ture decomposition on each channel of the RGB to obtain feature vectors and feature values and then alter the intensity of each channel. Color jittering, a method similar to PCA jittering, can be used to change the contrast and brightness of an image. Geometric transformation methods can increase the model’s robustness to some extent. However, the amount of information in the data before and after the geometric trans formation is largely similar, thus limiting the augmentationeffect.A.?

數(shù)據(jù)增強

對圖像來說，最常見的數(shù)據(jù)增強方法是幾何變換。Rajpurkar [18] 和 Valvano 等人[19] 對2D圖像執(zhí)行了水平翻轉(zhuǎn)、旋轉(zhuǎn)、縮放和其他變換，以增加數(shù)據(jù)量。趙等人[20]通過小幅度平移、在小范圍內(nèi)放大（[0.8,1.15]倍）以及在提取3D補丁時旋轉(zhuǎn)原始數(shù)據(jù)。在訓練過程中，實時數(shù)據(jù)增強方法對模型有很強的正則化效果。關(guān)等人[21]對上述幾何變換進行了隨機選擇，不改變數(shù)據(jù)的總量，從而提高了網(wǎng)絡的魯棒性。Alex [22]提出了主成分分析（PCA）抖動，對RGB的每個通道進行特征分解，以獲得特征向量和特征值，然后改變每個通道的強度。類似于PCA抖動的顏色抖動方法，可以用來改變圖像的對比度和亮度。幾何變換方法在某種程度上可以增加模型的魯棒性。然而，幾何變換前后數(shù)據(jù)中的信息量大體相似，從而限制了增強效果。

Figure

圖

Fig. 1. Examples of pancreatic cancer dataset. The first column of each modal image is the original data, and the second column is the enlarged region. Pancreatic cancers have small size and unclear boundaries.

圖 1. 胰腺癌數(shù)據(jù)集的示例。每個模態(tài)圖像的第一列是原始數(shù)據(jù)，第二列是放大區(qū)域。胰腺癌體積小且邊界不清晰。

Fig. 2. The framework of the proposed model. The pipeline includes modules for spiral transformation, feature extraction, feature fusion, and output.?Three parts, mutation prediction loss, intra-modal feature selection loss, and inter-modal prediction constraint loss are combined to supervise training?process.

圖 2. 所提模型的框架。流程包括螺旋變換、特征提取、特征融合和輸出模塊。

三個部分，突變預測損失、內(nèi)模態(tài)特征選擇損失和跨模態(tài)預測約束損失結(jié)合起來監(jiān)督訓練過程。

Fig. 3. Coordinate System of Spiral-transform. The coordinate origin O is the midpoint of spiral transformation. And the spiral line was calculated?by Θ, Ψ and r.

圖 3. 螺旋變換的坐標系統(tǒng)。坐標原點O是螺旋變換的中點。螺旋線通過Θ、Ψ和r計算得出。

Fig. 4. The data augmentation results using the spiral transformation in different coordinate systems. To show more intuitively, coordinate systems are fixed in the same view. The coordinate systems (b) and (c) are rotated by 45 degrees and 120 degrees, respectively, relative to (a) in the x-y plane. The same 3D object obtains different mappings in the three 2D spiral transformed images as shown in the third column.

圖 4. 在不同坐標系統(tǒng)中使用螺旋變換的數(shù)據(jù)增強結(jié)果。為了更直觀地展示，坐標系統(tǒng)在相同視圖中固定。坐標系統(tǒng)(b)和(c)分別相對于(a)在x-y平面中旋轉(zhuǎn)了45度和120度。如第三列所示，相同的3D對象在三個2D螺旋變換圖像中獲得不同的映射。

Fig. 5. Two types of examples from augmented data by spiral transfor mation (a) and 2D geometric transformation (b), respectively. The upper left of each sub-figure is the original data, and others are augmenteddata.

圖 5.通過螺旋變換（a）和2D幾何變換（b）分別增強的數(shù)據(jù)中的兩種示例。每個子圖的左上角是原始數(shù)據(jù)，其他的是增強后的數(shù)據(jù)。

Fig. 6. t-SNE visualization. (a) Geometric transformation (b) Spiral transformation. Points with same number are from same original data.

圖 6. t-SNE可視化。(a) 幾何變換 (b) 螺旋變換。相同編號的點來自同一原始數(shù)據(jù)。

Fig. 7. ACC and AUC of different imaging inputs. Each point represents the results of one fold in the five-fold CV experiments.

圖 7. 不同影像輸入的ACC和AUC。每個點代表五折交叉驗證實驗中一折的結(jié)果。

Fig. 8. ROC curve of the prediction model with/without the model driven items. Experiment 1: Loss is Lmain. Experiment 4: Loss is Lmain + β**Lreg-intra + γL**reg-inter.

圖 8.帶/不帶模型驅(qū)動項目的預測模型的ROC曲線。實驗1：損失為Lmain。實驗4：損失為Lmain + β**Lreg-intra + γL**reg-inter。

Fig. 9. (a) ROC curve of different β and γ. (b) Change rate of AUC in different β and γ. (c) p-value of ROC curves between different (β, γ) with (β = 0.001, γ = 0.01).

圖 9. (a) 不同β和γ的ROC曲線。(b) 在不同的β和γ中AUC的變化率。(c) 在不同的(β, γ)與(β = 0.001, γ = 0.01)之間ROC曲線的p值。

Fig. 10. (a) The relationship between the AUC variation and the origin coordinates of the spiral transformation. The x-axis and y-axis represent?the percentage of the origin offset along coronal axis and sagittal axis, respectively. (b) AUC values at different times of augmentation.

圖 10. (a) AUC變化與螺旋變換原點坐標之間的關(guān)系。x軸和y軸分別代表沿冠狀軸和矢狀軸的原點偏移百分比。(b) 在不同增強次數(shù)下的AUC值。

Fig. 11. (a) AUC values at different times of augmentation on H&N1 dataset. (b) The ROC curve of spiral model and 2D model at 23 augmentation times on H&N1 dataset.

圖 11. (a) 在H&N1數(shù)據(jù)集上不同增強次數(shù)下的AUC值。(b) 在H&N1數(shù)據(jù)集上23次增強后螺旋模型和2D模型的ROC曲線。

Table

表

TABLE I comparison of two data augmentation methods

表 I兩種數(shù)據(jù)增強方法的比較

TABLE II the prediction performance of our proposed model based on five-fold cv

表 II基于五折交叉驗證的我們提出的模型的預測性能

TABLE III prediction performance comparison the multi-modal models with different input (MEAN±STD)

表 III 不同輸入的多模態(tài)模型的預測性能比較（均值±標準差）

TABLE IV complexity comparison of the multi -modal models with different input

表 IV不同輸入的多模態(tài)模型的模型復雜性比較

TABLE V multi-modal clssification results of different models

表 V不同模型的多模態(tài)分類結(jié)果

TABLE VI prdiction performance comparison between single-modal and multi-modal imaging (MEAN±STD)

表 VI 單模態(tài)與多模態(tài)成像的預測性能比較（均值±標準差）

TABLE VII ?performance comparison of the prediction model with/without the model-driven items

表 VII 預測模型使用/不使用模型驅(qū)動項目的性能比較

TABLE VIII HPV prediction results of existing methods on H&N1 dataset

表 VIII現(xiàn)有方法在H&N1數(shù)據(jù)集上的HPV預測結(jié)果文章來源地址http://www.zghlxwxcb.cn/news/detail-846930.html

到了這里，關(guān)于文獻速遞：深度學習胰腺癌診斷--螺旋變換與模型驅(qū)動的多模態(tài)深度學習方案相結(jié)合，用于自動預測胰腺癌中TP53突變麥田醫(yī)學的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！