基于深度学习的医学影像大数据分析 -...

基于深度学习的医学影像大数据

分析

Zhao Di

Computer Network Information Center

Chinese Academy of Sciences

2016年图形处理器技术大会

北京国际饭店会议中心，2016年9月13日

关于我

关于我：博士毕业论文

细胞间的热传导；

采用微分方程模型，全文上千公式；

关于我：哥伦比亚大学工作

关于我：哥伦比亚大学生物医学信息中心

生物医学信息分析；采用贝叶斯模型；

关于我：OSU脑科学中心

关于我：OSU脑科学中心

MRI成像与图像分析的训练，西门子公司；采用贝叶斯模型；导师：Mark A. Pitt教授，Jay Myung教授，Zhong-Lin Lu教授，认知科学

基于深度学习的医学图像分析

• 医学影像技术提供了新的契机

PET SPECT

描述人体组织解剖结构信息

描述人体组织代谢信息

基于卷积神经网络的医学图像分析

基于深度神经网络的医学图像分析

深度学习中的主要网络：（1）自编码机 Auto Encoder （2）稀疏编码 Sparse Coding （3）限制性玻尔兹曼机 RBM （4）深度置信网络 Deep Belief Network （5）深度卷积神经网络

Hinton 于2006年提出，与GPU计算结合，使得大规模的数据学习和多层网络训练成为可能。

深度学习——疾病的分类

• 脑：脑血管病，神经退行性疾病（阿尔茨海默病，帕金森病，癫痫），精神疾病（抑郁症，精神分裂症），脑瘤

• 胸：心脏疾病，肺结节/肺癌，乳腺结节/乳腺癌 • 颈：颈动脉检测，甲状腺癌 • 眼：糖尿病眼病 • 皮肤：皮肤癌 • 腹部：胃癌 • 男性骨盆：前列腺癌 • 女性骨盆：子宫颈癌 • 耳 • 鼻 • 背 • 四肢 • 臀 • 腰

阿尔茨海默病（Alzheimer's disease）

A

B

C

阿尔茨海默病（Alzheimer‘s disease）

是一种起病隐匿的进行性发展的神经

系统退行性疾病。阿尔茨海默病导致

脑神经细胞死亡，脑组织缺失。如左

图所示，阿尔茨海默病将导致严重

的脑萎缩

临床上以记忆障碍、失语、失用、失认、视空间技能损害、执行功能障碍以及人格和行为改变等全面性痴呆表现为特征，病因迄今未明。

阿尔茨海默病（Alzheimer‘s disease）是一

种起病隐匿的进行性发展的神经系统退行

性疾病。阿尔茨海默病导致脑神经细胞死

亡，脑组织缺失。如左图所示，阿尔茨海

默病将导致严重的脑萎缩；

阿尔茨海默病尚未找到有效的治疗手段，

临床上”早期发现，早期干预”，对于减轻

病人脑部损害有非常重要的意义。

临床上以记忆障碍、失语、失用、失认、

视空间技能损害、执行功能障碍以及人格

和行为改变等全面性痴呆表现为特征，病

因迄今未明；

Classification of Alzheimer’s Disease Structural MRI Data by Deep Learning Convolutional Neural Networks

In this paper, we used convolutional neural network to classify Alzheimer’s brain from normal healthy brain. The importance of classifying this kind of medical data is to potentially develop a predict model or system in order to recognize the type disease from normal subjects or to estimate the stage of the disease. Using Convolutional Neural Network (CNN) and the famous architecture LeNet-5, we successfully classified structural MRI data of Alzheimer’s subjects from normal controls where the accuracy of test data on trained data reached 98.84%.

Saman Sarraf, Ghassem Tofighi, Classification of Alzheimer's Disease Structural MRI Data by Deep Learning Convolutional Neural Networks, http://arxiv.org/abs/1607.06583

Classification of Alzheimer’s Disease Using fMRI Data and Deep Learning Convolutional Neural Networks

In this paper, we used convolutional neural network to classify Alzheimer’s brain from normal healthy brain. of clinical data such as Alzheimer’s disease has been always challenging and most problematic part has been always selecting the most discriminative features. Using Convolutional Neural Network (CNN) and the famous architecture LeNet-5, we successfully classified functional MRI data of Alzheimer’s subjects from normal controls where the accuracy of test data on trained data reached 96.85%.

Saman Sarraf, Ghassem Tofighi, Classification of Alzheimer's Disease using fMRI Data and Deep Learning Convolutional Neural Networks, arXiv:1603.08631 [cs.CV].18:58 2016/9/12

DeepAD: Alzheimer′s Disease Classification via Deep Convolutional Neural Networks using MRI and fMRI

This work outlines state-of-the-art deep learning-based pipelines employed to distinguish Alzheimer's magnetic resonance imaging (MRI) and functional MRI data from normal healthy control data for the same age group. Using these pipelines, which were executed on a GPU-based high performance computing platform, the data were strictly and carefully preprocessed. Next, scale and shift invariant low- to high-level features were obtained from a high volume of training images using convolutional neural network (CNN) architecture.

Saman Sarraf, Ghassem Tofighi, DeepAD: Alzheimer′s Disease Classification via Deep Convolutional Neural Networks using MRI and fMRI, http://dx.doi.org/10.1101/070441

深度学习—— 脑疾病图像分析

数据源：ADNI；

合作单位：北京天坛医院、等


本项目获得了“2016年北京市自然科学基金重点项目”的资助

深度学习—— 具有专业知识的深度学习

更高的准确率、更小的训练样本

与首都医科大学附属宣武医院合作研究

1. AD诊断

MRI

PET

CSF 其它临床诊断

fMRI

ROIs

ROIs

ROIs

多通道深度学习模型1



深度学习模型N

加权贝叶斯网络

权重w1 权重w2

权重w3

权重w(n)权重w(n-1)

临床诊断决

策结果

通道1

通道N

……

通道1

通道N

……

通道1

通道N

……

……

……

……

1. AD诊断-ADNI

AD生物标记物：脑内Aβ沉积，脑脊液的tau，PET，结构性MRI

1. AD诊断-临床

1. AD诊断-预处理

• 分割各个脑区


杏仁核、海马、内嗅皮层结构标记（线性测量，体积）

• 线性指标中，颞角宽度敏感度最高

• 双侧颞角水平正好代表海马的中部，在此区

域包含了大量的联系纤维(内嗅区、海马、

海马旁回等与大脑额、颞叶的纤维联系)，

早期的病理改变常局限在此区域

• 线性测量的重复性和特异性较低


• 面积测量 • 测量前后连合间层面的额

叶、颞叶、侧脑室体部断

面、颞角和外侧裂平均横

断面面积

• 萎缩率最高的是颞角，然

后依次为侧脑室体部、外

侧裂、颞叶，最后为额叶

• 面积测量忽略了皮层厚度

对于萎缩程度的影响

• 轻度AD海马萎缩约为15%-

22%，严重者可达40%

• 内嗅皮层、旁嗅皮层及颞极皮

层在AD均显示萎缩，以内嗅

皮层萎缩最明显

• 海马及内嗅皮层萎缩可以早期

诊断AD，体积测定诊断AD的

准确度约85%-94%

• 体积测量

Deep Learning-Parkinsons Diseases

PD Disease State Assessment in Naturalistic Environments using Deep Learning

In this work we propose an assessment system that abides practical usability constraints and applies deep learning to differentiate disease state in data collected in naturalistic settings. Based on a large data-set collected from 34 people with PD we illustrate that deep learning outperforms other approaches in generalisation performance, despite the unreliable labelling characteristic for this problem setting, and how such systems could improve current clinical practice.

Nils Y. Hammerla, James M. Fisher, Peter Andras, Lynn Rochester, Richard Walker, Thomas Plotz, PD disease state assessment in naturalistic environments using deep learning, AAAI'15 Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 1742-1748.

深度学习—肺癌筛查

• 医学诊断现状：

仅通过单次CT图像不能判断

碘灌注增强

仅通过医学图像不能判断

病理分析

• 医学诊断期望：

通过医学图像判断肿瘤的良恶性并给出量化的特征，如某一评分指标或是尺寸等

深度学习— 肺癌筛查划分标准类型

肿瘤肉眼形态管内型，管壁浸润型，结节型，块状型，弥漫浸润型

肿瘤发生部位中央型，周围型，弥漫型

组织病理鳞状上皮细胞癌，腺癌，细支气管肺泡癌，小细胞未分化肺癌，大细胞未分化肺癌

类型特征

鳞状上皮细胞癌息肉状腔内肿块和(或) 支气管阻塞；肺门的肿块（累及淋巴结）

腺癌圆形或分叶状的孤立结节邻近肺浸润，其边缘常不规则呈毛刺边缘

Using Deep Learning for Pulmonary Nodule Detection & Diagnosis This study uses a revolutionary image recognition method, deep learning, for the classification of potentially malignant pulmonary nodules. Deep learning is based on deep neural networks.

Ross Gruetzemacher, Ashish Gupta, Using Deep Learning for Pulmonary Nodule Detection & Diagnosis, Twenty-second Americas Conference on Information Systems, San Diego, 2016.

Computer-aided classification of lung nodules on computed tomography images via deep learning technique

In this study, we attempted to simplify the image analysis pipeline of conventional CAD with deep learning techniques. Specifically, we introduced models of a deep belief network and a convolutional neural network in the context of nodule classification in computed tomography images. Two baseline methods with feature computing steps were implemented for comparison. The experimental results suggest that deep learning methods could achieve better discriminative results and hold promise in the CAD application domain.

Hua KL, Hsu CH, Hidayati SC, Cheng WH, Chen YJ, Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther. 2015 Aug 4;8:2015-22.

Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique In the CT images, a massive region is first detected using an active contour filter, which is a type of contrast enhancement filter that has a deformable kernel shape. Subsequently, high-uptake regions detected by the PET images are merged with the regions detected by the CT images. FP candidates are eliminated using an ensemble method; it consists of two feature extractions, one by shape/metabolic feature analysis and the other by a CNN, followed by a two-step classifier, one step being rule based and the other being based on support vector machines.

Teramoto A, Fujita H, Yamamuro O, Tamaki T, Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique, Med Phys. 2016;43(6):2821

深度学习—肺癌筛查 • 设计的CNNs网络 • 训练数据（三个病

人的三张Slice，取1600个28*28大小正样本，600个28*28大小负样本）

正样本负样本

超声图像

采用CNN+CSF等算法

参加Kaggle‘s data science competitions：Ultrasound Nerve Segmentation

CCF HPCChina 2016已经投稿

The Problem:

深度学习— 颈部超声图像识别

1、存在问题


Loss / Accuracy曲线

Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks

We use deep max-pooling convolutional neural networks to detect mitosis in breast histology images. The networks are trained to classify each pixel in the images, using as context a patch centered on the pixel. Simple postprocessing is then applied to the network output. Our approach won the ICPR 2012 mitosis detection competition, outperforming other contestants by a significant margin.

Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J, Mitosis detection in breast cancer histology images with deep neural networks, Med Image Comput Comput Assist Interv. 2013;16(Pt 2):411-8.

Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images

In this work we present a computational imaging framework using deep and ensemble learning for reliable detection of blood vessels in fundus color images. An ensemble of deep convolutional neural networks is trained to segment vessel and non-vessel areas of a color fundus image. During inference, the responses of the individual ConvNets of the ensemble are averaged to form the final segmentation. In experimental evaluation with the DRIVE database, we achieve the objective of vessel detection with maximum average accuracy of 94.7% and area under ROC curve of 0.9283.

Debapriya Maji, Anirban Santara, Pabitra Mitra, Debdoot Sheet, Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images, arXiv:1603.04833 [cs.LG].

H&E-stained Whole Slide Deep Learning Predicts SPOP Mutation State in Prostate Cancer

We constructed a statistical model that predicts whether or not SPOP is mutated in prostate cancer, given only the digital whole slide after standard hematoxylin and eosin [H&E] staining. Using a cohort of 177 prostate cancer patients where 20 had mutant SPOP, we trained multiple ensembles of residual networks, accurately distinguishing SPOP mutant from SPOP wild type patients.

Andrew J Schaumberg, Mark A Rubin, Thomas J Fuchs, H&E-stained Whole Slide Deep Learning Predicts SPOP Mutation State in Prostate Cancer, http://dx.doi.org/10.1101/064279

Prostate cancer detection using photoacoustic imaging and deep learning

This paper introduces a new database which consists of a large sample size of patients gathered using multispectral photoacoustic imaging. As an alternate to the standard two class labeling (malignant, normal), our voxel based ground truth diagnosis consists of three classes (malignant, benign, normal). We explore deep neural nets, experiment with three popular activation functions, and perform different sub–feature group analysis. Our initial results serve as a benchmark on this database. Greedy based feature selection recognizes and eliminates noisy features. Ablation feature ranking at the feature and group level can simplify clinician effort and results are contrasted with medical literature. Our database is made freely available to the scientific community.

Arjun Raj Rajanna, Raymond Ptucha, Saugata Sinha, Bhargava Chinni, Vikram Dogra, Navalgund A. Rao, Prostate cancer detection using photoacoustic imaging and deep learning, IS&T International Symposium on Electronic Imaging 2016.

深度学习—— 脑模拟与对人脑视觉系统的深入理解

拟用大规模神经网络建模和高性能计算机仿真方法来探明视觉认知的动态交互

机制，并在此基础上与深度学习理论结合，研发出高效的类脑物体识别算法，

为新一代智能视觉应用提供最基本的信息处理加工技术。

仿真规模神经元达到1亿量级、突触达到1000亿量级；

能模拟高级视皮层在1秒内完成的轮廓识别任务的精细神经活动过程；

与北京师范大学合作研究

获得了北京市科委 “大脑高级视觉系统的仿真计算模型与应用验证”的资助。

深度学习——人脑如何学习和记忆？

人的学习过程中，大脑对外部环境进行感知，注意机制对感兴趣的信息保持关注；在工作记忆中，新知识在旧知识的基础上通过检索被快速建立起来；而后经过神经元的加工整理，形成难被遗忘的长时记忆。由此，人不断地从生活经验中建立并整合知识，从而学会处理日益复杂的任务。在持续不断的学习过程中，对以往知识检索利用，使得人们只需要少量的训练就能快速地学会新的任务。综上，一个真正的智能系统应具备以下两方面的作用:在长时记忆系统中建立一个可检索的知识库，在交互过程中持续不断的整合更新知识库。

感谢厚爱与支持！

基于深度学习的医学影像大数据 分析 -...

Documents

基于深度学习的医学影像大数据分析 -...