医学图像在对肿瘤患者的诊断中有很重要的作用，通过这种无创的手段可以为肿瘤的诊断和治疗提供很多的指导，从而实现针对患者情况的个性化治疗。医学图像在指导治疗方面具有很大的潜力是因为它可以提供肿瘤的综合信息并且在后续治疗中观察到肿瘤的变化，了解患者对治疗的反应和预后等。基于这一点，有学者提出用影像组学方法对大量的影像数据进行定量分析来预测肿瘤的表现型。目前，已经有不同类型的肿瘤分类及预后用基于影像组学的方法得以实现。要实现不同类型的肿瘤分类，首先都要将肿瘤从图像中分割出来，再进一步用特征提取算法得到图像中的信息。而通过手动分割的方法获取肿瘤耗时费力，因此自动而又可靠的分割肿瘤图像算法在肿瘤诊断中将有很大的作用，分割的好坏将直接影响后续识别的效果。目前，针对图像分割的算法有多类，只能针对特定的问题来进行分割，在可靠性和速度方面都需要很大的提高才能够满足实际的需求。肿瘤的自动分割有很大的挑战性，因为肿瘤的大小，形状和位置各异，并且和正常的组织相交叠，还可能伴随着水肿。肿瘤图像的边界比较模糊复杂，内部可能会有空洞，这些都增加了对肿瘤图像进行自动分割的难度。

胶质瘤是中枢神经系统中最常见而又最难治的恶性肿瘤之一，我们迫切需要一种无创而且易操作的生物标记来描述潜在的分子活动并预测治疗反应。对肿瘤进行分类后做有针对性的治疗对患者有很大的好处，医学图像能够在医务人员对患者进行诊断和制定治疗方案时提供帮助。例如对肿瘤疾病的诊断往往是依靠医生对肿瘤病变轮廓特征信息的分析来进行的，通过对病灶的形状，边界，截面面积和体积等进行测量和分析来制定治疗方案。已经有论文提出用MRI（Magnetic Resonance Imaging）等广泛使用的影像技术来提取这些肿瘤的特征并进行特定的分类。

本论文研究的内容是基于深度学习卷积神经网络(Convolutional Neural Networks，CNN)和全卷积网络(Fully Convolutional Networks，FCN)来对胶质瘤图像进行自动分割。CNN是人工神经网络中的一种，是深度学习中的卷积神经网络，在分类分割与语音识别及自然语言处理等方面是研究热点。它具有权值共享的网络结构，这样既减少了权值的数量也降低了网络的复杂度。FCN是全卷积神经网络，可以将待分割图像直接作为网络的输入，自动提取图像的特征，与传统算法中复杂的特征提取和数据重建过程完全不同。卷积神经网络通过组合低层特征形成更加抽象的高层特征来发现数据的分布式特征，网络具有从输入数据中学习来对未知样本进行预测的能力。

随着数据量的扩大和硬件技术的不断提高，GPU运算能力增强，深度学习有了充分发展的空间，出现了许多深度学习框架。例如，贾扬清博士写的清晰而高效的开源深度学习框架Caffe（Convolutional Architecture for Fast Feature Embedding）。Caffe支持命令行，Python和Matlab接口，可以在CPU或者GPU上运行。本文所用的是基于Caffe框架的CNN模型和FCN模型来对胶质瘤的MRI图像进行自动分割，最后的分割精度可以分别达到50.08%和83.09%。CNN模型和FCN模型都能应用在胶质瘤图像的分割中，其中FCN模型可以实现端对端的分类，输入图片的大小不再固定，输出图像与输入图像大小相同，为分类后的标签。

Medical images are very important in the diagnosis of patients with tumor, this noninvasive method would provide useful guidance for the diagnosis and treatment of tumor, achieving personalized treatment. Medical images have great potential in guiding treatment because it provide comprehensive information. The information in the images would observe the change of tumor in the subsequent treatment as well as the patients’ response to treatment and prognosis, etc. Based on this, radiomics are proposed to use large amount of image data to quantitativly analysis tumor phenotype. At present, there have been different kinds of tumor classification and prognosis based on radiomics.To classify different types of tumor, the first step is image segmentation, then with further feature extraction algorithms to obtain useful information from the image. Manual segmentation is time-consuming and tedious, so automatic and reliable tumor image segmentation algorithms are required. The subsequent identification and tumor diagnosis results will be affected by the segmentation directly. At present, there are many algorithms of image segmentation, they are effective only for a specific segmentation problem and need to be improved in reliability and speed to meet practical requirement. Automatic segmentation of tumor is a big challenge for us, because the size, shape and position of the tumor are different. In addition, the tumor and normal tissue are overlapped, edema may accompany as well. The boundary of the tumor image is vague and complex, and there may be cavity inside, these factors make tumor image automatic segmentation very challenging.

Glioma is one of the most common and aggressive malignant tumors, a noninvasive and easy-to-operate biomarkers are required to describe molecular activities and predict treatment response. Targeted treatment produce significant benefits for the patients after classifying the tumor, so medical images are of great help in diagnosis and treatment. For example, the diagnosis of tumor diseases depends on the analysis of contour information of the tumor. Doctors make treatment plan by measurement and analysis of the shape, lesions, borders, area and volume of the tumor. Some scholars have presented to use widespread imaging techniques such as MRI(Magnetic Resonance Imaging) to extract specific acteristics of the tumor for the classification.

 

The research contents of this thesis is automatic segmentation of glioma based on                                                                                                                                                                                                             CNN and FCN. CNN is one of the artificial neural networks-the convolutional neural network. It was widely used in classification, segmentation, speech recognition and natural language processing. Its structure of shared-weight reduced the number of weights and the complexity of the network. FCN is fully convolutional networks, the images would be directly used as the network input, the network extracts the image feature automatically, it is completely different from the traditional algorithms which contain complex feature extraction and data reconstruction procession. Convolutional neural network obtains the distributed acteristics of data by combining low-level features to form abstract high-level features, the network learns from the input data automatically to obtain the ability of prediction.

With the enlargement of data, the improvement of hardware technology, and the enhancement of GPU computing ability, a good development opportunity is provied for deep learning. There are a lot of deep learning frameworks, such as Caffe(Convolutional Architecture for Fast Feature Embedding), which is written by Dr Yangqing Jia. Caffe is an effective open source deep learning framework and it supports command line, Python and Matlab interface. Caffe can run with CPU or GPU mode. In this Thesis, CNN and FCN based on Caffe framework is used for automatic segmentation of glioma, reaching the precision of 50.08% and 83.09% respectively. FCN can accomplish the end-to-end classification, the size of the input image is no longer fixed, the size of the output image and input image are the same, the final output is the corresponding label image after classification.

[1] Radner H, Blümcke I, Reifenberger G, et al. WHO Classification of Tumour Pathology and Genetics of Tumours of the Nervous System[J]. Der Pathologe, 2002, 23(4):260-283.
[2] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[3] Newman M. E. J. The structure and function of complex networks[J], SIAM review, 2003, 45(2): 167-256.
[4] Domingos P. A few useful things to know about machine learning[M]. ACM, 2012, 55(10): 78-87.
[5] 周志华, 杨强. 机器学习及其应用[M]. 清华大学出版社, 2011.
[6] 赵永科. 深度学习:21天实战Caffe[M]. 电子工业出版社, 2016.
[7] Clarke L P, Velthuizen R P, Camacho M A, et al. MRI segmentation: methods and applications[J]. Magnetic resonance imaging, 1995, 13(3):343-368.
[8] Pal N R, Pal S K. A review on image segmentation techniques[J]. Pattern recognition, 1993, 26(9):1277-1294.
[9] Kumar V, Gu Y, Basu S, et al. Radiomics: the process and the challenges[J]. Magnetic Resonance Imaging, 2012, 30(9):1234.
[10] Huang Y Q, Liang C H, He L, et al. Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer[J]. J Clin Oncol. 2016, 34(18):2157-2164.
[11] Huang Y, Liu Z, He L, et al. Radiomics Signature: A Potential Biomarker for the Prediction of Disease-Free Survival in Early-Stage (I or II) Non-Small Cell Lung Cancer[J]. Radiology, 2016, 281(3):947.
[12] Gevaert O, Xu J, Hoang C D, et al. Non-small cell lung cancer: identifying prognostic imaging biomarkers by leveraging public gene expression microarray data--methods and preliminary results[J]. Radiology, 2012, 264(2):387-96.
[13] Aerts H J W L, Velazquez E R, Leijenaar R T H, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach[J]. Nature communications, 2014, 5:4006.
[14] Ferrigan L, Wallace W A. Predicting non-small cell lung cancer expression of epidermal growth factor receptor and matrix metalloproteinase 9 from immunohistochemical staining of diagnostic biopsy samples[J]. European Journal of Cancer, 2004, 40(10):1589-1592.
[15] Duncan J S, Ayache N. Medical image analysis: Progress over two decades and the challenges ahead[J]. IEEE transactions on pattern analysis and machine intelligence, 2000, 22(1):85-106.
[16] Y.J.Zhang, A survey on Evaluation methods for image segmentation, Pattern Recognition, 1996, 29(8):1335-1346.
[17] Pham D L, Xu C, Prince J L. Current methods in medical image segmentation 1[J]. Annual review of biomedical engineering, 2000, 2(1):315-337.
[18] Fiedler M. Algebraic connectivity of graphs[J], Czech. Math. J., 1973, 23(23):298-305.
[19] Merris R. Laplacian matrices of graphs: a survey[J]. Linear Algebra and Its Applications, 1994, 197-198(2):143-176.
[20] Zhang Y, Brady M, Smith S. Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm[J]. IEEE Transactions on Medical Imaging, 2001, 20(1):45-57.
[21] Pereira S, Pinto A, Alves V, et al. Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images[J]. IEEE Transactions on Medical Imaging, 2016, 35(5):1240-1251.
[22] Udupa J K, Samarasekera S. Fuzzy connectedness and object definition: theory, algorithms, and applications in image segmentation[J]. Graphical models and image processing, 1996, 58(3): 246-261.
[23] Unser M. Multigrid adaptive image processing[C]. International Conference on Image Processing. IEEE Computer Society, 1995: 49.
[24] Ahmed M N, Farag A A. Two-stage neural network for volume segmentation of medical images[C]. International Conference on Neural Networks. IEEE, 1997:1373-1378 vol.3.
[25] Wang Z, Vemuri B C. DTI segmentation using an information theoretic tensor dissimilarity measure[J]. IEEE Transactions on Medical Imaging, 2005, 24(10):1267.
[26] Deng W, Iyengar S S. A new probabilistic relaxation scheme and its application to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996, 18(4): 432-437.
[27] Sahoo P K, Soltani S, Wong A K C. A survey of thresholding techniques[J]. Computer vision, graphics, and image processing, 1988, 41(2): 233-260.
[28] Nakagawa Y, Rosenfeld A. Some experiments on variable thresholding[J]. Pattern Recognition, 1979, 11(3):191-204.
[29] Lee C, Huh S, Ketter T A, et al. Unsupervised connectivity-based thresholding segmentation of sagittal brain MR images[J]. Computers in biology and medicine, 1998, 28(3): 309-338.
[30] Zhao Y, Li M. A modified fuzzy c-means algorithm for segmentation of MRI[C].Computational Intelligence and Multimedia Applications, 2003. ICCIMA 2003. Proceedings. Fifth International Conference on. IEEE, 2003: 391-395.
[31] Singh M, Patel P, Khosla D, et al. Segmentation of functional MRI by K-means clustering[J]. IEEE Transactions on Nuclear Science, 1996, 43(3): 2030-2036.
[32] Ahmed M N, Yamany S M, Mohamed N, et al. A modified fuzzy C-means algorithm for bias field estimation and segmentation of MRI data.[J]. IEEE Transactions on Medical Imaging, 2002, 21(3):193-199.
[33] Rosenfeld A. On connectivity properties of grayscale pictures[J]. Pattern recognition, 1983, 16(1): 47-50.
[34] S.Z. Li. Markov random field modeling in computer vision. Springer, 1995.
[35] Reed T R, Buf J M H D. A review of recent texture segmentation and feature extraction techniques[M]. Academic Press, Inc. 1993.
[36] Geman S, Geman D. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images[J]. IEEE Transactions on pattern analysis and machine intelligence, 1984 (6): 721-741.
[37] Canny J. A computational approach to edge detection[J]. IEEE Transactions on pattern analysis and machine intelligence, 1986 (6): 679-698.
[38] Qian R J, Huang T S. Optimal edge detection in two-dimensional images[J]. IEEE Transactions on Image Processing, 1996, 5(7): 1215-1220.
[39] Falcão A X, Udupa J K, Samarasekera S, et al. User-Steered Image Segmentation Paradigms: Live Wire and Live Lane[J]. Graphical Models and Image Processing, 1998, 60(4):233-260.
[40] 田捷，罗希平，基于Live-Wire的交互式分割及其在医学图像中的应用，《中国学术期刊文摘》，2000, Vol. 6, No. 9, pp.1174-1175.
[41] Griffin L D, Colchester A C F, Robinson G P. Scale and segmentation of grey-level images using maximum gradient paths[J]. Image and Vision Computing, 1992, 10(6): 389-402.
[42] Maintz J B A, Viergever M A. A survey of medical image registration[J]. Medical image analysis, 1998, 2(1): 1-36.
[43] Zhao Y, Li M. A modified fuzzy c-means algorithm for segmentation of MRI[C].Computational Intelligence and Multimedia Applications, 2003. ICCIMA 2003. Proceedings. Fifth International Conference on. IEEE, 2003: 391-395.
[44] Babaguchi N, Yamada K, Kise K, et al. Connectionist model binarization [M]. Neural Networks In Pattern Recognition And Their Applications. :127-142.
[45] Blanz W E, Gish S L. A connectionist classifier architecture applied to image segmentation[C]. International Conference on Pattern Recognition, 1990. Proceedings. IEEE, 1990:272-277 vol.2.
[46] Vincent L, Soille P. Watersheds in digital spaces: an efficient algorithm based on immersion simulations[J]. IEEE transactions on pattern analysis and machine intelligence, 1991, 13(6): 583-598.
[47] Kotsiantis S B, Zaharakis I, Pintelas P. Supervised machine learning: A review of classification techniques[J]. 2007.
[48] Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: Machine learning in Python[J]. Journal of Machine Learning Research, 2011, 12(10):2825-2830.
[49] Michalski R S. A theory and methodology of inductive learning[J]. Artificial intelligence, 1983, 20(2):111-161.
[50] Ruck D W, Rogers S K, Kabrisky M, et al. The multilayer perceptron as an approximation to a Bayes optimal discriminant function[J]. IEEE Transactions on Neural Networks, 1990, 1(4): 296-298.
[51] 边肇祺, 张学工. 模式识别[M]. 清华大学出版社, 2000.
[52] Quinlan J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1):81-106.
[53] Quinlan J R. C4.5: programs for machine learning[J]. 1992, 1.
[54] Quinlan J R, Cameron-Jones R M. FOIL: A Midterm Report[C]. European Conference on Machine Learning. Springer-Verlag, 1993:1-20.
[55] Schölkopf B, Burges C J C. Advances in kernel methods: support vector learning[M]. MIT press, 1999.
[56] Scholkopf B, Smola A J. Learning with kernels: support vector machines, regularization, optimization, and beyond[M]. MIT press, 2001.
[57] 崔伟东, 周志华. 支持向量机研究[J]. 计算机工程与应用, 2001, 37(1): 58-61.
[58] Kales A, Rechtschaffen A. A manual of standardized terminology, techniques and scoring system for sleep stages of human subjects[M]. US Department of Health, Education and Welfare, Public Health Service, National Institutes of Health, National Institute of Neurological Diseases and Blindness, Neurological Information Network, 1968.
[59] Iber C, Ancoli-Israel S, Chesson A, et al. The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications[M]. Westchester, IL: American Academy of Sleep Medicine, 2007.
[60] Lapierre O, Montplaisir J. Polysomnographic features of REM sleep behavior disorder Development of a scoring method[J]. Neurology, 1992, 42(7): 1371-1371.
[61] Aydın S, Saraoǧlu H M, Kara S. Singular spectrum analysis of sleep EEG in insomnia[J]. Journal of medical systems, 2011, 35(4): 457-461.
[62] Berthomier C, Drouot X, Hermanstoïca M, et al. Automatic analysis of single-channel sleep EEG: validation in healthy individuals.[J]. Sleep, 2007, 30(11):1587-1595.
[63] Fell J, Röschke J, Mann K, et al. Discrimination of sleep stages: a comparison between spectral and nonlinear EEG measures[J]. Electroencephalography and clinical neurophysiology, 1996, 98(5): 401-410.
[64] Flexer A, Gruber G, Dorffner G. A reliable probabilistic sleep stager based on a single EEG signal[J]. Artificial Intelligence in Medicine, 2005, 33(3): 199-207.
[65] Carskadon M A, Rechtschaffen A. Monitoring and staging human sleep[J]. Principles and practice of sleep medicine, 2000, 3: 1197-1215.
[66] Hsu Y L, Yang Y T, Wang J S, et al. Automatic sleep stage recurrent neural classifier using energy features of EEG signals[J]. Neurocomputing, 2013, 104: 105-114.
[67] Bajaj V, Pachori R B. Automatic classification of sleep stages based on the time-frequency image of EEG signals[J]. Computer methods and programs in biomedicine, 2013, 112(3): 320-328.
[68] Vural C, Yildiz M. Determination of sleep stage separation ability of features extracted from EEG signals using principle component analysis[J]. Journal of medical systems, 2010, 34(1): 83-89.1
[69] Koprinska I, Pfurtscheller G, Flotzinger D. Sleep classification in infants by decision tree-based neural networks[J]. Artificial intelligence in Medicine, 1996, 8(4): 387-401.
[70] Haykin S, Network N. A comprehensive foundation[J]. Neural Networks, 2004, 2(2004): 41.
[71] Whittle P, Kay J, Hand D J, et al. Neural Networks and Related Methods for Classification-Discussion[J]. Journal of the Royal Statistical Society Series B-Methodological, 1994, 56(3): 437-456.
[72] McCulloch W S, Pitts W. A logical calculus of the ideas immanent in nervous activity[J]. The bulletin of mathematical biophysics, 1943, 5(4): 115-133.
[73] Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators[J]. Neural networks, 1989, 2(5): 359-366.
[74] Hechtnielsen R. Theory of the backpropagation neural network[M]. Neural networks for perception. Harcourt Brace and Co. 1992:593-605 vol.1.
[75] Werbos P J. Backpropagation through time: what it does and how to do it[J]. Proceedings of the IEEE, 1990, 78(10): 1550-1560.
[76] Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128: 1-58.
[77] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[78] Ngiam J, Khosla A, Kim M, et al. Multimodal deep learning[C].Proceedings of the 28th international conference on machine learning (ICML-11). 2011: 689-696.
[79] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]. Advances in neural information processing systems. 2012: 1097-1105.
[80] Lecun Y. LeNet-5, convolutional neural networks[J]. 2015.
[81] Sharif Razavian A, Azizpour H, Sullivan J, et al. CNN features off-the-shelf: an astounding baseline for recognition[C].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014: 806-813.
[82] Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S, et al. Beyond short snippets: Deep networks for video classification[C].Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 4694-4702.
[83] Socher R, Huval B, Bath B P, et al. Convolutional-Recursive Deep Learning for 3D Object Classification[C].NIPS. 2012, 3(7): 8.
[84] Gong Y, Wang L, Guo R, et al. Multi-scale orderless pooling of deep convolutional activation features[C].European conference on computer vision. Springer International Publishing, 2014: 392-407.
[85] Hinton G E, Srivastava N, Krizhevsky A, et al. Improving neural networks by preventing co-adaptation of feature detectors[J]. arXiv preprint arXiv:1207.0580, 2012.
[86] Han S, Pool J, Tran J, et al. Learning both weights and connections for efficient neural network[C].Advances in Neural Information Processing Systems. 2015: 1135-1143.
[87] Jia Y, Shelhamer E, Donahue J, et al. Caffe: Convolutional architecture for fast feature embedding[C].Proceedings of the 22nd ACM international conference on Multimedia. ACM, 2014: 675-678.
[88] http://ethereon.github.io/netscope/#/editor1.
[89] Johnson J, Karpathy A, Fei-Fei L. Densecap: Fully convolutional localization networks for dense captioning[C].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 4565-4574.
[90] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 3431-3440.