PPT.pptx

AUTOMATIC DETECTION OF LUNG NODULE CANDIDATES USING EFFECTIVE
AND RELIABLE 3D CNN FRAMEWORK
Presented By Supervisor

1) Introduction
2) Literature survey
3) Problem definition
4) Objectives of the research
5) Research Contribution
5.1. Capturing contextual information for lung nodule candidate detection
5.2. Optimization of Multi-Resolution 3 Dimensional CNN for automatic weak label initialization
5.3. Reducing the complexity of Multi-Resolution 3D CNN by adding depthwise separable architecture
5.4. Geometrical features with CNN features for enhancing lung nodule candidate classification
6) Overall result and discussion
7) Conclusion
8) Reference
Agenda

 Lung cancer, also known as lung carcinoma, is a malignant tumor
characterized by uncontrolled growth of the cell in tissues of the lung.
Fundamental to the diagnosis of lung cancer in CT scans is the detection and
interpretation of lung nodules.
 It is mandatory to treat this to avoid spreading its growth by metastasis to
other parts of the body.
 Most cancers that start in the lung are carcinomas.
1.1 Lung Cancer

 Early diagnosis is key to improving patients' survival rates.
 CT is one of the modest medical imaging methods to diagnose
the lung cancer. The performance of optimization algorithms to
extract the tumor from the lung image has been implemented
and analyzed.
 Use of MRI in the evaluation of pulmonary nodules has thus far
been limited. The reasons include limited spatial resolution, high
susceptibility differences between air spaces and pulmonary
interstitium, and the presence of respiratory and cardiac motion
artifacts.
 To improve diagnostic accuracy, computer-aided diagnosis
(CAD) algorithms have been developed to assist lung cancer
detection.
1.2 Lung cancer detection techniques

 The automatic lung cancer detection process is divided into two processes such as extracting all suspected
candidate nodules, and classifying the extracted nodules into two categories as positive and false-positive
nodules.
 A multi-resolution 2D Convolutional Neural Network (CNN) model was proposed by means of
knowledge transfer for automatic lung cancer detection.
 However, this model is limited in capturing the contextual information in the images.
 This research focuses on improving the lung cancer detection accuracy by reducing false identification of
nodule and reducing computational complexity.
1.3 Computer-Aided Diagnosis (CAD)

1.3 Computer-Aided Diagnosis (CAD)

Authors (Year) Methods used Merits Demerits
Liu & Kang (2017) DCNN MV-CNN uses
multiple views to
identify lung nodules
in thoracic CT as
input channels.
Result of the methodology, for
binary classification quite
competitive in terms of both
accuracy and AUC.
Shen et al. (2017) MC-CNN MC-CNN resolves
the challenging
problem of
classification of lung
nodule malignancy
suspicion.
An automatic nodule detection
process is required to speed up the
diagnosis process.
Lung nodule detection Techniques

Aresta et al. (2017) Multiscale LoG,
FPs
Performance is better
than art-to-art method
when compared with
high number of small
radius in juxta-pleural
nodules.
By the detection method, produces
a high number of FPs.
Zhang et al. (2018) Laplacian of
Gaussian (LoG),
DCNN
Estimate their
diameters accurately
in CT Scan, Low
false positive rate and
high sensitivity.
Ground-Glass Opacity (GGO) in
nodule detection process was not
detected.

Tang et al. (2018) DCNN Strong clinical device
that takes advantage
of state-of - the-art
architectures to
capture the spatial
character of CT data
Need to improve the sensitivity
Gong et al. (2018) CFS, Random
Forest Classifier
(RFC)
Reduce the False
positive rate
It requires multi Classifier to
support larger data set.

Da Silva et al. (2018) PSO Reduce False
positive rate,
Making
scanning
analyzes more
effective and
less challenging
by radiologists
Additional tests with other
databases are required to enhance
Rodrigues et al. (2018) SCM Accuracy level
is high.
Malignancy levels of nodules are
not efficiently extracted by SCM.

Onishi et al. (2019) DCNN and GAN High
classification
accuracy
On the basis of CT images
alone, DCNN in the device
will recognize high precision
pulmonary nodules
Zuo et al. (2019) Multi-Resolution CNN High accuracy 2D CNN model used in
MRCCNN-KTmethod is
limited in capturing the
contextual information
between slices.

Problem definition
The 2D CNN model used in MRCNN-KT method is limited
in capturing the Contextual information between slices.
In MRCNN-KT method, some types of nodules are not fully
represented or not fully highlighted which may lead to the
false identification of nodules.
The computational complexity of 3D CNN is high, requiring
significant computational resources and sized-based
assessment of the lung nodule is a challenging task.

4. Objectives of the Research Work

Objectives of the Research Work
To capture contextual information between slices, a 3D CNN model is considered.
To reduce false positive and false negative for lung nodule identification, a new
iteratively optimized deep learning method is proposed.
To reduce the complexity of 3D CNN, the parameter size of its convolutionary
filters is reduced without the use of principled learning methods to cause
degradation in its accuracy.
To increase the detection of various sizes of lung nodules, a two-stream model is
proposed for CNN feature and geometric features.

5. Research Contributions
19
5.1 Capturing contextual information for lung nodule
candidate detection

20
MR3DCNN-KT
 The radiological heterogeneity might result in the invisibility of some nodules whereas other non-
nodules are highlighted.
 The invisibility highlight is easy to give rise to the difficulty of identifying nodules and non-
nodules, directly leading to an increase in false positive candidates and false negative candidates.
 This reduces the efficiency of lung nodule classification.
 Moreover, the lung nodules are typically in various size and shapes.
 Some lung tissues are very similar to the real nodules in shape which may lead to an increase in
false positives.
 A network model identifies larger nodules effectively than the small lung nodules.
 Hence, the variable shapes of lung nodules also pose a challenge for lung nodule classification.
 To address those problems, a Multi Resolution-Convolutional Neural Network (MR-CNN) is
introduced for lung nodule candidate classification by the way of knowledge transfer.
 With this method, both small nodules seemingly in low resolution and large nodules seemingly in
high resolution can be recognized.
Multi-Resolution Convolutional Neural Network

 In MR-CNN model, five sigmoid classifiers are
reduces with an appended number from 1 to 5.
 Each side-output branch is connected to different
layers from the backbone of the network.
 The receptive field of each side output branch can be
obtained by calculating the receptive field of the
corresponding layer in the backbone of the network.
 According to this each side, the corresponding
relationship between each branch and the receiving
field is calculated.
 Each side-output branch has different receptive field
size.
 Calculate complete the extraction of different
resolution features by integrating all of these feature
maps in different receptive field sizes.
21
MR3DCNN-KT
Structure of Multi-Resolution Convoltional Neural Network model

22
MR3DCNN-KT
 Knowledge transfer can be carried out from the training process of the source task into the
target algorithm to help improve the learning of the target predictive function.
 However, since the classification method of the target field (malignant nodule) is based on the
whole image rather than on pixels, the loss function must be improved, such as, being
calculated over the whole image.
 Besides that, in the test phase, in order to meet the requirements of image-wise classification
accuracy for the target domain, the test method is improved too.
 In MRCNN-KT, 2D CNN is used.
 In 2D CNN, convolutions are applied on the 2D feature maps to compute features from the
spatial dimensions only.
 It limited to handle the contextual information in images.
 In order to handle the contextual information in the images, 3D CNN is used in MRCNN-KT.
Knowledge Transfer

23
MR3DCNN-KT
 3D CNN generates multiple information from various image slice and performs convolution and sub
sampling separately in each slice of images.
 The final feature representation is obtained by combining information from all slices of image.
 3D convolution is performed in convolutional stages of 3D CNN to compute features from both spatial
and temporal dimensions.
 The 3D convolution is achieved by convolving a 3D kernel to the cube formed by stacking multiple
contiguous frames together.
 By this construction, the feature maps in the convolution layer are connected to multiple contiguous
frames in the previous layer, thereby capturing motion information.
 Formally, the value at position x, y, z on the jth feature map in the ith layer is given by
3D CNN

24
MR3DCNN-KT
Extraction of multiple features from different slices

25
Experimental Analysis
Techniques Evaluated
1. Image Processing Algorithm for Microaneurysm Candidate
Detection
2. The Shape-based Genetic Algorithm Template Matching
(GATM)
3. Automated Pulmonary Nodule Detection on CT images
with Morphological Matching Algorithm
4. Multi-Resolution CNN and Knowledge Transfer (MRCNN-
KT)
5. Multi-Resolution 3 Dimensional CNN and Knowledge
Transfer (MR3DCNN-KT)... Proposed

26
Dataset Description
 The performance of proposed lung cancer detection techniques is analysed by
comparing with other existing techniques using MATLAB 2018a.
 In this experiment, a patient lung CT scan dataset from Kaggle’s Data Science Bowl
2017 dataset is used.
 This dataset contains labeled data for 2101 patients, in which 1261 data are used for
training and 840 data are used for testing process.
 For each patient the data consists of CT scan data and a label (0 for no cancer, 1 for
cancer).

Performance Metrics
 Accuracy
In general, the accuracy metric measures the ratio of correct lung cancer detection over the total number of
instances evaluated. It is calculated as
 Precision
Precision value is computed is based on lung cancer detection at true positive prediction, false positive. The
precision value should be more in the proposed methodology than the existing approach for the better system
performance. It is calculated as
 Recall
Recall value is calculated is based on lung cancer detection at true positive prediction, false negative. It is
calculated as

28
Performance Metrics
 F measure
F-measure metric represents the harmonic mean between recall and precision values. It is calculated as
 Error rate
Error rate is calculated as.
 Seperability
To measure the separability of the data representation in different layers. It is calculated as follows:

29
(a) (b) (c)
Figure 5.1 Outcomes of Lung Nodule Detection Models: (a) Input Image (b) Detected Nodules using MA_Detection
System (c) Detected Nodules using GATM

30
(a) (b) (c)
Figure 5.1 Outcomes of Lung Nodule Detection Models: (a) Detected Nodules using Automated Nodule detection
(b) Detected Nodules using MRCNN-KT (c) Detected Nodules using MR3DCNN-KT

31
Accuracy
Fig 5.2. Comparison of Accuracy
Table 5.1. Comparison of Accuracy
Scans
MA_Detec
tion
System
GATM
Automa
ted
Nodule
Detectio
n
MRCN
N_KT
MR3DC
NN-KT
10 0.81 0.83 0.87 0.88 0.91
20 0.83 0.85 0.89 0.9 0.93
30 0.84 0.87 0.91 0.91 0.96
40 0.86 0.89 0.94 0.95 0.97
50 0.88 0.91 0.96 0.97 0.99
0
0.2
0.4
0.6
0.8
1
1.2
10 20 30 40 50
Accuracy
Scan
Accuracy
MA_Detection System
GATM
Automated Nodule Detection
MRCNN_KT
MR3DCNN-KT

32
Precision
Fig 5.3. Comparison of Precision
Table 5.2. Comparison of Precision
Scans
MA_Dete
ction
System
GATM
Autom
ated
Nodule
Detecti
on
MRCN
N_KT
MR3D
CNN-
KT
10 0.879 0.8802 0.8822 0.8866 0.9145
20 0.88 0.8809 0.8821 0.8869 0.9146
30 0.8772 0.8811 0.8835 0.8872 0.9152
40 0.8773 0.8817 0.8838 0.8873 0.9157
50 0.8791 0.8919 0.884 0.8901 0.9158
0.85
0.86
0.87
0.88
0.89
0.9
0.91
0.92
10 20 30 40 50
Precison
Scans
Precison
MA_Detection System
GATM
MRCNN_KT
MR3DCNN-KT

33
Recall
Fig 5.4. Comparison of Recall
Table 5.3. Comparison of Recall
Scans
MA_Detec
tion
System
GATM
Automa
ted
Nodule
Detecti
on
MRCN
N_KT
MR3D
CNN-
KT
10 0.8431 0.8752 0.887 0.8868 0.9135
20 0.8436 0.8755 0.889 0.8871 0.9137
30 0.8441 0.8757 0.901 0.8872 0.9139
40 0.8445 0.8759 0.904 0.8874 0.9142
50 0.8447 0.8761 0.906 0.8877 0.9146
0.8
0.82
0.84
0.86
0.88
0.9
0.92
10 20 30 40 50
Recall
Scan
Recall
MA_Detection System
GATM
MRCNN_KT
MR3DCNN-KT

34
F measure
Fig 5.5. Comparison of Correctness
Table 5.4. Comparison of Correctness
Scans
MA_Detec
tion
System
GATM
Automa
ted
Nodule
Detecti
on
MRCN
N_KT
MR3D
CNN-
KT
10 0.8844 0.8856 0.8851 0.8867 0.914
20 0.8848 0.8858 0.8854 0.8871 0.916
30 0.8852 0.8862 0.8858 0.8873 0.919
40 0.8856 0.8865 0.8863 0.8875 0.921
50 0.8861 0.8868 0.8866 0.8878 0.924
0.86
0.87
0.88
0.89
0.9
0.91
0.92
0.93
10 20 30 40 50
Fmeaure
Scan
Fmeaure
MA_Detection System
GATM
MRCNN_KT
MR3DCNN-KT

35
Error rate
Fig 5.6. Comparison of error rate
Table 5.5. Comparison of error rate
Trainin
g
Epoch
MA_Detec
tion
System
GATM
Automa
ted
Nodule
Detecti
on
MRCN
N_KT
MR3D
CNN-
KT
0 5.2 4.92 4.9 4.8 4.8
100 5.1 4.88 4.7 4 3.8
200 4.7 4.72 4.6 3.6 3.4
300 4.3 4.66 4.4 3.1 3
400 4.1 4.62 4.2 2.9 2.4
500 3.9 4.58 4 2.5 2.1
0
1
2
3
4
5
6
0 100 200 300 400 500
Error
rate
Training Epoch
Error rate
MA_Detection System
GATM
MRCNN_KT
MR3DCNN-KT

36
Separability
Fig 5.7. Comparison of seperability
Table 5.6. Comparison of seperability
CNN Layer
MA_Detecti
on System
GATM
Automate
d Nodule
Detection
MRCNN_
KT
MR3DCN
N-KT
Input 0.11 0.18 0.15 0.2 0.25
convolutional 0.13 0.25 0.29 0.3 0.31
Revolution 0.54 0.59 0.64 0.9 1.1
Pooling 0.91 0.94 1.01 1.1 1.3
FCL 1.24 1.25 1.32 1.5 1.7
Softmax 2.18 2.27 2.27 2.3 2.45
0
0.5
1
1.5
2
2.5
3
separability
CNN Layer
Separability
MA_Detection System
GATM
Automated Nodule
Detection
MRCNN_KT
MR3DCNN-KT

5.Research Contribution
37
5.2 Optimization of Multi-Resolution 3 Dimensional CNN
for automatic weak label initialization

38
IO-MR3DCNN-KT
 The training data preparation is one of the biggest obstacle in these supervised deep learning models for lung
nodule detection.
 Since, the manual labeling requires tedious and time-consuming labors.
 Sometimes, even make mistakes on the label.
 Especially for mapping functional network in large scale datasets such as hundreds of thousands of lung
images are used.
 It leads to the manual labelling method will become almost infeasible.
 To overcome this problem, a new Iteratively Optimized deep learning CNN (IO-CNN) framework was
introduced to tackle both network recognition and training data labelling tasks.
 In this framework, it enables the functional brain networks recognition task to a fully automatic large-scale
classification procedure.
 IO-CNN framework has superior spatial pattern modelling capability in dealing with various types lung
nodule images, and the iterative optimization algorithm can gradually accommodate the mistaken labels
introduced by the fully automatic but week label initialization, eventually converging to a fine-grained
classification accuracy.
 The core idea of weak initialization for the IO-CNN is to use spatial overlap rate to roughly model the
training data label distribution, and then optimize the distribution through IO-CNN training.
IOCNN

 By using deep iterative CNN with week label initialization, the week label images are discarded and
also the high optimal lung images are identified more accurately.
 In this phase, the lung nodule detected is optimize.
 By using iterative training in IO-CNN framework, the optimized lung node images are identified
easily.
 In the next phase of this work, extracting the accuracy of lung nodule images are determined between
the slices of image.
39
IO-MR3DCNN-KT

40
Experimental analysis
1. Multiview-ConvNets
2. Deep Network based 3D Landmark Detection
3. Interleaved 3D CNN
4. Multi-Resolution 3 Dimensional CNN and Knowledge Transfer
(MR3DCNN-KT)
5. Iteratively Optimized Multi-Resolution 3 Dimensional CNN and
Knowledge Transfer (IO-MR3DCNN-KT)...proposed

(a) (b) (c)
Fig. 5.8 Results of Lung Nodule Candidate Detection Models: (a) Input Image (b) Detected Nodules using
McovNets (c) Detected Nodules using 3D based DN

(a) (b)
Fig. 5.8 Results of Lung Nodule Candidate Detection Models: (a) Detected Nodules using
Interleaved 3D CNN (b) Detected Nodules using IO-MR3DCNN-KT

Accuracy
Table 5.7 Comparison of Accuracy
Scans
McovN
ets
3D
based
DN
Interlea
ve
3DCN
N
MR3D
CNN-
KT
IO-
MR3D
CNN-
KT
10 0.73 0.82 0.84 0.91 0.94
20 0.75 0.84 0.85 0.93 0.96
30 0.76 0.85 0.86 0.95 0.97
40 0.79 0.86 0.87 0.96 0.98
50 0.81 0.87 0.89 0.97 0.99
0
0.2
0.4
0.6
0.8
1
1.2
10 20 30 40 50
Accuracy
Scan
Accuracy
McovNets
3D based DN
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

Precision
Scans
McovN
ets
3D
based
DN
Interle
aved
3DCN
N
MR3D
CNN-
KT
IO-
MR3D
CNN-
KT
10 0.82 0.85 0.89 0.9145 0.9385
20 0.85 0.88 0.8943 0.9235 0.9387
30 0.87 0.89 0.8948 0.9252 0.9789
40 0.9 0.91 0.912 0.9257 0.9791
50 0.93 0.94 0.9524 0.9558 0.9792
0.7
0.75
0.8
0.85
0.9
0.95
1
10 20 30 40 50
Precison
Scans
Precison
McovNets
3D based DN
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

Recall
Scans
McovNet
s
3D
based
DN
Interleave
d 3DCNN
MR3DCNN-
KT
IO-
MR3DCNN-
KT
10 0.8875 0.8958 0.8986 0.9115 0.9123
20 0.8879 0.8961 0.8988 0.9116 0.9125
30 0.8881 0.8964 0.899 0.9117 0.9127
40 0.8885 0.8967 0.8992 0.9118 0.9128
50 0.8889 0.8969 0.8995 0.9119 0.9129
0.87
0.875
0.88
0.885
0.89
0.895
0.9
0.905
0.91
0.915
10 20 30 40 50
Recall
Scan
Recall
McovNets
3D based DN
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

F measure
Fig 5.12. Comparison of F Measure
Table 5.10. Comparison of F Measure
Scans
McovNet
s
3D
based
DN
Interleav
e
3DCNN
MR3DCN
N-KT
IO-
MR3DCN
N-KT
10 0.8872 0.8892 0.8981 0.914 0.9263
20 0.8873 0.8894 0.8983 0.916 0.9265
30 0.8875 0.8896 0.8985 0.919 0.9267
40 0.8877 0.8897 0.8987 0.921 0.9268
50 0.8878 0.8899 0.8989 0.924 0.9269
0.86
0.87
0.88
0.89
0.9
0.91
0.92
0.93
10 20 30 40 50
Fmeaure
Scan
Fmeaure
McovNets
3D based DN
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

Error rate
Training
Epoch
McovNets
3D based
DN
Interleave
3DCNN
MR3DCN
N-KT
IO-
MR3DCN
N-KT
0 5.3 5.1 4.9 4.8 4.8
100 4.3 4.1 4 3.8 3.5
200 4.1 3.9 3.8 3.4 3.1
300 3.6 3.5 3.1 3 2.8
400 3.5 3.4 2.7 2.4 2.1
500 3.4 3.2 2.3 2.1 1.8
0
1
2
3
4
5
6
0 100 200 300 400 500
Error
rate
Training Epoch
Error rate
McovNets
3D based DN
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

Separability
CNN Layer
McovNet
s
3D
based
DN
Interleav
e
3DCNN
MR3DCN
N-KT
IO-
MR3DCN
N-KT
Input 0.2 0.21 0.23 0.25 0.3
convolutiona
l
0.23 0.26 0.28 0.31 0.34
Revolution 0.92 0.95 1 1.1 1.3
Pooling 0.96 0.98 1.1 1.3 1.6
FCL 1.2 1.3 1.5 1.7 1.8
Softmax 2.6 2.27 2.35 2.45 2.6
0
0.5
1
1.5
2
2.5
3
separability
CNN Layer
Separability
McovNets
3D based DN
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

5.3. Reducing the complexity of Multi-Resolution
3D CNN by adding depthwise separable
architecture

 In this phase of the research work , the lung nodule images are extracted both the spatial and temporal information
by using the bottleneck-based 3D depthwise separable CNN architecture.
 In this architecture, the images are captured in the basis of 3d dimensional features.
 Moreover, the lung nodule image is scattered.
 The efficient learning of specific location information is achieved by using the concept of depthwise convolution
in each lung image.
 For the temporal information the 3d pointwise convolution can be used to learn the linear combination among
sequential frames.
 3d model performs two types of the convolutional filter methods for lung nodule detection are
1) point-wise convolutional filter
2) depthwise convolutional filter
50
IO-MR3D-DSCNN -KT

51
IO-MR3D-DSCNN -KT
 A 3D depth wise separable convolutional neural network is used to effectively infer the intraction
force only from a lung nodule images.
 The spatial-temporal information are identified by using 3D Depthwise convolutional neural
network.
 spatial feature extraction: The 2D depthwise convolution was applied to each frame of the input
video, that is the process of learning the spatial information independent of the channel was applied
to each frame.
 Temporal feature extraction: The 3D pointwise convolution was applied to learn the linear
combination among the channels of adjacent lung nodule images.
Depthwise Convolution Filter

52
IO-MR3D-DSCNN -KT
Depthwise Convolution Filter
 The proposed 3D depthwise separable CNN method first extract the spatial feature information
based on the 2D DWC method in the first sequential image from the video clip; the same DWC
filters are used for the other sequential images.
 Thus, the diagram shared marks were found on frames 2 and 3.
 In that, shared weight parameters are used in the 3D Depthwise separable CNN, and the no of a
parameters are not significantly increased when compared with the traditional 3D CNN.
 At last of the 3D depthwise separable CNN method, the 3D Point-wise convolutional filter were
employed to extract the temporal feature information.

53
IO-MR3D-DSCNN -KT
(a) 3D CNN (b) 3D depthwise separabele 3D CNN

54
IO-MR3D-DSCNN -KT
Depthwise Vs pointwise convolutional network

 DWC filters are trained independently for each channel and do not successfully utilize the spatial
information among lung nodule images.
 Consequently, this can lead to performance degradation. In, the depthwise separable convolution
(DSC) filter, which combines the DWC filter with the pointwise convolution (PWC) filter, FPWC ∈
K1×1, for learning the correlation among the channels in the layer ends are used.
 The dsc-based methods could overcome the dwc’s deficiency (i.E., The absence of learning the
information among channels) and simultaneously reduce the computational complexity involved in
the deep learning network.
 In the end, the dsc-based approach is similar to decomposing the existing 2d convolutional layer
into two separate procedures are dwc layer and pwc layer.
55
IO-MR3D-DSCNN -KT

56
Experimental analysis
1. Hybrid Spectral Convolutional Neural Network (HybridSN)
2. Fully Convolutional Neural Network (CFCNs)
3. Interleaved 3D CNN
4. Iteratively Optimized Multi-Resolution 3 Dimensional CNN and
Knowledge Transfer (IO-MR3DCNN-KT)
5. Iteratively Optimized Multi-Resolution 3 Dimensional Depthwise
Separable CNN and Knowledge Transfer (IO-MR3D-DSCNN-
KT)...proposed

57
(a) (b) (c)
Fig 5.15 Results of Lung Nodule Candidate Detection Models: (a) Input Image (b) Detected Nodules
using HybridSN (c) Detected Nodules using CFCNs

58
(a) (b) (c) (d)
Fig 5.15 Results of Lung Nodule Candidate Detection Models: (a) Detected Nodules using Interleaved
CN (b) Detected Nodules using MR3DCNN-KT (c) Detected Nodules using IO-MR3DCNN-KT (d)
Detected Nodules using IO-MR3D-DSCNN-KT

59
Accuracy
Scans
Hybrid
SN
CFCNs
Interlea
ved
3DCNN
MR3D
CNN-
KT
IO-
MR3D
CNN-
KT
IO-
MR3D-
DSCN
N-KT
10 0.81 0.83 0.84 0.91 0.94 0.95
20 0.82 0.84 0.85 0.93 0.96 0.97
30 0.83 0.85 0.86 0.95 0.97 0.99
40 0.84 0.86 0.87 0.96 0.98 0.9956
50 0.86 0.87 0.89 0.97 0.99 0.9959
0
0.2
0.4
0.6
0.8
1
1.2
10 20 30 40 50
Accuracy
Scan
Accuracy
HybridSN
CFCNs
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT

60
Precision
Scans HybridSN CFCNs Interleave
d 3DCNN
MR3DCNN-
KT
IO-
MR3DCNN
-KT
IO-MR3D-
DSCNN-KT
10 0.8788 0.8892 0.8952 0.9145 0.9385 0.9558
20 0.8879 0.8893 0.8943 0.9146 0.9387 0.9556
30 0.8881 0.8894 0.8948 0.9152 0.9389 0.9562
40 0.8882 0.8897 0.8951 0.9157 0.9391 0.9563
50 0.8884 0.8899 0.8952 0.9158 0.9392 0.9563
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
10 20 30 40 50
Precison
Scans
Precison
HybridSN
CFCNs
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT

61
Recall
Scans HybridSN CFCNs
Interleaved
3DCNN
MR3DCNN-
KT
IO-
MR3DCNN
-KT
IO-MR3D-
DSCNN-KT
10 0.8891 0.8924 0.8986 0.9115 0.9143 0.9249
20 0.8893 0.8925 0.8988 0.9116 0.9145 0.9249
30 0.8895 0.8927 0.899 0.9117 0.9147 0.9251
40 0.8896 0.8929 0.8992 0.9118 0.9148 0.9253
50 0.8899 0.8931 0.8995 0.9119 0.9149 0.9254
0.87
0.88
0.89
0.9
0.91
0.92
0.93
10 20 30 40 50
Recall
Scan
Recall
HybridSN
CFCNs
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT

62
F measure
Fig 5.19. Comparison of F measure
Table 5.16 Comparison of F measure
Scans
Hybrid
SN
CFCNs
Interlea
ved
3DCNN
MR3D
CNN-
KT
IO-
MR3D
CNN-
KT
IO-
MR3D-
DSCN
N-KT
10 0.8891 0.8924 0.8981 0.914 0.9263 0.9354
20 0.8893 0.8926 0.8983 0.916 0.9265 0.9356
30 0.8895 0.8927 0.8985 0.919 0.9267 0.9358
40 0.8896 0.8928 0.8987 0.921 0.9268 0.936
50 0.8898 0.8929 0.8989 0.924 0.9269 0.9363
0.86
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
10 20 30 40 50
Fmeaure
Scan
Fmeaure
HybridSN
CFCNs
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT

63
Error rate
Trainin
g
Epoch
Hybrid
SN
CFCNs
Interlea
ved
3DCNN
MR3D
CNN-
KT
IO-
MR3D
CNN-
KT
IO-
MR3D-
DSCN
N-KT
0 5.3 5.1 4.9 4.8 4.8 4.8
100 4.3 4.1 4 3.8 3.5 3.2
200 3.8 3.7 3.5 3.4 3.1 2.9
300 3.7 3.3 3.1 3 2.8 2.6
400 2.8 2.7 2.6 2.4 2.1 1.8
500 2.9 2.6 2.3 2.1 1.8 1.5
0
1
2
3
4
5
6
0 100 200 300 400 500
Error
rate
Training Epoch
Error rate
HybridSN
CFCNs
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

64
Separability
Fig 5.21. Comparison of separability
Table 5.18. Comparison of separability
CNN
Layer
Hybrid
SN
CFCNs
Interlea
ved
3DCNN
MR3D
CNN-
KT
IO-
MR3D
CNN-
KT
IO-
MR3D-
DSCN
N-KT
Input 0.21 0.22 0.23 0.25 0.3 0.31
convolutio
nal
0.23 0.26 0.28 0.31 0.34 0.4
Revolution 0.96 0.97 0.98 1.1 1.3 1.4
Pooling 0.98 1 1.1 1.3 1.6 1.8
FCL 1.1 1.3 1.5 1.7 1.8 2
Softmax 2.21 2.24 2.35 2.45 2.6 2.8
0
0.5
1
1.5
2
2.5
3
separability
CNN Layer
Separability
HybridSN
CFCNs
Interleaved 3DCNN
MR3DCNN-KT
IO-MR3DCNN-KT

65
5.4. Geometrical features with CNN features for
enhancing lung nodule candidates classification

In the earlier phase of this research work, only dimensional lung nodules are considered
when determining and characterizing lung nodules.
It is more difficult to manually and semi-automatically calculate nodules and their error of
measuring instrument when used under reference condition.
In this respect, the error calculates the difference in the calculation of nodule.
In this phase of the research work, the different sizes of lung nodules are predicted for the
diagnosis of lung cancer.
Two different phase are considered in the iteratively optimized multi-resolution feature
enriched 3 dimensional depthwise separable cnn and knowledge transfer (io-mr3d-
fedscnn-kt) for prediction of various size lung nodule.
66
5. Research Contribution
IO-MR3D- FEDSCNN-KT

 Two stream bilinear model are used for prediction of various sizes of lung nodule.
 The first stream aims to predict keypoints in 3-d feature maps which is pretrained with the supervision of lung
image positions.
 Since it functions as taking attention on discriminative regions automatically.
 It is called an attention stream.
 The second feature stream aims to capture geometrical features such as
 Nodule range,
 Location of nodule,
 Comparative brightness variations of the central pixel,
 Number of pixel inside nodule,
 Primary and secondary rotational moments,
 Gradient orientation central pixel,
 Perimeter,
 Ratio between the highest and least rotational moment,
 Perimeter of nodule,
 Average,
67
IO-MR3D-FEDSCNN-KT

68
5. Research Contribution
Two stream bilinear C3D
68
Geometrical features

69
EMRDJDA
Experimental Analysis on Performance metrics
1. CAD based Low Dose CT (LDCT)
2. Multiclass Support Vector Machine (SVM)
3. Iteratively Optimized Multi-Resolution 3 Dimensional Depthwise
Separable CNN and Knowledge Transfer (IO-MR3D-DSCNN-KT)
4. Iteratively Optimized Multi-Resolution Feature Enriched 3 Dimensional
Depthwise Separable CNN and Knowledge Transfer (IO-MR3D-
FEDSCNN-KT)...Proposed

70
(a) (b) (c)
Figure 5.22 Results of Lung Nodule Candidate Detection Models: (a) Input Image (b) Detected Nodules
using LDCT (c) Detected Nodules using Multiclass SVM

71
(a) (b) (c) (d)
Figure 5.22 Results of Lung Nodule Candidate Detection Models: (a) Detected Nodules using IO-
MR3DCNN-KT (b) Detected Nodules using IO-MR3D-DSCNN-KT IO-MR3D-DSCNN–KT (c) Detected

72
Accuracy
Scans LDCT
Multiclas
s SVM
MR3DCNN-
KT
IO-
MR3DCNN-
KT
IO-MR3D-
DSCNN-
KT
IO-MR3D-
FEDSCNN-
KT
10 0.8925 0.8992 0.91 0.94 0.95 0.97
20 0.8927 0.8994 0.93 0.96 0.97 0.98
30 0.8929 0.8995 0.95 0.97 0.99 0.99
40 0.8931 0.8997 0.96 0.98 0.9956 0.9987
50 0.8934 0.8998 0.97 0.99 0.9959 0.9989
0.82
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
1
1.02
10 20 30 40 50
Accuracy
Scan
Accuracy
LDCT
Multiclass SVM
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT
IO-MR3D-FEDSCNN-KT

73
Precision
Table 5.20 Comparison of Precision
Scans LDCT
Multiclass
SVM
MR3DCNN-
KT
IO-
MR3DCN
N-KT
IO-MR3D-
DSCNN-KT
IO-MR3D-
FEDSCNN-
KT
10 0.8862 0.8893 0.9145 0.9385 0.9558 0.9604
20 0.8864 0.8895 0.9146 0.9387 0.9556 0.9604
30 0.8866 0.8896 0.9152 0.9389 0.9562 0.9606
40 0.8868 0.8897 0.9157 0.9391 0.9563 0.9608
50 0.887 0.8898 0.9158 0.9392 0.9563 0.961
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
10 20 30 40 50
Precison
Scans
Precison
LDCT
Multiclass SVM
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT
IO-MR3D-FEDSCNN-KT

74
Recall
Scan
s
LDC
T
Multiclas
s SVM
MR3DCNN-
KT
IO-
MR3DCN
N-KT
IO-MR3D-
DSCNN-KT
IO-MR3D-
FEDSCNN
-KT
10 0.8878 0.8888 0.9115 0.9143 0.9249 0.9337
20 0.888 0.889 0.9116 0.9145 0.9249 0.9339
30 0.8881 0.8895 0.9117 0.9147 0.9251 0.9334
40 0.8883 0.8897 0.9118 0.9148 0.9253 0.9342
50 0.8884 0.8899 0.9119 0.9149 0.9254 0.9345
0.86
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
10 20 30 40 50
Recall
Scan
Recall
LDCT
Multiclass SVM
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT
IO-MR3D-FEDSCNN-KT

75
F Measure
Fig 5.26. Comparison of F measure
Table 5.22. Comparison of F measure
Scan
s
LDC
T
Multiclas
s SVM
MR3DCNN
-KT
IO-
MR3DCNN-
KT
IO-MR3D-
DSCNN-KT
IO-MR3D-
FEDSCNN
-KT
10 0.8862 0.8892 0.914 0.9263 0.9354 0.9404
20 0.8864 0.8894 0.916 0.9265 0.9356 0.9408
30 0.8865 0.8896 0.919 0.9267 0.9358 0.941
40 0.8867 0.8898 0.921 0.9268 0.936 0.9414
50 0.8869 0.8899 0.924 0.9269 0.9363 0.9418
0.85
0.86
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
10 20 30 40 50
Fmeaure
Scan
Fmeaure
LDCT
Multiclass SVM
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT
IO-MR3D-FEDSCNN-KT

76
Error rate
Training
Epoch
LDCT
Multicla
ss SVM
MR3DC
NN-KT
IO-
MR3DC
NN-KT
IO-
MR3D-
DSCNN-
KT
IO-
MR3D-
FEDSC
NN-KT
0 4.8 4.8 4.8 4.8 4.8 4.8
100 4.2 4 3.8 3.5 3.2 3
200 3.9 3.7 3.4 3.1 2.9 2.5
300 3.5 3.2 3 2.8 2.6 2.2
400 2.4 2.3 2.4 2.1 1.8 0.94
500 2.3 2.2 2.1 1.8 1.5 0.5
0
1
2
3
4
5
6
0 100 200 300 400 500
Error
rate
Training Epoch
Error rate
LDCT
Multiclass SVM
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT

77
Separability
Fig 5.28. Comparison of separability
Table 5.24. Comparison of separability
CNN Layer LDCT
Multiclass
SVM
MR3DCNN
-KT
IO-
MR3DCNN
-KT
IO-
MR3D-
DSCNN
-KT
IO-MR3D-
FEDSCNN
-KT
Input 0.2 0.22 0.25 0.3 0.31 0.4
convolutional 0.27 0.29 0.31 0.34 0.4 0.45
Revolution 0.96 1 1.1 1.3 1.4 1.6
Pooling 1 1.2 1.3 1.6 1.8 2
FCL 1.2 1.5 1.7 1.8 2 2.4
Softmax 2.36 2.4 2.45 2.6 2.8 2.9
0
0.5
1
1.5
2
2.5
3
separability
CNN Layer
Separability
LDCT
Multiclass SVM
MR3DCNN-KT
IO-MR3DCNN-KT
IO-MR3D-DSCNN-KT

6. Over all Result and Discussion
78

79
Overall Result and Discussion
(a) (b) (c) (d) (e) (f)
Figure 7.1. Results of lung nodule detection techniques: (a) Input Image (b) Results of MRCNN-KT. (c) Results of
MR3DCNN -KT. (d) Results of IO- MR3DCNN -KT. (e) Results of IO-MR3D-DSCNN -KT. (f) Results of IO-
MR3D- FEDSCNN.

80
Accuracy
Table 6.1. Comparing Accuracy
Fig 6.2. Comparing Accuracy
MRCNN-KT
MR3DCNN
-
KT
IO-
MR3DCNN
-KT
IO-MR3D-
DSCNN
-KT
IO-MR3D-
FEDSCNN
Accuracy 0.88 0.91 0.94 0.95 0.97

81
Precision
Table 6.2. Comparing Precision
Fig 6.3 Comparing Precision
MRCNN-KT
MR3DCNN
-
KT
IO-
MR3DCNN
-KT
IO-MR3D-
DSCNN
-KT
IO-MR3D-
FEDSCNN
Precision 0.8866 0.9145 0.9385 0.9558 0.9704

82
Recall
Table 6.3. Comparing Recall
Fig 6.4. Comparing Recall
MRCNN-KT
MR3DCNN
-
KT
IO-
MR3DCNN
-KT
IO-MR3D-
DSCNN
-KT
IO-MR3D-
FEDSCNN
Recall 0.8868 0.9135 0.93 0.9549 0.97

83
F measure
Table 6.4. Comparing F measure
Fig 10.5. Comparing F measure
MRCNN-KT
MR3DCNN
-
KT
IO-
MR3DCNN
-KT
IO-MR3D-
DSCNN
-KT
IO-MR3D-
FEDSCNN
F-Measure 0.8867 0.9140 0.935 0.9554 0.9704

84
Error rate
Table 6.4. Comparing Error rate
Fig 10.5. Comparing Error rate
CNN
LAYER
MRCNN-
KT
MR3DCN
N
-KT
IO-
MR3DCN
N
-KT
IO-
MR3D-
DSCNN
-
KT
IO-
MR3D-
FEDSCN
N
0 4.8 4.8 4.8 4.8 4.8
100 4 3.8 3.5 3.2 3
200 3.6 3.4 3.1 2.9 2.5
300 3.1 3 2.8 2.6 2.2
400 2.9 2.4 2.1 1.8 0.94
500 2.5 2.1 1.8 1.5 0.5

85
Separability
Table 6.4. Comparing Separability
Fig 10.5. Comparing separability
CNN
LAYER
MRCNN-
KT
MR3DCN
N
-KT
IO-
MR3DCN
N
-KT
IO-MR3D-
DSCNN
-
KT
IO-MR3D-
FEDSCNN
Input 0.2 0.25 0.3 0.31 0.4
Convolutional 0.3 0.31 0.34 0.4 0.45
Revolution 0.9 1.1 1.3 1.4 1.6
Pooling 1.1 1.3 1.6 1.8 2
FCL 1.5 1.7 1.8 2 2.4
Softmax 2.3 2.45 2.6 2.8 2.9

86
 The performance of proposed lung cancer detection techniques is analysed by
comparing with other existing techniques using MATLAB 2018a.
 In this experiment, a patient lung CT scan dataset from Kaggle’s Data Science
Bowl 2017 dataset is used. This d dataset contains labeled data for 2101
patients, in which 1261 data are used for training and 840 data are used for testing
process.
 For each patient the data consists of CT scan data and a label (0 for no cancer, 1
for cancer). The comparative analysis is made in terms of accuracy, precision,
recall, F-measure, error rate and seperability.
 It is proved that the proposed IO-MR3D-FEDSCNN-KT technique has high
accuracy, precision, recall, f-measure and seperability and lower error rate than
other lung cancer detection techniques.
 The accuracy of proposed method (IO-MR3D- FEDSCNN -KT) is 10.22% greater
than existing method (MRCNN-KT) for lung nodule detection.

87
 The precision of proposed method (IO-MR3D- FEDSCNN -KT) is 9.45% greater
The Recall of proposed method (IO-MR3D- FEDSCNN -KT) is 9.38% greater than
existing method (MRCNN-KT) for lung nodule detection.
The F Measure of proposed method (IO-MR3D- FEDSCNN -KT) is 9.43% greater
The error rate of proposed method (IO-MR3D- FEDSCNN -KT) is 66.6% lower than
existing method (MRCNN-KT) for lung nodule detection.
The seperability of proposed method (IO-MR3D- FEDSCNN -KT) is 3.57% greater
 This is concluded that the IO-MR3D-FEDSCNN-KT technique effectively detects
the lung cancer.

 The main aim of this research is increasing the detection accuracy of lung cancer.
 Initially, MR3DCNN-KT method is proposed to extract the features from spatio-temporal dimensions for obtaining the
contextual information between slices.
 But, it is time-consuming for large-scale datasets.
 So, IO-MR3DCNN-KT method is proposed to enable an automatic labeling for large-scale datasets and reduce the false
detection rate.
 But, this method has high complexity.
 As a result, IO-MR3D-DSCNN-KT method is proposed for reducing the parameter size of convolutional filters without
degrading the detection accuracy.
 However, the sized-based measurement of the lung nodule is still a challenging process.
 Therefore, IO-MR3D-FEDSCNN method is proposed to predict the various sizes of lung nodules based on different
features for increasing the detection accuracy efficiently.
 Finally, the experimental results proved that the proposed IO-MR3D-FEDSCNN method achieves better performance
than the other method in terms of precision, recall, f-measure and accuracy.
88
7. Conclusion

1. Huang, Z. K., & Chau, K. W. (2008). A new image thresholding method based on Gaussian mixture
model. Applied mathematics and computation, 205(2), 899-907
2. Bach, P. B., Mirkin, J. N., Oliver, T. K., Azzoli, C. G., Berry, D. A., Brawley, O. W., & Sabichi, A. L. (2012).
Benefits and harms of CT screening for lung cancer: a systematic review. Jama, 307(22), 2418-2429
3. Ji, S., Xu, W., Yang, M., & Yu, K. (2012). 3D convolutional neural networks for human action
recognition. IEEE transactions on pattern analysis and machine intelligence, 35(1), 221-231.
4. Li, F., Arimura, H., Suzuki, K., Shiraishi, J., Li, Q., Abe, H., & Doi, K. (2005). Computer-aided detection of
peripheral lung cancers missed at CT: ROC analyses without and with localization. Radiology, 237(2), 684-
690
5. Rubin, G. D., Lyo, J. K., Paik, D. S., Sherbondy, A. J., Chow, L. C., Leung, A. N., & Napel, S. (2005).
Pulmonary nodules on multi–detector row CT scans: performance comparison of radiologists and
computer-aided detection. Radiology, 234(1), 274-283
6. Armato III, S. G., McLennan, G., Bidaut, L., McNitt‐Gray, M. F., Meyer, C. R., Reeves, A. P., & Kazerooni, E.
A. (2011). The lung image database consortium (LIDC) and image database resource initiative (IDRI): a
completed reference database of lung nodules on CT scans. Medical physics, 38(2), 915-931
7. Liu, K., & Kang, G. (2017). Multiview convolutional neural networks for lung nodule
classification. International Journal of Imaging Systems and Technology, 27(1), 12-22
8. Shen, W., Zhou, M., Yang, F., Yu, D., Dong, D., Yang, C., ... & Tian, J. (2017). Multi-crop convolutional
neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognition, 61, 663-673
90

9. Aresta, G., Cunha, A., & Campilho, A. (2017, March). Detection of juxta-pleural lung nodules in
computed tomography images. In Medical Imaging 2017: Computer-Aided Diagnosis (Vol.
10134, p. 101343N). International Society for Optics and Photonics
10. Zhang, J., Xia, Y., Zeng, H., & Zhang, Y. (2018). NODULe: Combining constrained multi-scale
LoG filters with densely dilated 3D deep convolutional neural network for pulmonary nodule
detection. Neurocomputing, 317, 159-167
11. Tang, H., Kim, D. R., & Xie, X. (2018, April). Automated pulmonary nodule detection using 3D
deep convolutional neural networks. In 2018 IEEE 15th International Symposium on Biomedical
Imaging (ISBI 2018) (pp. 523-526). IEEE
12. Gong, J., Liu, J. Y., Wang, L. J., Sun, X. W., Zheng, B., & Nie, S. D. (2018). Automatic detection
of pulmonary nodules in CT images by incorporating 3D tensor filtering with local image feature
analysis. Physica Medica, 46, 124-133
13. Da Silva, G. L. F., Valente, T. L. A., Silva, A. C., de Paiva, A. C., & Gattass, M. (2018).
Convolutional neural network-based PSO for lung nodule false positive reduction on CT
images. Computer methods and programs in biomedicine, 162, 109-118
14. Rodrigues, M. B., Da Nóbrega, R. V. M., Alves, S. S. A., Rebouças Filho, P. P., Duarte, J. B. F.,
Sangaiah, A. K., & De Albuquerque, V. H. C. (2018). Health of things algorithms for malignancy
level classification of lung nodules. IEEE Access, 6, 18592-18601.
15. Ali, I., Hart, G. R., Gunabushanam, G., Liang, Y., Muhammad, W., Nartowt, B., & Deng, J. (2018).
91

9. Tran, G. S., Nghiem, T. P., Nguyen, V. T., Luong, C. M., & Burie, J. C. (2019).
Improving Accuracy of Lung Nodule Classification Using Deep Learning with Focal
Loss. Journal of healthcare engineering, 2019
10.Li, Y., Zhang, L., Chen, H., & Yang, N. (2019). Lung Nodule Detection With Deep
Learning in 3D Thoracic MR Images. IEEE Access, 7, 37822-37832.
11.Onishi, Y., Teramoto, A., Tsujimoto, M., Tsukamoto, T., Saito, K., Toyama, H. & Fujita,
H. (2019). Automated Pulmonary Nodule Classification in Computed Tomography
Images Using a Deep Convolutional Neural Network Trained by Generative
Adversarial Networks. BioMed research international, 2019
12.Zuo, W., Zhou, F., Li, Z., & Wang, L. (2019). Multi-Resolution CNN and Knowledge
Transfer for Candidate Classification in Lung Nodule Detection. Ieee Access, 7,
32510-32521.
13.Gu, Y., Lu, X., Zhang, B., Zhao, Y., Yu, D., Gao, L., ... & Zhou, T. (2019). Automatic
lung nodule detection using multi-scale dot nodule-enhancement filter and weighted
support vector machines in chest computed tomography. PloS one, 14(1)
92

PPT.pptx

More Related Content

What's hot (6)

Similar to PPT.pptx (20)

Recently uploaded (20)

PPT.pptx