Lec11: Active Contour and Level Set for Medical Image Segmentation

MEDICAL IMAGE COMPUTING (CAP 5937)
LECTURE 11: Active Contour and Level Set for Medical Image
Segmentation
Dr. Ulas Bagci
HEC 221, Center for Research in Computer
Vision (CRCV), University of Central Florida
(UCF), Orlando, FL 32814.
bagci@ucf.edu or bagci@crcv.ucf.edu
1SPRING 2017

Outline
• Active Contour (Snake)
• Level Set
• Applications
2

Motivation
• Active contours and active surfaces are means of model-
driven segmentation. Their use enforces closed and smooth
boundaries for each segmentation irrespective of the image
content.
3

Motivation
content.
• Data-driven (region) approaches: Objects in an image
appear homogeneous
4

Motivation
content.
appear homogeneous
• Model-driven (boundary) approaches: Ideal object
boundary are predicted. The boundary is assumed to be
smooth and closed.
5

Motivation
content.
appear homogeneous
• Model-driven (boundary) approaches: Ideal object
boundary are predicted. The boundary is assumed to be
smooth and closed.
6

Active Contours (Snake)
• First introduced in 1987 by Kass et al, and gained popularity
since then.
• Represents an object boundary or some other salient image
feature as a parametric curve.
• An energy functional E is associated with the curve.
• The problem of finding object boundary is cast as an energy
minimization problem.
13

A Snake is a parametric curve!
14
The course of the snake smoothly follows high intensity gradients if the gradients
reliably reflect the object boundary.Otherwise,a smooth boundary is generated
bridging regions of noisy data or missing gradients. Such an active contouris
particularly well suited to segmentan object instance in an image where the
data are distorted by noise or artefacts

Frameworks for Snakes
• A higher level process or a user initializes any curve close to
the object boundary.
• The snake then starts deforming and moving towards the
desired object boundary.
• In the end it completely “shrink-wraps” around the object.
15

Deformable Models
• Deformable models are curves or surfaces defined within an
image domain that can move under the influence of internal
forces,
16

Deformable Models
forces, which are defined within the curve or surface itself,
17

Deformable Models
and external forces, which are computed from the image
data.
18

Deformable Models
and external forces, which are computed from the image
data.
19
The internal forces are designed to keep the model smooth during deformation.
The external forces are defined to move the model toward an object boundary
or other desired features within an image.

Active Contour Modeling
• The contour is defined in the (x, y) plane of an image as a parametric
curve
• Contour is said to possess an energy (Esnake) which is defined as the
sum of the three energy terms.
• The energy terms are defined cleverly in a way such that the final position
of the contour will have a minimum energy (Emin)
• Therefore our problem of detecting objects reduces to an energy
minimization problem.
20
int intsnake ernal external constraE E E E= + +
What are these energy terms which do the trick for us ?
1s0))s(y),s(x()s( ≤≤=ν

Internal Energy
• The smoothness energy at contour point v(s) could be
evaluated as
21
Elasticity/stretching Stiffness/bending
sd
d
ds
d
sssEin
2
2
)()())((
22
νν βαν +=
Then, the interior energy (smoothness) of the whole snake
∫=
1
0
inin ds))s((EE ν]}1,0[s|)s({ ∈= νC

Internal Energy
22
5v
4v
3v
2v
1v 6v
7v
8v
10v
9v
elastic energy
(elasticity)
i1iv
ds
d
ν
ν
−≈ +
bending energy
(stiffness)
1ii1i1iii1i2
2
2)()(
ds
d
−+−+ ν+ν−ν=ν−ν−ν−ν≈
ν
)( iii y,x=ν
2n
)( ℜ∈= −1n210 ,....,,, ννννC

Internal Energy
10/13/15
23
Elasticity Stiffness
i1iv
ds
d
ν
ν
−≈ +
11112
2
2)()( −+−+ +−=−−−≈ iiiiiii
ds
d
ννννννν
ν
∑
−
=
−++ +−+−=
1
0
2
11
2
1 |2|||
n
i
iiiiiinE νννβννα
)( iii y,x=ν
2n
)( ℜ∈= −1n210 ,....,,, ννννC
Min energy when curve minimizes length of contour…..... …...........is smooth

External Energy
• The external energy describes how well the curve matches
the image data locally
• Numerous forms can be used, attracting the curve toward
different image features
24

External (Image) Energy
• Suppose we have an image I(x,y)
• Can compute image gradient at any point
• Edge strength at pixel (x,y) is
• External energy of a contour point v=(x,y) could be
25
|)y,x(I|∇
)I,I(I yx=∇
22
|),(||)(|)( yxIIEex ∇−=∇−= vv
∑
−
=
=
1
0
)(
n
i
iexex EE ν discrete case
}ni0|{ i <≤= νC
∫=
1
0
))(( dssEE exex ν continuous case
]}1,0[s|)s({ ∈= νC
■ External energy term for the whole snake is

Basic Elastic Snake
26
• The total energy of a basic elastic snake is
continuous case
discrete case
∫∫ ∇−⋅=
1
0
2
1
0
2
ds|))s(v(I|ds|
ds
dv
|E α
∑∑
−
=
−
=
+ ∇−−⋅=
1n
0i
2
i
1n
0i
2
i1i |)v(I||vv|E α
elastic smoothness term
(interior energy)
image data term
(exterior energy)
]}1,0[s|)s({ ∈= νC
}ni0|{ i <≤= νC
(PS. bending energy can be added under elastic term)

Basic Elastic Snake
27
This can make a curve shrink
(to a point)
∑
−
=
⋅=
1
0
2
n
i
iin LE α
2
1
1
0
2
1 )()( ii
n
i
ii yyxx −+−⋅= +
−
=
+∑α
∑
−
=
∇−=
1n
0i
2
iiex |)y,x(I|E
2
1
0
2
|),(||),(| iiy
n
i
iix yxIyxI∑
−
=
+−=
)y,x,....,y,x,y,x()ni0|( 1n1n1100i −−=<≤= νC
C
i
i-1 i+1
i+2
Li-1 Li
Li+1

Find Contour C that minimizes E(C)
28
2
iiy
1n
0i
2
iix
2
i1i
1n
0i
2
i1i |)y,x(I||)y,x(I|)yy()xx()(E ∑∑
−
=
+
−
=
+ +−−+−⋅= αC
Optimization problem for function of 2n variables
- can compute local minima via gradient descent
- more robust option: dynamic programming

Constraint Forces (Econstraints)
• Initial snake result can be nudged where it goes wrong, simply add
extra external energy terms to
• Pull nearby points toward cursor, or
• Push nearby points away from cursor
29
∑
−
= −
−=
1
0
2
2
||
n
i i
pull
p
r
E
ν
∑
−
= −
+=
1
0
2
2
||
n
i i
push
p
r
E
ν

Ex: If Only External Force is Used
30
Red: initial contour
Green: final contour
Credit: Scot Acton

Gradient Descent
31
⎥
⎦
⎤
⎢
⎣
⎡
−
−
=
∂
∂
∂
∂
y
E
x
E
E∇−
negative gradient at point (x,y) gives direction of the
steepest descent towards lower values of function E
• Example: minimization of functions of 2 variables
),( 00 yx
),( yxE
y
x

Gradient Descent
32
Et ∇⋅Δ−=′ pp
),( yxE
⎟
⎟
⎠
⎞
⎜
⎜
⎝
⎛
⋅Δ−⎟⎟
⎠
⎞
⎜⎜
⎝
⎛
=⎟⎟
⎠
⎞
⎜⎜
⎝
⎛
′
′
∂
∂
∂
∂
y
E
x
E
t
y
x
y
x
Stop at a local minima where 0
!
=∇E
y
x
),( 00 yx
update equation for a point p=(x,y)

Gradient Descent
33
High sensitivity wrt. the initialisation !!
),( yxE
x
y

Gradient Descent in Snakes
34
simple elastic snake energy
tE' Δ⋅∇−= CC
update equation for the whole snake
t...
y
x
...
y
x
'y
'x
...
'y
'x
1n
1n
0
0
y
E
x
E
y
E
x
E
1n
1n
0
0
1n
1n
0
0
Δ⋅
⎟
⎟
⎟
⎟
⎟
⎟
⎠
⎞
⎜
⎜
⎜
⎜
⎜
⎜
⎝
⎛
−
⎟⎟
⎟
⎟
⎟
⎟
⎠
⎞
⎜⎜
⎜
⎜
⎜
⎜
⎝
⎛
=
⎟⎟
⎟
⎟
⎟
⎟
⎠
⎞
⎜⎜
⎜
⎜
⎜
⎜
⎝
⎛
−
−
∂
∂
∂
∂
∂
∂
∂
∂
−
−
−
−
C
2
1
1
0
2
1 )()( ii
n
i
ii yyxx −+−⋅+ +
−
=
+∑α
2
iiy
1n
0i
2
iix1n01n0 |)y,x(I||)y,x(I|)y,,y,x,,x(E ∑
−
=
−− +−=!!
here, energy is a function of 2n variables
C

Dynamic Programming for Snakes
• Please Read
– Interactive Segmentation with Intelligent Scissors by E. Mortensen
and W. Barrett,
– Using Dynamic Programming for Solving variational Problems in
Vision by AA. Amini et al, where authors used dynamic programming
for image segmentation tasks.
35

Problems with Snakes
10/13/15
37
• Depends on number and spacing of control points
• Snake may over-smooth the boundary
• Initialization is crucial
• Not trivial to prevent curve self intersecting
• May not follow topological changes of objects

Level Sets
• A limitation of active contours based on parametric curves of
the form f(s) (snakes, b-snakes,…) is that it is challenging to
change the topology of the curve as it evolves.
• If the shape changes dramatically, curve reparameterization
may also be required.
• An alternative representation for such closed contours is to
use level sets (LS).
– LS evolve to fit and track objects of interest by modifying the
underlying embedding function instead of curve function f(s)
38

Image Segmentation with Level Sets
• Contour evolution(Sethian and Osher, 1988)
• Level sets for closed contours
– Zero-crossing(s) of a characteristic function define the
curve
– Fit and track objects of interest by modifying the underlying
embedding function instead of the curve f(s)
– Efficient algorithm
• A small strip around the locations of the current zero-crossing
needs to updated at each step
39
( , )x yφ
Fast Marching Methods

Moving Interfaces
• 2D Moving Curves
• 3D Moving Surfaces
Ex:
– Interfaces between water and oil
– Propagating front of bush fire
– Deformable elastic solid
40
water
air

Evolving Curves and Surfaces
41
Only velocity component
normal to surface is important!

Describe curve as Level Sets of a Function
42
(x, y) = x2
+ y2
1 = 0
Isocontouris the unit circle (implicit represt.)

43
(x, y) = x2
+ y2
1 = 0
A few isocontours of two
dimensional function (circle)
Along with some representative
normals.
GRADIENT:
5 = (
@
@x
,
@
@y
)

44
~N =
5
| 5 |
Then, unit normal (outward) is

45
~N =
5
| 5 |
On Cartesian grid, we need to approximate this equation (ex. Finite difference techniques):
@
@x
⇡
i+1 i
x

46
~N =
5
| 5 |
@
@x
⇡
i+1 i
x
Mean curvature of the interface is defined as the divergence of the normal ~N = (n1, n2)
 = r. ~N =
@n1
@x
+
@n2
@y
= r.(
r
|r |
)

47
~N =
5
| 5 |
@
@x
⇡
i+1 i
x
Mean curvature of the interface is defined as the divergence of the normal ~N = (n1, n2)
 = r. ~N =
@n1
@x
+
@n2
@y
= r.(
r
|r |
)

Variational Formulations and LS
• Transition from Active Contours:
– contour v(t) → front γ(t)
– contour energy → forces FA FC
– image energy → speed function kI
• Level set:
– The level set c0 at time t of a function ψ(x,y,t)
is the set of arguments { (x,y) , ψ(x,y,t) = c0 }
– Idea: define a function ψ(x,y,t) so that at any time,
γ(t) = { (x,y) , ψ(x,y,t) = 0 }
• there are many such ψ
• ψ has many other level sets, more or less parallel to γ
• only γ has a meaning for segmentation,not any other level set of ψ
48

49
Usual choice for ψ: signed distance to the front γ(0)
⎧ - d(x,y, γ) if (x,y) inside the front
ψ(x,y,0) = ⎨ 0 “ on “
⎩ d(x,y, γ) “ outside “
0
0
0
0 0 0 0
0 0
0
0
0
0
0
0
0
0
000
0
0
000
0
-1-1
-1
-1
-1
-1
-1
-1
-1-1-1
-1
-1
-1-1-1
-1
-1
-1 -1 -1 -1
-2 -2 -2 -2
-2
-2
-2
-2-2-2
-2
-2
-2 -2 -3 -3
-3
-3
-3
-3
1
1
1
1
1
1
1
1
111
1
1
111
1
1
1
1
1
1
1 1 1 1
1 1
1
1
2
2
2
2
2
2
2
22
2
222
2
2
2
2
2
2
2
2
2 2 2 2 2
2
2
2
3
3
3
3
3
3
3
3
3
33
333
3
3
3
3
3
3
3 4
4
4
4
4
4
4
4444
4
4
4
4
4
4
7
5
5
5
5
55
5
5
5
5
6
6
6 6
γ(t)ψ(x,y,t)
ψ(x,y,t)
0
-2
5

Front Propagation
50
€
∂ψ
∂t
+ ˆkI ⋅ FA + FG (κ)( )⋅ ∇ψ = 0
link between spatial and temporal derivatives,
but not the same type of motion as contours!
€
κ = div
∇ψ
∇ψ
%
&
'
(
)
*
constant “force”
(balloon pressure)
ψ(x,y,t+1) - ψ(x,y,t)
extension of the
speed function kI
(image influence)
smoothing “force”
depending on the
local curvature κ
(contour influence)
spatial
derivative
of ψ
product of influences

Front Propagation
51
• Speed function:
– kI is meant to stop the front on the object’s boundaries
– similar to image energy: kI(x,y) = 1 / ( 1 + || ∇I (x,y) || )
– only makes sense for the front (level set 0)
– yet, same equation for all level sets
→ extend kI to all level sets, defining
– possible extension:
^kI
^kI (x,y) = kI(x’,y’)
where (x’,y’) is the point in the front closest to (x,y)
^( such a kI (x,y) depends on the front location )

Reconstruction of Surfaces from Unorganized
Data Points
52
Reconstruction of a rat brain from data of MRI slices

CVPR 2005: Level Set Segmentation (>1800
citation)
53
Ultrasound image segmentation.
Chunming Li et al.
LS Evolution without reinitialization: a
new variational formulation.

Vein Segmentation with Level Set
54

Spinal Cord Quantification - MRI
• Atrophy (Multiple-
Sclerosis) is generally
assessed by measuring
the cross-sectional areas
at specific levels
(typically C2–C5) along
the cervical cord.
• Spinal cord under
analysis can be
characterized by a bright
structure against a dark
background.
• Segmentation is
necessary for accurate
and automatic
quantification
55

Spinal Cord Segmentation in MRI
56
Surface evolution during
the segmentation process
of spinal cord from the
MRI image (the number in
the left corner of each
image represents the
number of elapsed
iterations)
Selective contrast
Credit: Dougherty,MIP.

Cyst Segmentation from Breast US Images
57
Contour extraction of cyst form ultrasound breast image via merging multiple
initial level sets. Images courtesy of Yezzi, Georgia Institute of Technology.

Shape Constraints for LV Segmentation – Cardiac MRI
(Yuanquan Wang, et al, Shape Analysis in Medical Image Analysis )
• Extensive techniques available for cardiac imaging provide
qualitative and quantitative information about the morphology
and function of the heart and great vessels
58

• Many clinically established diagnosis indices such as wall
thickness, myocardial motion, ejection fraction, and
circumferential shortening of myocardial fibers are evaluated
by the segmentation results of MRIs.
59

• Many clinically established diagnosis indices such as wall
thickness, myocardial motion, ejection fraction, and
circumferential shortening of myocardial fibers are evaluated
by the segmentation results of MRIs.
• In clinical practice, the LV segmentation task is often
performed manually by an experienced clinician. Manual
segmentation of the LV, however, is tedious, time consuming,
subjective and irreproducible.
60

Cardiac MRI – Short Axis
61
• A major difficulty in segmentation of the cardiac MR images is the intensity inhomogeneity
due to the radio-frequency coils or acquisition sequences.
• The myocardium and surrounding tissues such as the liver have almost the same intensity
profile, leading to low contrast between them.

Endocardium Segmentation - MRI
62
a Failed active contoursegmentations without the circle-shape constraint. b
Succeeded segmentations with the circle-shape constraint

Epicardium Segmentation - MRI
63
a Epicardium extraction using new external force. b Comparison of segmentation
results with and without shape (similarity) energy
the externalforce
without shape constraint w/ without shape constraint w/

Shape Similarity Constraint
• There would be spurious edges on the myocardium, and the contrast
between myocardium and surrounding structures would be low. Authors
employ the endocardium result as a priori shape and construct a new
shape-similarity based constraint given by
64
R(si )−R measures the deviation of the snake contourfor epicardium from a
circle with radius R at snaxelsi . r’s are for endocardium

LS Evolution with Region Competition (Ho et al., ICPR 2003)
• Good initialization à one major problem in snakes
• Shape constraint based LS is good, but not easy to construct
shape constraint
• Missing/fuzzy boundary -> leakage due to constant
propagation force
– Two adjacent regions compete for the common boundary
65

• Good initialization à one major problem in snakes
• Shape constraint based LS is good, but not easy to construct
shape constraint
• Missing/fuzzy boundary -> leakage due to constant propagation
force
– Two adjacent regions compete for the common boundary
• Tumors vary in shape, texture, size, and intensity
• T1-MRI is used for detailed neuroanatomy, but not good for
precisely distinguishing tumor regions
• T2-MRI is good for tumor and edema identification, but often it is
difficult to obtain high resolution
• Post-contrast T1-weighted MRI is more suitable for tumor
segmentation
66

67
Without (left) and with (right) contrast agent, T1-weihted MRI

• New formula modulates the propagation term using image forces to change the
direction of propagation,so that the snake shrinks when the boundary encloses
parts of the background (B), and grows when the boundary is inside the tumor
region (A):
68

region (A):
69
Region competition

region (A):
70
Region competition
Controls strength of
Smoothing (on active contour)

region (A):
71
Smoothing (on LS contour)
Region competition
Smoothing (on active contour)

72
Tumor probability map (orange: highly likely
tumor regions) is obtained after registering pre-
and post-contrast T1 MR images.

73
Tumor probability map (orange: highly likely
tumor regions) is obtained after registering pre-
and post-contrast T1 MR images.
This map is used to initialize proposed LS
segmentation method.

Level Set Segmentation in Slicer
• Following examples (slides) are from NA-MIC
74

Lec11: Active Contour and Level Set for Medical Image Segmentation

Minimal curvature
Upwind Vector

Slide Credits and References
• Credits to: M.Brady and R.Szelisky, Bagci’s CV Course 2015 Fall.
• TF. Chan and L. Vese, IEEE TIP, 2001.
• TF. Cootes et al. ASM and their training and applications, 1995.
• Osher and Paragios (2003), Paragios, Faugeras, Chan et al.
(2005), Paragios and Sgallari (2009)
• G. Strang, Lecture Notes, MIT.
• Malladi, Sethian, Vemuri. IEEE PAMI 1995.
• R. Szelisky, Lecture Presentations.
• Sethian, JA. Fast Marching. PNAS 1996.
• Osher and Fedkiw. Level set methods and dynamic implicit
surfaces.
• Lim, Bagci, and Li. IEEE TBME 2013 [Willmore Flow and Level Set]
• K.D. Toennies, Guide to Medical Image Analysis,
80

Lec11: Active Contour and Level Set for Medical Image Segmentation

More Related Content

What's hot (20)

Similar to Lec11: Active Contour and Level Set for Medical Image Segmentation (20)

More from Ulaş Bağcı (11)

Recently uploaded (20)

Lec11: Active Contour and Level Set for Medical Image Segmentation