A Survey OF Image Registration

Manjusha P. Deshmukh & Udhav Bhosle
International Journal of Image Processing (IJIP), Volume (5) : Issue (3) , 2011 245
A SURVEY OF IMAGE REGISTRATION
Manjusha Deshmukh manju0810@yahoo.com
Asst. Prof., Saraswati College of Engineering
Kharghar Navi Mumbai, 410210, India
Udhav Bhosle udhavbhosle@gmail.com
Prof., Rajiv Gandhi College of Engineering.
Andheri, Mumbai , India
Abstract
In image processing, for example, when combining the information content of image, we are
interested in the relationship between two or more images. The analysis of this relationship
usually becomes tractable once a correspondence is set up between the images. Image
registration is the task of setting up this correspondence. This paper overviews the theoretical
aspects of an image registration problem. The purpose of this paper is to present a survey of
image registration techniques. Registration is a fundamental task in image processing used to
match two or more pictures taken, for example, at different times, from different sensors, or from
different viewpoints. It geometrically aligns two images the reference and sensed images.
Specific examples of systems where image registration is a significant component include
matching a target with a real-time image of a scene. Various applications of image registration
are target recognition, monitoring global land usage using satellite images, matching stereo
images to recover shape for navigation, and aligning images from different medical modalities for
diagnosis.
Keywords: Image Registration, Image Transformation, Image Mosaicing.
1. INTRODUCTION
Image registration is establishment of correspondence between images of the same scene. Many
image processing applications like remote sensing for change detection, estimation of wind speed
and direction for weather forecasting, fusion of medical images like PET-MRI, CT-PET etc need
image registration. Image registration is a process of aligning two images acquired by
same/different sensors, at different times or from different viewpoint. To register images, we need
to determine geometric transformation that aligns images with respect to the reference image.
The most common transformations are rigid, affine, projective, perspective and global. A variety
of approaches have been published on this topic. Over the years, a large range of techniques has
been developed for various types of problems. All these techniques have been independently
studied for different applications. This paper organizes this research by establishing the
relationship between the variations in the images and the type of registration techniques which
can most appropriately be applied. All registration techniques explained in paper are useful for
understanding the merits and relationships between the wide variety of existing techniques and
for supporting in the selection of the most appropriate technique for a particular problem.
A widespread survey of image registration methods was published in 1992 by Brown [1]. A
comprehensive survey of image registration methods is presented by Barbara Zitova and Jan
Flusser [2]. They have classified the image registration techniques as area based methods and
feature based methods. J. B. Antoine Maintz presented a survey of medical image registration in
1998.[3].Leszek Chmielewski presented various methods of image registration in
2001.[4].Mohammad Essadiki presented a technique for combining panchromatic and
multispectral spot images [5].Subunku in his thesis presented various entropy based image
registration techniques[6].S. K. Bose presented various tools for medical image registration [7].J.

Flusser used moment based approach to correct affine distortion, he has also done degraded
image analysis to locate invariants in images[8,9]. Sangit Mitra and B.S. Manjunath explained
various contour based approaches for multispectral image registration in their different
papers[11]. Cahill, N D Williams, C.M. Shoupu propose an approach to incorporate spatial
information into the estimate of entropy to improve multimodal image registration [15]. J.P.W.
Plum presented another servey on medical image registration [16 ]. Frederik Maes and Andre
Collignon apply mutual information to measure the statistical dependence or information
redundancy between the image intensities of corresponding voxels in both images [17]..Xiaoxiang
Wang and Jie Tian in their paper proposed a mutual information based registration method using
gradient information rather than pixel intensity information [18]. Frederik Maes ,Andre present
novel histogram based method for estimating and maximizing mutual information between two
multimodal and possibly multiband signals [19]. J. P. Queiroz developed method for automatic
registration of satellite images acquired on different dates, for both geometric and radiometric
correction with respect to reference image [20]. Shannon’s paper, titled “A mathematical theory
of communication “is widely accepted as the origin of information theory. In this paper, Shannon
used probability theory to modal information sources, i.e. data produced by a source is treated as
a random variable [21]. Haim Schweitzer in his paper proposed that large collection of images
can be indexed by projections on a few “ eigenfeatures “ , the dominant eigenvectors of the
images covariance matrix [22]. Ma Debao and Liwagao introduced the new matrix characteristic
methods like eigenvalues and eigenvectors and achievable accuracy is derived theoretically and
verified by tests using simulated interferometric data.[23]. Haim Schweitzer demonstrated that
eigenspace based algorithm registers multiple images and produces improved
eigenfeatures[24].Wen Cao and Bicheng proposed PCAT (principal component analysis
transform) and WPT (wavelet packet transform ) for remotely sensed image fusion[25]. Yu-
Te Wu, Takeo Kanade, Ching-Chung Li and Jeffrey Cohn proposed that their wavelet-based
algorithm produced better motion estimates with error distributions having a smaller mean and
smaller standard deviation [26]. Tarek A El-hazawi explained wavelet based image registration
[27]. Hala S. Own and Aboul Ella Hassanien in their paper presents an efficient image registration
technique using the Q-shift complex wavelet transform (Q-shift CWT). The experimental results
proved that the proposed algorithm improves the computational efficiency and yields robust and
consistent image registration compared with the classical wavelet transform [28].Azhar Quddus
and Otman Basir proposed a novel, fully automatic, multistage wavelet-based image registration
technique for image retrieval applications. They used multiscale wavelet representation with
mutual information (MI) to facilitate matching of important anatomical structures at multiple
resolutions. The proposed approach has several novel aspects including the use of MI in
multistage wavelet domain [29].Stone, Harold S.,Le Moigne, Jacqueline; McGuire and Morgan
proposed fast image registration by progressively registering wavelet representations at different
resolutions[30].Ghazaw presented wavelet based image registration on parallel computing[31].
Jiangsheng explained image matching techniques by using Radon transform [32].
2. TRANSFORMATIONS
Image registration consists of establishing correspondence means matching of identical
shapes in related image pair. This requires geometric transformation of one image into
another. Change in viewpoint or relative motion between the camera and object planes
introduces distortion in the features of an image e.g. a circle may appear ellipse when
observed from non-fronto-parallel vantage point. However, certain features of object shape
remain intact even after such transformations. These features are called invariants. The
fundamental characteristic of any image registration technique is the type of spatial
transformations or mapping used to properly overlay two images. The most common
transformations are rigid, affine, projective, perspective, and global [1, 2].

2.1 Euclidean or Rigid Body Transformation
where
is a rotation matrix and t=[tx ty]
T
is translation vector. p’ and p are the transformed and
original 2-D points, respectively, represented in non homogeneous coordinates [x' ,y']
T
and
[x ,y]T
respectively. Euclidean invariants are lengths and angles.
2.2 Similarity Transform
Here s is a scaling factor. Similarity invariants are angles, ratios of lengths, and ratios of areas.
2.3 Affine Transformation
The most commonly used registration transformation is the affine transformation which is
sufficient to match two images of a scene taken from the same viewing angle but from different
position. It is composed of scaling, translation, and rotation. It is global transformation which is
rigid. Affine transformations are more general than rigid.
The general 2D affine transformation
Angles and lengths are not preserved. Parallel lines remain parallel.
2.4 Projective Transform
This is the most general geometric transformation. Here, two 2-D points p' and p (represented in
homogeneous coordinates), are related by a 3 × 3 non-singular transformation matrix
(homography).
Projective invariants include the cross ratio of four collinear points, or four concurrent lines [1, 2,
3].
3. REGISTRATION ALGORITHMS
Image registration algorithms can be classified in various ways, like based on modality,
intensity or methods used for registration. Barbara Zitova and Jan Flusser [2] classified the
image registration techniques as area based methods and feature based methods. Area based
methods are preferably applied when in images prominent details are absent and distinctive
information is provided by gray levels / colors rather by local shapes and structure. Feature based
matching methods are applied when local structural information carried by image intensities are
more. These methods make use of image features derived by feature extraction algorithm. Point
of sharp variations such as edges, corners, contours, surfaces, point of intersection etc. what
carries valuable information about images are used for matching.

3.1 Pixel Based Method
Cross-correlation is the basic statistical approach of registration. It is often used for template
matching or pattern recognition in which the location and orientation of a template or pattern is
found in picture. Cross correlation is a similarity measure or match metric. For template T and
image I, where T is small compared to I, the two dimensional normalized cross-correlation
function measures the similarity for each translation[1,2,3,4].
If template matches the image, then cross correlation will have it’s peak at C( i ,j ) . Cross
correlation must be normalized since local image intensity would otherwise influence the
measure.
Implementation on Dataset-1
FIGURE 1: (a) Original image SCOE-1. (b) translated image SCOE-2. (c) Correlation plot for
SCOE 1 & SCOE 2.
Observations
Two main drawbacks of the correlation-like methods are the flatness of the similarity measure
maxima (due to the self-similarity of the images) and high computational complexity. The
maximum can be sharpened by preprocessing or by using the edge or vector correlation. It is
used in next section.
3.2 Feature Based Methods or Point Mapping Method
Feature based matching techniques do not use the grey values to describe matching entities. It
makes use of image features derived by feature extraction algorithm. The purpose of feature
extraction is to abstract substantial information from original data input and filter out the
redundant information. Features are selected which are likely to be uniquely found in both images
and more tolerant of local distortions. Computing of proper transformation depends on these
features. Therefore sufficient number of features must be detected to perform calculation. After
detecting features in each image, they must be matched. This technique is primary approach to
register two images whose type of misalignment is unknown. This occurs if class of
transformations cannot be easily categorized as translations or rigid-body movements. In this we
can use landmarks and match them using general transformation. The method of point mapping
consist of three stages-
• Computing features in the images
• Control points in reference image are corresponded with feature points in data image.
• Spatial mapping.
Control points for point matching play an important role in this method. Control points may be
corners, line of intersections, points of locally maximum curvature on contour lines, centers of
windows having locally maximum curvature and centers of gravity of closed-boundary regions [1,
2, 14]. Under this method we have computed edges of images in color vector space by using
color gradients For a scalar function f(x, y), the gradient is a vector pointing in the direction of
maximum rate of change of f at coordinates (x ,y ) .

Implementation on Dataset 1
FIGURE 2: (a) mar1 image adopted from MATLAB toolbox. (256 X 256). (b) mar2.
(c) Features extracted from MAR-1.(d) Features extracted from MAR-2.(e) Correlation
plot between (c) & (d). (f) registered mar1 and mar2
In this experiment image correction of rotated and stretched image of clown is done by point
mapping method. In this method features are extracted using color gradients. Invariants are
identified and projective transformation is applied on image to correct it.
FIGURE 3: (a) Clown 1 original image(adopted from MATLAB toolbox).(b) : Clown 2
original image stretched and rotated.(c) Image corrected by projective transformation.
Observation
Feature based registration methods overcomes the limitation of correlation method and are often
in use, particularly because of their easy hardware implementation, which makes them useful for
real-time applications. Disadvantage of the these methods refers to the ‘remarkableness’ of the
window content. There is high probability that a window containing a smooth area without any
prominent details will be matched incorrectly with other smooth areas in the reference image due
to its non-saliency. The features for registration should be preferably detected in distinctive parts

of the image. Windows, whose selection is often not based on their content evaluation, may not
have this property.
3.3 Registration Based on High Level Features (Contour Based Image Registration)
This method makes use of high statistical features for matching image feature points. For
extracting regions of interest from the image color image segmentation is used. Given a set of
color of interest, obtain mean of color ‘m’. For segmentation classify each RGB pixel in image as
having a color in the specified range or not. Euclidean distance is used to measure similarity. ‘z’ is
an arbitrary point in RGB space and T is threshold. Euclidean distance between ‘z’ and ‘m’ is
given by
Here RGB denotes RGB components. The locus of points such that D (z, m) <=T is a solid
sphere of radius T and point contained within, or on the surface of the sphere satisfy specified
color criterion. Coding these two sets of points in the image with black and white produces a
binary, segmented image. After segmentation remove noise by ‘Gaussian’ filter. Threshold
blurred image and then obtain contour of image. For registration of jup1 and jup2 contours of red
colors are extracted by above said method .Centroid of contours i.e. (x, y) coordinates of centre
of gravity of jup2 are corresponded with centroid of jup1 contour. Similar procedure is repeated to
extract contours of berry images .Angle between x-axis and major axis of contour is calculated to
find angle of rotation .
FIGURE 4 : (a) original image jup-1 (256 X256).(b) translated image jup-2(256
X256). (c) Counter extracted from jup-1(d) Counter extracted from jup-2(e)
Mosaic of jup-1 and jup-2

FIGURE 5: (a) Original image Berry1(256 X256). (b) Berry2 rotated by 90 deg. (256
X256).
Actual angle of rotation = 90 deg, Calculated angle of rotation = 88.85 deg.
Observations:
Feature based methods do not use the gray values for matching and hence overcomes the
limitations of spatial methods. Feature based method filter out the redundant information.
Accuracy of this method is more but the limitation is , it is manual and slow.
3.4 Multimodal Image Registration Using Mutual Information
Multispectral image registration is also called as multimodal analysis. Images of the same scene
are acquired by different sensors. The aim is to integrate the information obtained from different
source streams to gain more complex and detailed scene representation. Different types of
application are, in remote sensing fusion of information from sensors with different characteristics
like panchromatic images, offering better spatial resolution, color/multispectral images with better
spectral resolution, or radar images independent of cloud cover and solar illumination. Medical
imaging applications are, combination of sensors recording the anatomical body structure like
MRI, ultrasound or CT with sensors monitoring functional and metabolic body activities like PET,
SPECT. Result can be applied, in radiotherapy and nuclear medicine.Registration of multispectrul
/ multisensor images is a challenging area. In general such images have different gray level
characteristics and simple techniques such as those based on area correlation cannot be applied
directly. This section is an attempt to solve this difficult problem by employing a basic concept
from information theory.
Entropy of image
For a high dimensional discrete random variable ),,,( 21 dXXXX −−−= that has a probability
mass function of ),,,,( 21 dxxxP L the entropy is
),,,(
1
log),,,()(
21,,,
21
2 d
d
xxx xxxP
xxxPXH
d
L
L
L
∑=
From the properties of entropy, entropy doesn’t depend on value of random variable, but only
depends on distribution. This is explained with following example.
E.g. consider image shown in following figure 6.
FIGURE 6: (a)The image of size 500 X 500 with 256 gray levels. (b)Histogram of
image.

Entropy (gray image) = 7.7540
By applying bijective mapping to the intensity values, we get synthetic image as shown below in
figure 7.
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
0
5 0 0
1 0 0 0
1 5 0 0
2 0 0 0
2 5 0 0
no.ofpixels
n o o f g r a y l e v e l s
FIGURE 7: (a) Synthetic image of size 500X500 (left) with 256 gray
levels.(b)Histogram of image
Synthetic Entropy (synthetic image) =7.7549
From Figure 6 and 7, it is easy to see that entropy of image does not change even if histogram
has changed. Even after randomly shuffling pixels of image, entropy of image remains same.
From above discussion it is clear that natural images contain less uncertainty. In this type of
images pixel intensity values depend on neighboring pixels. In other words, in natural image the
value of pixel is likely to be close to some of its neighbors. Hence this dependency reduces the
total entropy [6].
Entropy as Alignment Measure
The objective of using entropy as alignment measure is simple. Corresponding features extracted
from the images should become statistically more dependent with better alignment. This is
explained with following example.
FIGURE 8: ( a) Subject. (b ) subject rotated by 5 0
.(c) Subject rotated by 200
.(d) Subject with rotated neck. (e) Joint histogram a Vs a. (f) Joint histogram a
Vs b. (g) Joint histogram a Vs c. (h) Joint histogram a Vs d .
In Figure 8, scatter plots (joint histogram) display pixel intensity value pairs from all images.
Notice that, since in Figure e. images are aligned, pixel samples cluster around x-y line. At bad
alignment, samples are scattered, i.e. joint histogram is more dispersed. This indicates that if

alignment is good joint distribution tends to be sharper with peak at good alignment. This
illustrates misregistration measured by dispersion of 2-D histogram of image intensities[6].
Mutual Information to Describe Dispersive Behavior of 2-D Histogram
Mutual information measures statistical dependence between two random variables. Mutual
Information criteria presented here states that, mutual Information of image intensity values of
corresponding voxel pairs is maximum if images are geometrically aligned. Let A and B represent
random variables and PA (a) and PB (b) represents its marginal probability distributions. Let
),( baP AB
represents joint probability distribution
then BA & are independent if )()(),( bPaPbaP BAAB ∗= Mutual Information I (A, B) is
given by
)}]()(/{),(log[),(),(
,
bPaPbaPbaPBAI BAAB
ba
AB ⋅= ∑
Mutual Information is related to entropy by following equations.
),()()(),( BAHBHAHBAI −+=
Where ),( BAH is joint entropy and )/(&)/( ABHBAH are conditional entropy
)(log)()( aPaPAH A
a
A ⋅−= ∑
),(log),()/(
,
baPbaPBAH AB
ba
AB ⋅−= ∑
)/(log),()/(
,
/ baPbaPBAH
ba
BAAB∑ ⋅−=
Consider that x & y is image intensity values of a pair of corresponding voxel in two images
which are to be registered. Intensities x and y are related through the geometric transformation
Tα defined by registration parameter α. The MI registration criteria states that the images are
geometrically aligned by the transformation Tα for which I(A,B) is maximum. This is explained
in following example. Figure 9 (d) and (e), shows 2-D histogram of image intensity values in non-
registered and registered position. We observe that at good alignment i.e. at x=160, joint
distribution is sharper and shows peak. Mutual Information between SCOE 1 & SCOE 2 is 3.4026
and matching point is at 160. Similarly Mutual Information between SCOE 1 & SCOE 3 is
2.97and matching point is at 161.
Implemetation on Dataset-1

FIGURE 9 : (a-c)Sample images SCOE-1, SCOE-2 and SCOE-3.(d) I (A, B) as a
function of mis registration for SCOE 1 and SCOE 2.(e) I (A, B) as a function of
misregistration for SCOE 1 and SCOE 3.(f) mosaic of SCOE1 and SCOE2.(g)
mosaic of SCOE1 and SCOE3 in multiple spectrums
FIGURE 10: (a) Image in thermal infrared bands (450 X 450 ). (b)Template in near
infrared (150 X 150).(c)Matched template (150 X 150).
For images a, b and c, Template matched at x = 124 and Y = 300
Mutual information = 0.8522
Implementation on Dataset 3 to Correct Rotation and Scale

FIGURE 11: (a) MRI image ( 230 X 230) rotated 4
0
.(b) CT image (512 X 512).(c-d)Registered CT image(233
X 233)
Implementation on Dataset 4 to Correct Rotation and Scale
FIGURE 12: (a) Prutha image ( 230 X 230) .(b) Prutha image (512 X 512) rotated by
60
.(d)Registered Prutha image(231 X 231)

Observation
It is observed that proposed method yields a more accurate registration than any other
registration method. But this method has its own limitations. When images are of low resolution,
when images contain little information, or when the region of overlap is small then mutual
information result in mis-registration .It has one more limitation of speed , when time is an
important constraint one cannot use this method . Although it has some limitations entropy and
mutual information are best approaches for multimodal image registration.
3.5 Image Registration in Frequency Domain
Correlation theorem has one useful property. Correlation theorem states that, the Fourier
transform of the correlation of two images is the product of Fourier transform of one image and
complex conjugate of Fourier transform of other. The Fourier transform of an image f(x,y) is a
complex function, each function value has real part R(ωx,ωy) and an imaginary part I(ωx,ωy)at
each frequency (ωx,ωy) of frequency spectrum.
Where
is magnitude , and is phase angle
Cross power spectrum of two images is defined as
Shift theorem guarantees that phase of cross power spectrum is equivalent to the phase
difference between the images. If we represent the phase of cross power spectrum in it’s spatial
form, i.e. by taking the inverse Fourier transform of the representation in the frequency domain,
then we will have a function which is an impulse, that is approximately zero everywhere except at
displacement which is needed to optimally register two images. Above method is used to register
images having only translation.
Implementation on Dataset-1 to Correct Translation
FIGURE 13: (a) Edges extracted from SCOE-1. (b) Edges extracted from SCOE-2.
(c) cross power spectrum for SCOE 1 & SCOE 2 in frequency domain.
Observation
It is observed that in frequency based method accuracy is more than correlation method but less
as compared to other methods. But if we extract image features and then apply Fourier method

accuracy increases. In frequency domain it should be noted that some form of interpolation must
be used[1,2].
3.6 Wavelet Transform Based Image Registration
Registration of image using wavelets is being discussed in this section. Image Registration using
wavelets has already been studied by many researchers. In this step feature that is extracted
using wavelet or other transforms used as input and correspondence between these features will
be recognized. To find out correspondence between two images so that transform model can be
calculated using these correspondence points is very important part of overall problem. Cross
correlation or Mutual Information could be used as a measure of similarity
Wavelet transform decomposes an image into various sub images based on local frequency
content. Using discrete wavelet transform (DWT), a function f(t) can be represented by
)()(
,
, tatf jk
kj
kj ψ∑=
Where aj,k are wavelet coefficients , Ψj,k(t) are basis function , j is scale , k is translation of mother
wavelet Ψ(t). Two dimensional DWT can be obtaine by applying DWT across rows and columns
of an image.The two dimensional DWT of image f(x,y)is
),(],[),(),(),(
,, ,
,,,,
, 0
0
yxlkDyxlkCyxf
DVHS JJ lk
S
lkj
S
jlkj
kj
J ∑ ∑∑∑ =
∞
=
+= ψφ
Where C J0 is approximation coefficient, φj,k,l (x,y) is scaling function, Dj
S
is set of detail coefficients
and Ψ
S
j ,k, l is set of wavelet function. The DWT coefficients are computed by using a series of
low pass filter h[k], high pass filters g[k] and down samplers across both rows and columns. The
results are the wavelet coefficient the next scale. The filter bank approach to calculate two
dimensional dyadic DWT is shown in Figure 14. The wavelet coefficients are of smaller spatial
resolution as they go from finer scale to coarser scale. The coefficients are called the
approximation (A), horizontal detail (H), vertical detail (V) and diagonal detail (D) coefficient.
FIGURE 14: Two-dimensional orthogonal wavelet decomposition
In wavelet transformation due to sampling, the image size is halved in both spatial directions at
each level of decomposition process thus leading to a multi-resolution signal representation. The
decomposition and reconstruction of wavelet pyramid of source images are based on Mallet’s
theories[25,26,27,28,29,30,31].

FIGURE 15: (a)Original image SCOE-1(256X256).(b)Translated image SCOE-
2(256X256).(c)Wavelet decomposition at level 2 by Harr wavelet for SCOE-1.(d)
Wavelet decomposition at level 2 by Harr wavelet for SCOE-2.
Mutual Information as a Measure of Similarity
Origin of MI is from information theory, MI is an entropy-based concept and denotes the amount
of information that one variable can offer to the other. Mutual information measures statistical
dependence between two random variables. Mutual Information criteria presented here states
that, mutual Information of image intensity values of corresponding voxel pairs is maximum if
images are geometrically aligned. Let A and B represent random variables and PA (a) and PB (b)
represents its marginal probability distributions. Let ),( baP AB
represents joint probability
distribution then BA & are independent if )()(),( bPaPbaP BAAB ∗=
Mutual Information I (A, B) is given by
)}]().(/{),(log[),(),(
,
bPaPbaPbaPBAI BAAB
ba
AB∑=
Mutual Information is related to entropy by following equations.
),()()(),( BAHBHAHBAI −+=
FIGURE16: I (A, B) as a function of mis registration for SCOE 1 and SCOE 2
Observations
It is observed that combinational approach of wavelet and mutual information gives better results
as compared to wavelet - correlation combination and Fourier based image registration. Even

wavelet mutual information combination can be used in case of multimodal image registration.If
we involve mutual information as similarity criteria, speed decreases and hence we can not use
this combination in applications where speed is required.
3.7 Hotelling Transform Based Image Registration
In this section automatic method of template matching using Hotelling transform as well as
Hotelling transform for image alignment is presented. The proposed method is validated on pair
of remotely sensed scenes and medical images.
Principal Component Analysis
In this section we present Hotelling Transform (Principal Component Analysis) for template
matching and image alignment to register images. The proposed method is validated on pair of
remotely sensed scenes and medical images. Proposed method for template matching is
compared with other algorithms.
Principal component analysis is one of the most frequently used dimension reduction method.
Principal component analysis also called as Hotelling Transform. Hotelling Transform is based on
stastical properties of vector representations. The material discussed here can be used as the
basis for describing sets of images, that are registered spatially, but their corresponding pixel
values are different. If we have ‘n’ component images having different pixel values, These images
can be treated as a unit by expressing each group of ‘n’ corresponding pixels as a vector. Let
, ----, are values of first pixel in each of the n’ images then ‘n’ elements can be expressed
as follow
Where, x is column vector. This one vector represents one commen pixel in all two image. If
images are of size P X Q, there will be total of N=P*Q such n- dimensional vectors.
FIGURE 17: Forming a vector from corresponding pixels in a stack of images of the same size
We can assume these vectors as random quantities,
Mean vector of population is
The covariance matrix of vector population is
Element of is the variance of xi , the i
th
component of x vectors and element is
covariance between components xi and xj .The matrix is symmetric and real. For a sample of
N vectors from a random population, the mean vector and covariance matrix can be given by
expression

Thus covariance matrix can be estimated as
Since Cx is real and symmetric, it is possible to find a set of N ortho-normal eigenvectors. Let ei
and λi be eigenvectors and corresponding eigenvalues of Cx where i = 1,2,---,N . ‘A’ is a matrix
whose rows are eigenvectors of covariance matrix Cx . Then A is ordered so that the first row of
A is eigenvectors corresponding to the largest eigenvalue , and last row corresponds its smallest
eigen value. If we use A as transformation matrix to map the x’s into y. Then y is given by
Above expression of ‘y ‘ is called Hotelling Transform or Principal Component Transform.
Alternatively, the Hotelling Transform can be viewed as the discrete version of the Karbunen-
Loeve transform (KLT).This transform has some useful properties . Mean of ‘y’ vector is zero i.e.
Covariance matrix of y’s is estimated as
It is observed that is a diagonal matrix.Elements of along main diagonal are eigenvalues
of .
are eigenvalues of . The important property of Hotelling Transform is we can
reconstruct vector x from vector y.
The error between vector x and reconstructed vector can be minimized by using the
eigenvectors corresponding to the largest eigenvalues. The fact that zero values are everywhere
except along the main diagonal in indicates that the components of the transformed vector y
are uncorrelated i.e. the correlation previously existing between the different components of
random vector has been removed in transformed domain. Therefore, if the input is split into
blocks and Hotelling Transform is applied block wise, the coding may be more efficient since the
data in the transformed block are uncorrelated.
Geometrical Interpretation of Hotelling Transform (PCA)
Here we are giving a geometrical interpretation of transform coding. For this, we use 2-D vectors
instead on N-D vectors. Hotelling Transform can also be used to align region or boundaries with
the eigenvectors of the object. We have to form two-dimensional vectors from the coordinates of
the boundary or region. Figure 18(a) shows that vectors are formed from the coordinates of the
pixel in the object or we can use coordinates of points on the boundary. The resulting vectors are
treated as 2-D population of random vectors. Let each pixel in the object is a 2-D vector x-(a,b)T
,
where a and b are coordinate values of that pixel with respect to x1 and x2 axis.

FIGURE 18: (a) Arbitrary object in x1-x2 coordinate system. (b) origine of new
coordinate system coordinate system .(c) After Hotelling transform, the object is
aligned with its principle axes.
These vectors are used to calculate mean vector mx and covariance matrix Cx. Then the effect of
using equation of Hotelling Transform is to establish a new coordinate system. The origin of this
new coordinate system is at the centroid of the population and its axes are in the direction of
eigenvectors of Cx. It is shown in figure 18 (b). It is observed that transformation in equation
y=A(x-mx) is a rotation and translation that aligns the object with eigenvectors. Two eigenvectors
are perpendicular.The ‘y’ axis is also called as eigen axis. The ability of Hotelling Transform to
align the object with its principal axis provides a reliable means for removing the effects of
rotation. This is a rugged alignment procedure which uses all coordinates of object (region or
boundary) to compute the transformation matrix and aligns the data in the direction of its principal
spread [15].
Implementation on Dataset-1 for Image Alignment
In this experiment, Hotelling Transform is applied to binary images .Fig. 19 (a) shows gradient
image of MRI of head and fig 19 (b) shows same image but head is at translated and rotated
position. .
FIGURE 19: (a) MRI image of brain (400X400). (b) Translated and rotated version of
same image (400X400). (c) Hotelling transform of the image a . (d). ( Hotelling
Transform of the images b.
Figure 19(c,d) shows Hotelling Transform of MRI image and its rotated version. It can be seen
that Hotelling Transform automatically positioned the two images in such a way that
corresponding points are practically aligned. This removes rotational effects. Translational effects
are also removed since the object is centered on its mean. After this alignment images can be
fused to get complete information for diagnosis. In this section we have given background theory

of principal component analysis and discussed implementation details. It is observed that if we
use eigenvalues and eigenvectors for registration of images accuracy is more precise than that of
other methods. We can use his method for template matching. It is observed that Hotelling
Transform can be used to align region or boundaries with the eigenvectors of object but this
method can be applied only to regions and boundaries. In terms of future work, effort could be
directed at the problems of an unknown image scaling and unknown global rotation between the
target and reference images. Both are outstanding issues requiring a solution and should prove
useful for multimodal images. There are possible solutions to the problem that could be
investigated in future work.
Implentation on Dataset -2 for Template Matching
Template matching is the process of finding the location of sub image, called a template, inside
an image. Template matching involves comparing a given template with windows of the same
size in an image and identifying the window that is most similar to the template. The accuracy of a
template matching process depends on the accuracy of metric used to determine the similarity
between a template and a window. There are many similarity measures known to produce best
results .Hotelling Transform can be used for template matching. It is observed that when template
match with the window of the same size in an image then eigenvector corresponding to that
location shows largest eigenvalue and eigenvalue of second eigenvector shows minimum value.
FIGURE 20: (a) Sample image (50 X 50). (b) Template (25 X 25). (c) Matched
template (25 X 25).
4. COMPARATIVE STUDY OF IMAGE REGISTRATION METHODS.
All methods explained in the paper are implementated on the three datasets . This made
comparison of all the methods easy.
Implentation on Dataset-1 on Dataset 1
Main application of image registration is in remote sensing for image mosaicking of surveyed area
for monitoring of global land usage; landscape planning etc. A camera typically has a limited field
of view. A lens with a wide field of view incurs substantial distortion. In addition, capturing the
entire scene with the limited camera resolution compromises the image quality. Image mosaicing
algorithms register or stitch a sequence of images into a composite image.

H1 H2 H3 H4 H5
H6 H7 H8 H9 H10
H11 H12 H13 H14 H15
FIGURE 21: Sample images and mosaic of VR images from H1 to H 16
Observations
Location of maximum match point is shown in table
TABLE I: Location of Maximum Match using Different Methods
S.N.
Image
combination
Correlation
Method
Fourier
Transform
Method
Mutual
Information
Method
Principal
Component
Analysis
Wavelet
Transform
Method
Feature
Based
Method
1 H1-H2 63 62 65 62 59 61
2 H2-H3 110 111 113 112 113 114
3 H3-H4 62 63 57 62 63 63
4 H4-H5 79 77 78 80 80 82
5 H5-H6 147 147 150 147 148 150
6 H6-H7 93 92 89 93 93 95
7 H7-H8 72 74 73 74 74 75
8 H8-H9 73 74 73 74 74 74
9 H9-H10 88 88 90 89 88 90
10 H10-H11 83 84 84 83 83 82
11 H11-H12 79 79 82 80 80 80
12 H12-H13 81 81 78 81 81 82
13 H13-H14 85 84 89 89 83 89
14 H14-H15 128 126 129 128 125 129

TABLE II: Location of Maximum Match using Different Methods
S.N. Image
combination
Correlation
Method
Fourier
Transform
Method
Mutual
Information
Method
Principal
Component
Analysis
Wavelet
Transform
Method
Feature
Based
Method
1 H1-H3 169 167 171 169 145 168
2 H2-H4 171 172 167 170 165 170
3 H3-H5 136 133 140 136 133 135
4 H4-H6 223 233 235 223 224 224
5 H5-H7 231 237 240 232 233 232
6 H6-H8 161 158 150 160 162 162
7 H7-H9 141 140 130 140 141 140
8 H8-H10 158 156 160 159 155 157
9 H9-H11 143 143 130 142 142 142
10 H10-H12 163 162 150 164 163 164
11 H11-H13 160 161 132 159 158 159
12 H12-H14 165 165 165 165 163 164
13 H13-H15 210 201 210 210 210 209
Implentation on Dataset-2
track1 track -2 track-3 track-4 track-5
track-6 track-7 track-8 track-9 track-10
track-11 track-12 track-13 track-14
FIGURE 22: Sample images in infrared band and mosaic of IR images from track-1 to track-14
Observation
Location of maximum match point is shown in table.

TABLE III : Location of Maximum Match using Different Methods
S.N. Image
combination
Correlation
Method
Fourier
Transform
Method
Mutual
Information
Method
Principal
Component
Analysis
Wavelet
Transform
Method
Feature
Based
Method
1 T1-T2 44 45 46 45 46 44
2 T2-T3 51 50 48 49 50 50
3 T3-T4 19 21 22 20 20 20
4 T4-T5 22 21 21 20 19 20
5 T5-T6 20 20 21 20 22 20
6 T6-T7 57 59 50 57 58 58
7 T7-T8 30 32 32 30 34 31
8 T8-T9 22 24 20 24 22 23
9 T9-T10 20 20 11 19 22 18
10 T10-T11 60 63 70 61 60 62
11 T11-T12 28 28 30 27 28 27
12 T12-T13 50 50 51 51 50 49
13 T13-T14 27 27 28 27 28 27
TABLE IV: Location of Maximum Match using Different Methods
S.N. Image
combination
Correlation
Method
Fourier
Transform
Method
Mutual
Information
Method
Principal
Component
Analysis
Wavelet
Transform
Method
Feature
Based
Method
1 T1-T3 94 95 99 95 96 94
2 T2-T4 71 71 71 71 70 70
3 T3-T5 41 41 41 41 39 41
4 T4-T6 41 40 41 41 40 40
5 T5-T7 77 78 70 77 78 77
6 T6-T8 90 90 98 89 92 90
7 T7-T9 56 56 58 56 55 55
8 T8-T10 40 43 40 43 42 41
9 T9-T11 82 82 75 81 82 81
10 T10-T12 90 90 83 89 88 90
11 T11-T13 78 81 90 78 77 77
12 T12-T14 77 77 77 77 77 77
Implentation on Dataset-3
An arbitrary set of images of Saraswati College of Engg., Kharghar, Navi Mumbai were collected,
using a panoramic set up. To capture the images the camera was mounted on a leveled tripod.
While capturing the images camera was operated in manual mode where all camera parameters
like aperture, shutter speed, focal length were constant. Since images are taken from different
planes, the topology of mosaic was unknown. The motion between the images was unknown and
was not assumed to be constant.
SCOE-1 SCOE-2 SCOE-3
SCOE-4 SCOE-5 SCOE-6

FIGURE 23: Sample images of SCOE and mosaic of images from SCOE1 to SCOE6
Observation
TABLE V : Location of maximum match
5. CONCLUSION
Various methods are reported in literature to register images which are in same band. In pixel
based method cross correlation is used as similarity measure. It is observed that in natural
images like buildings or scenery, correlation method shows match at multiple points. The feature
based method makes use of features like point of intersection, edges, corners, centers of
contours etc. for matching sample template with reference image. But this method is manual and
hence time consuming. The method combining image features with correlation method have
many advantageous properties of both feature-based and intensity based. It overcomes the
limitation of intensity based method. Contour based methods do not use the gray values for
matching and hence overcomes the limitations of spatial methods. Feature based method filter
out the redundant information. Accuracy of this method is more but the limitation is, it is manual
and slow. In frequency based method accuracy is more than correlation method but less as
compared to other methods. But if we extract image features and then apply Fourier method
accuracy increases. In frequency domain it should be noted that some form of interpolation must
be used. These are some of the conclusions about methods used for registration of images which
are in same spectral band. Image registration is difficult when images are obtained through
different sensor types. Mutual Information, Hotelling Transform, Fuzzy logic are some of the
approaches that can be used for multimodal image registration. For three different set of images
SCOE1 to SCOE6, VR images form H1 to H15 and Infrared images track1 to track14 we carried
out some of the popular algorithms and compared on the basis of point of match. Table I to Table
V show the comparison. Combinational approach of wavelet and mutual information gives better
s.
n.
Combin
ation of
images
Pixel based
method
Fourier based
method
Feature based
method
Mutual
Information
based method
Principal
component
analysis
x-
transla
tion
y-
transla
tion
x-
translati
on
y-
translati
on
x-
transl
ation
y-
transl
ation
x-
transla
tion
y-
transla
tion
x-
translat
ion
y-
translati
on
1
SCOE1
-
SCOE2
157 -2 157 -5 156 -2 160 -2 159 -2
2
SCOE1
-
SCOE2
163
35
164 35 163 35 163 35 164 35
3
SCOE1
-
SCOE2
220 15 223 17 219 16 221 16 222 17
4
SCOE1
-
SCOE2
163 10 161 5 162 10 161 09 162 10
5
SCOE1
-
SCOE2
120 08 118 04 118 08 118 07 119 08

results as compared to wavelet - correlation combination and Fourier based image registration.
Even wavelet mutual information combination can be used in case of multimodal image
registration. If we involve mutual information as similarity criteria, speed decreases and hence we
can not use this combination in applications where speed is required. If we use eigen values and
eigenvectors for registration of images accuracy is more precise than that of other methods. We
can use this method for template matching. It is observed that Hotelling Transform can be used to
align region or boundaries with the eigenvectors of object but this method can be applied only to
regions and boundaries. It is observed that method based on information theory yields a more
accurate registration than any other registration method. But this method has its own limitations.
When images are of low resolution, when images contain little information, or when the region of
overlap is small then mutual information result in mis-registration .It has one more limitation of
speed , when time is an important constraint one cannot use this method . Although it has some
limitations entropy and mutual information are best approaches for multimodal image registration.
6. FUTURE SCOPE
A major challenge in the current literature is to perform population registration on large collections
of data sets. Currently available tools pre selects a reference dataset that is template and
registers in a pair wise fashion. The computational complexity and accuracy of this approach can
be eliminated by performing a simultaneous registration on the whole population. The methods,
we have explored in this paper have desirable computational speed for achieving population
registration. An immediate next step would be to investigate this open problem that may lead to a
significant contribution.
7. REFERENCES
[1] Brown Gottesfeld L., “Survey of Image Registration Techniques”, ACM Computing Surveys,
24, 4, 1992, 325-376.
[2] Barbara Zitova, Jan Flusser, “Image registration methods:A survey”, Image and Vision
Computing 21(2003)977-1000.
[3] J.B. Antoine Maintz and Max A. Vierger , “A Survey of Medical Image Registration”
,Medical Image Analysis,(1/98) volume 2. number1, pp 1-37
[4] Leszek Chmielewski, Dorota Kozinska , “Image Registration”, conf. Computer Pattern
Recognition SystemsKOSYR 2003, May 26-29, 2001, Milkow,Poland, pp.163-168.
[5] Mohammed ESSADIKI, “New technique for combining panchromatic and multispectrul spot
images for multipurpose image-maps”.
[6] Mert Rory Sabuncu, “Entropy-Based Image Registration”. Phd Thesis.
[7] S. K. Bose, “An Overview of Methods and Tools in Medical Image Registration.”, ‘National
workshop on Digital signl processing’ 3rd
to 5th
July, 2006.
[8] J. Flusser, T. Suk, “Degraded Image Analysis: an Invariant Approach”, IEEE transaction on
Pattern Analysis and Machine Intelligence 20(1998)590-603.
[9] J. Flusser, T. Suk,”A Moment Based Approach to Registration of Images with Affine
Geometric Distortion”, IEEE transactions on Geoscience and remote sensing 32(1994)382-
387.
[10] C. Zhang, “A Computational Vision Approach to Image Registration”, IEEE Transaction on
image processing, 2(3), pp. 311-326.

[11] H. Manjunath, B. S. Mitra, 1995. “A contour–based Approach to Multisensor Image
Registration”, IEEE Transactions on image processing, 4(3), pp. 320-334.
[12] Leszek Chmielewski, Dorota Kozinska , “Image Registration”, conf. Computer Pattern
Recognition SystemsKOSYR 2003, May 26-29, 2001, Milkow,Poland, pp.163-168.
[13] Mert Rory Sabuncu,”Spatial Information in Entropy – Based Image Registration,” in
Biomedical Image Registration, LNCS 2717, Springer – Verlag, 2003.
[14] Rafael C. Gonzalez, R. E. Woods, StevanL.Eddins, Digital Image Processing (Pearson
Education, 2003).
[15] Cahill, N. D. Williams, C.M. Shoupu, “Biomedical Imaging, Nano to Macro”, 2006, 3r
d
IEEE
International Symposium, issue, 6-9 April 2006, Page(s) 832-835.
[16] J.P.W. Pluim, J.B.A. Maintz,” Mutual Information Based Registration of Medical Images,
Survey “, IEEE, Medical Imaging.
[17] “Multimodal Image Registration using Generalized Survival Exponential Entropy” Medical
image Computing, Springer, volume - 4191/2006, Cat Brain Analysis and registration, Pg-
964-971, Friday, Sept 29, 2006.
[18] Xiaoxiang Wang and Jie Tian, “Image Registration based on Maximization of Gradient
Code Mutual Information “, Image Anal Streol 2005, 24:1-7.
[19] Frederik Maes; Andre Collignon ,”Multimodality Image Rregistration by Maximization of
Mutual Information”, IEEE Transactions on medical Imaging, vol.16, No. 2, April 1997.
[20] J.P.Queiroz-Neto,M.F.M. Campos,” Automatic Geometric and Radiometric Registration of
Landsat, TM Images using Mutual Information”.
[21] C. Shannon,”A Mathematical Theory of Communication “, Bell System Technical Journal,
vol 27, pp, 379-423, 623- 656, 1948.
[22] Haim Schweitzer, "Optimal Eigenfeature Selection by Optimal Image Registration,"
cvpr,pp.1219, 1999 IEEE Computer Society Conference on Computer Vision and Pattern
Recognition (CVPR'99) - Volume 1, 1999.
[23] Ma Debao Li Wugao Le Zhongxin Wang Jiefeng “The New Matrix Characteristic
Methods of Image Fine Registration for Synthetic Aperture Radar Interferometry”,
Geoscience and Remote Sensing Symposium, 2000. Proceedings. IGARSS 2000. IEEE
2000 International, 2000, Volume: 2, On page(s): 758-760, vol.2.
[24] Haim Schweitzer,” An Eigenspace Approach to Multiple Image Registration”, NASA
Goddard Space Flight Center, 1997, pages7—12.
[25] Wen Cao; Bicheng Li; Yong Zhang,” A Remote Sensing Image Fusion Method Based on
PCA Transform and Wavelet Packet Transform”, Neural Networks and Signal Processing,
2003. Proceedings of the 2003 International Conference on Volume 2, Issue, 14-17 Dec.
2003 page(s): 976 - 981 Vol.2.
[26] Yu-Te Wu1
, Takeo Kanade2
, Ching-Chung Li3
and Jeffrey Cohn4”
Image Registration Using
Wavelet-Based Motion Model”, International Journal of Computer Vision, Springer
Netherlands, 0920-5691 (Print) 1573-1405 (Online), volume 38, Number 2 / July, 2000,
pages 129-152.

[27] Tarek A. El-hazawi, Prachya , Chalermwat, Jacqueline Le Moigne, “Wavelet-based Image
Registration on Parallel Computers” ,Proceedings of the 1997 ACM/IEEE conference on
Supercomputing (CDROM), 1997, Pages: 1 - 9 .
[28] Hala S. Own and Aboul Ella Hassanien, “Q-shift Complex Wavelet-based Image
Registration Algorithm”, Springer Berlin / Heidelberg, 1615-3871 (Print) 1860-0794
(Online), Volume 30/2005, pages 403-410.
[29] Azhar Quddus, Otman Basir, "Wavelet-Based Medical Image Registration for Retrieval
Applications”, BMEI, vol. 2, pp.301-305, 2008 International Conference on BioMedical
Engineering and Informatics, 2008.
[30] Stone, Harold S.; Le Moigne, Jacqueline; McGuire, Morgan, ”Image Registration Using
Wavelet Techniques” , Proc. SPIE Vol. 3240, p. 116-125, 26th AIPR Workshop: Exploiting
New Image Sources and Sensors, J. Michael Selander; Ed.
[31] Ghazaw, A. Chalermwa, P.,” Wavelet-Based Image Registration on Parallel Computers”,
Supercomputing, ACM/IEEE 1997 Conference, ISBN: 0-89791-985-8, On page(s): 20- 20.
[32] Jiangsheng You,Weiguo Lu, Jian Li et. al. “Image Matching for Translation Rotation and
Uniform Scaling by Radon Transform” 0-8186-8821-1/98, 1998 IEEE.
[33] Tsao. J.,”Interpolation Artifacts in Multimodality Image Registration Based on Maximization
of Mutual Informioatn “, Medical imaging, IEEE volume 22, 7, July 2003 Page(s):854-864.

A Survey OF Image Registration

More Related Content

What's hot (20)

Similar to A Survey OF Image Registration (20)

Recently uploaded (20)

A Survey OF Image Registration