Super Resolution of Image

Super-Resolution of
Text Images using
Ant Colony Optimisation
Project By,
Gowtham Siddarth.D (2010115070)
Santhoshkumar.S (2010115101)
Satheesh.K (2010115102)
Guide : Dr.K.Vani

OBJECTIVE
To convert Multiple Low
resolution images of a
document into a high
resolution image .

SCOPE
 Recover original text image from
quantization noise and grid-alignment
effects that introduce errors in the low-
resolution image
 Avoid artifacts in the high-resolution
image such as blurry edges and
rounded corners
 Super resolution address the lack of
sharpness in the text image

LITERATURE STUDY
S.
No:
REFERENCE PAPER AND
AUTHOR
DESCRIPTION
1. JOINT IMAGE
REGISTRATION AND
SUPER-RESOLUTION
FROM LOW-
RESOLUTION IMAGES
WITH ZOOMING
MOTION by Yushuang
Tian and Kim-Hui Yap,
Senior Member, IEEE –
July 2013
This paper proposes a new framework for
joint
image registration and high-resolution (HR)
image reconstruction from multiple low-
resolution (LR) observations with zooming
motion. Conventional super-resolution (SR)
methods typically formulate the SR
problem as a two-stage process, namely,
image registration followed by HR
reconstruction
2. LEARNING SPATIALLY-
VARIABLE FILTERS
FOR SUPER-
RESOLUTION OF TEXT
by Adrian Corduneanu
and John C. Platt - 2005
The algorithm for super-resolution of text
magnifies images in real-time by
interpolation with a variable linear filter. The
coefficients of the filter are determined
nonlinearly from the neighborhood to which
it is applied. We train the mapping that
defines the coefficients to specifically
enhance edges of text, producing a
conservative algorithm that infers the detail
of magnified text

S.
No:
REFERENCE PAPER AND
AUTHOR
DESCRIPTION
3. ANT COLONY
OPTIMIZATION FOR
IMAGE
REGULARIZATION
BASED ON A
NONSTATIONARY
MARKOV MODELING by
Sylvie Le Hégarat-Mascle,
Abdelaziz Kallel, and
Xavier Descombes –
March 2007
The ants collect information through the
image, from one pixel to the others. The
choice of the path is a function of the pixel
label, favoring paths within the same image
segment. We show that this corresponds to
an automatic adaptation of the
neighborhood to the segment form, and that
it outperforms the fixed-form neighborhood
used in classical
Markov random field regularization
techniques
4. ANT COLONY
OPTIMIZATION BASED
FUZZY IMAGE FILTER
DESIGN FOR REMOVAL
OF IMPULSE NOISES by
Min-Chi Kao, Chia-Hung
Lin, and Tzuu-Hseng S. Li
– June 2013
The fuzzy system is utilized to
improve the traditional median filter, and an
ant colony optimization (ACO) algorithm is
used to adjust the parameters of fuzzy
image filter and make the filter to achieve
better
performance

S.
No:
REFERENCE PAPER AND
AUTHOR
DESCRIPTION
5. DISCRETE WAVELET
TRANSFORM-BASED
ANT COLONY
OPTIMIZATION FOR
EDGE DETECTION by
Aminu Muhammad,
Ibrahim Bala,
Mohammad Shukri
Salman and Alaa Eleyan
- 2013
Ant Colony Optimization (ACO) is used
to obtain the edges of an image which
is acquired from sampling and
quantization of a continuous image.
Such techniques generate a pheromone
matrix that epresents the edge
information at each pixel position on the
routes formed by ants dispatched on
the image.
6. SINGLE-FRAME TEXT
SUPER-RESOLUTION: A
BAYESIAN APPROACH
by Gerald Dalley, Bill
Freeman, Joe Marks -
2004
given a single image of text . return the
image that is generated from a
noiseless high-resolution scan. In doing
so, we : ( I ) avoid introducing artifacts
in the high-resolution image such as
blurry edges and rounded corners, (2)
recover from quantization noise and
grid-alignmont effects that introduce
errors in the low-resolution image

ARCHITECTURE
Fusion
Soft Classification
Super Resolution
Mapping

CONTROL POINT REGISTRATION
Input Image
1
Input Image
2
Select Matching Control
points
Estimate Transformation
Solve for Scale and Angle
Transform the image
Registered image

AUTOMATIC REGISTRATION
Input Image
1
Input Image
2
Feature Detection Using
SURF Algorithm
Extract Features
Match the relevant
features
Estimate Transformation
Recover original image
Registered image

FUSION - METHOD 1
(Intensity Based Fusion)
Registered
Image 1
Registere
d Image 2
Intensity 1
Intensity 2
+ Final Fused
Image
min(Intensity 1, Intensity 2)

FUSION – METHOD 2
(Discrete Wavelet
Transformation)
Registere
d Image 1
Registere
d Image 2
+
Final
Fused
Image
LLf(i,j)= ( LL1(i,j) + LL2(i,j) ) / 2
LL1 LH1
HL1 HH1
LL2 LH2
HL2 HH2
LLf LHf
HLf HHf
IDWT
DWT
DWT

Fused
Image
Identify
Classes
C2 C5
Classification Using Decision
Tree
Calculating similarity between
pixels
SOFT CLASSIFICATION
C1- 0 % C2- 25 %
C3C1 C4
C3- 50 % C4- 75 % C5- 100 %
Update class labels
Area proportional image

Initialize
Place each ant in each pixel in a
group
For each ant
Choose next
pixel
Find a
pixel of
class c1
Return to initial pixel
Update trace level using the tour cost for each
ant
Stopping
Criteria
Find the pixels with nearest class
c1
No
No
Yes
Yes

1. REGISTRATION
 Image registration is the process of
transforming different sets of data into
one coordinate system. Data may be
multiple photographs, data from different
sensors, times, depths, or viewpoints.
 It is used in computer vision, medical
imaging, military automatic target
recognition, and compiling and analyzing
images and data from satellites.
 Registration is necessary in order to be
able to compare or integrate the data
obtained from these

 When a picture is scanned using the
same sensor multiple times, there will
be disorientation in the pixel alignment
of the images.
 There are three types of alignment
disorder
 Vertical disorder
 Horizontal Disorder
 Angular Disorder

Input Image Input Image
Registered Image

STEPS FOR AUTOMATIC
REGISTRATION
1. Find Matching Features Between
Images
2. Detect features in both images
3. Extract feature descriptors
4. Match features by using their
descriptors
5. Retrieve locations of corresponding
points for each image
6. Estimate Transformation
7. Solve for Scale and Angle

PSEUDO CODE
do
do
do
for all interest area in given input image,
calculate Hessian Matrix H (5×5)
end
Identify two interest area with same determinant value;
Mark as a feature;
end
divide the feature (interest area) into 4×4 subarea;
find deviation in x and y axis (estimating transformation);
get the angle of deviation as a trace of Hessian matrix;
recover the original image by inverse transformation;
end

ALGORITHM USED
 SURF Algorithm
1. Detection
 Automatically identify interesting features
2. Description
 Each interest point should have a unique description
that does not depend on the features scale and
rotation.
3. Matching
 Given and input image, determine which objects it
contains, and possibly a transformation of the object,
based on predetermined interest points.

1. DETECTION
The determinant of a Hessian Matrix
expressed as
where

DETECTION
The matrix with same value are detected as feature

2. DESCRIPTION
The interest area is divided into 4×4 subareas that is
described by the values of a wavelet response in the
x and y directions.

3. MATCHING
Matching in SURF Algorithm is done by
where
Is the trace of Hessian Matrix

TEST CASE
Input Image 1 Input Image 2
Matched Features

TEST CASE (Cont…)
Matching Inliers
Registered Image

FUSION METHOD 1
 Image fusion is the process of
combining relevant information from
two or more images into a single
image
 The resulting image will be more
informative than any of the input
images

TEST CASE – INPUT (Auto Registered)
a
)
b)

TEST CASE – OUTPUT (Auto
Registered)

TEST CASE – INPUT (Control Point
Registered)
a
)
b)

OUTPUT (Control Point Registered)

COMPARISON
 Fusion of images registered using
Automatic Feature Detection is
always better than that registered
using Control Point Registration

MATHEMATICAL NOTATION FOR FUSION
The samples are passed through a low pass
filter with impulse response g resulting in a
convolution of the two
The signal is also decomposed simultaneously us
a high-pass filter. The outputs giving the detail
coefficients h and approximation coefficients g

2. SOFT CLASSIFICATION
(FUZZY CLASSIFICATION)
 Multiple images will not have distinct
values in a pixel.
 Pixel information is taken as a vector
of multiple classes.
 For higher resolution of the same
image, the vector information can be
used to resolve the percentage of
different class (black &white)

PSEUDO CODE
do
for each pixel use decision tree classification
Initialize: Set value for maximum no. of iteration. Set
maxItem(i)=-1 for each pixel i.
while item<maxItem
Start: for each pixel Ni
Initialize: Set the initial maximum similarity
maxSimilarity=0
For each tile center Nj from a 2 region Size*2
regionSizesquare neighborhood around Ni
1.Determine the 3×3 image patches INi and INj which
include the central pixel Nj, respectively
2.Calculate the pixel Similarity S(I,j) for Ni and Nj
If maxSimilarity < S(I,j)
maxSimilarity=S(I,j); l(i)=j
Item=item+1;
end

PIXEL INTENSITY SIMILARITY is
defined as
where ri,j,k denotes the kth pixel intensity
quotient of the patches INi
and INj
P(ri,j,k) is the Probability Distribution Function
FORMULA

FORMULA
The Pixel Location Distance is defined
as
where xi, yi, xj, yj are the spatial coordinates of Ni a
Nj , respectively. The Euclidean distance dXY (i, j) d
the pixel location distance

FORMULA
Pixel Location Similarity SXY (i, j) as

FORMULA
From Pixel Intensity Distance and Pixel
Location Similarity the Pixel Similarity
Measure is defined as

SUPER RESOLUTION
MAPPING
REDUCE PIXEL SIZE:
 Increase the number of pixels per unit
area.
 Advantage:
Increases spatial resolution.
 Disadvantage:
Noise introduced.

SUPER RESOLUTION
If the weight of the pixel is 100% then we
can
automatically fill the corresponding pixels

 Techniques for super resolution
mapping
 Hopfield Neural Networks
Genetic Algorithm
Support Vector Machine
Ant Colony Optimization

PSEUDO CODE
do
For all n , that is n = 1 : N identify the search starting point
and; initial value of each element of pheromone matrix t(
0)
do
For all m, that is m = 1 : M i.e M <= N
do
for every ant k=I:K
do
Locate the present position by the moving ants
and pheromone update, and then store r( n)
position
end
Update visited Pixel
end
end
end

FORMULA
 Initialization Stage
 Construction Stage
 Update
 Decision
 Global Update

INITIALIZATION STAGE
The Heuristic Matrix is calculated after the
initialization of the ants search position and
is given as
where Vc is the variation in intensity, Ii,j is the
intensity value and Z is the normalization factor
given as,

CONSTRUCTION STAGE
The Transition Probability Pi,j
(n) is used by ant
to determine the path to be followed

UPDATE SATGE
In this step, locally update for the
Pheromone Matrix is made as

DECISION
Here decision is made whether the ants are
moved or not, in reference to the pre-assign
Value called threshold as given by
Then determine the T(MBT) and T(MAT) which
are mean below and above threshold respectively
using T.
here T(i) is the threshold value,

GLOBAL UPDATE
The Global Update is obtained as

TEST CASE 1(Angular
Displacement)
INPUT IMAGE 1 INPUT IMAGE 2

SOFT CLASSIFIED AND
SUPER RESOLVED IMAGE

COMPARISION (IDEAL &
OP)
522 × 799 Resolution 1044 × 1598 Resolution

TEST CASE 2 (Angular and
Vertical Displacement)

OP)

TEST CASE 3 (Blurred
Image)

OP)

OUTPUT QUALITY
EVALUATION
CORRELATION FACTOR
(with reference to ideal image)
Noise Measurement of Peak Signal to
Noise Ratio PSNR
(with reference to ideal image)
Soft Classified Image Ant Colony
Optimization
Soft Classified
Image
Ant Colony
Optimization
Test case 1 0.4672 0.4753 21.8221 21.5878
Test case 2 0.2528 0.2626 18.0467 17.8114
Test case 3 0.3320 0.3594 18.3712 18.1159
Test case 4 0.3446 0.3608 17.2401 16.8948

CONCLUSION
 Using Ant Colony Optimization for super
resolution enhances quality of texts but
add noises to the image for some
characters.
 Eliminating those noises at pixel level is
difficult.
 But using Ant colony optimization for
super resolution takes only less time.
 So in the case where small amount of
noises are acceptable and if the
processing is to be made quickly then
Ant Colony Optimization can be adopted
for super resolution mapping of text
images.

REFERENCES
 Yushuang Tian and Kim-Hui Yap, “Joint Image Registration and Super-
Resolution from Low-Resolution Images with Zooming Motion”, IEEE
Transactions on Circuits and Systems for Video Technology, Vol. 23, No.
7, July 2013.
 Hankui Zhang , Bo Huang , “Support Vector Regression-based
Downscaling for Intercalibration of Multiresolution Satellite Images”,
IEEE Transactions on Geoscience and Remote Sensing, 2013.
 Aminu Muhammad, Ibrahim Bala, Mohammad Shukri Salman and Alaa
Eleyan , ”Discrete Wavelet Transform-based Ant Colony Optimization for
Edge Detection”.
 Robert A. Ulichney and Donald E. Troxel , “Scaling Binary Images with
the Telescoping Template”, IEEE Transactions on Pattern Analysis and
Machine Intelligence, Vol. 3, May 1982.
 Sylvie Le Hégarat-Mascle, Abdelaziz Kallel and Xavier Descombes ,
“Ant Colony Optimization for Image Regularization based on a
Nonstationary Markov Modeling”, IEEE Transactions on Image
Processing, Vol. 16, No. 3, March 2007.
 Adrian Corduneanu and John C. Platt, ”Learning Spatially-Variable
Filters for Super-Resolution of Text”.
 Gerald Dalley, Bill Freeman, Joe Marks , “SINGLE-FRAME TEXT
SUPER-RESOLUTION: A BAYESIAN APPROACH” , International
Conference on Image Processing (ICIP), 2004.

Super Resolution of Image

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to Super Resolution of Image (20)

Super Resolution of Image