A Survey on Different Relevance Feedback Techniques in Content Based Image Retrieval

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 02 | Feb -2017 www.irjet.net p-ISSN: 2395-0072
© 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 582
A Survey on Different Relevance Feedback Techniques in Content
Based Image Retrieval
Athira Mohanan1, Sabitha Raju2
PG student, Dept. Computer science engineering, VJCET, Vazhakulam, Kerala, India
Assistant professor, Dept. Computer science engineering, VJCET, Vazhakulam, Kerala, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - The conventional image retrieval methods like
Google, Bingo, Yahoo are based on the on textual annotation
of images to access the large collection of relevant database
images. Then Content Based Image Retrieval (CBIR) is a
technique, which takes visual contents of image to retrieve
relevant images from large databases. InContentBasedImage
Retrieval, there is a semantic gap between the low level
features and high level semantic concepts. Different relevance
feedback techniques bridge this semantic gap. In this paper
analyse different subspace learning based relevance feedback
algorithm to retrieve images.
Key Words: Content based image retrieval, Semantics gap,
Relevancefeedback,Feature modification,Subspacelearning
1. INTRODUCTION
Content Based Image Retrieval (CBIR) has attracted much
attention during the past decades. CBIR is an imageretrieval
techniques used to retrieve relevant images without using
any image annotations. CBIR systems uses visual content of
an image such as color, shape and texture features as image
index [9]. The CBIR systems adopt the Euclidean distance
metric in a high dimensional low level visual featurespace to
measure the similarity between the query image and the
images in the database. But the Euclidean distance metric in
a high-dimensional space is usually not very effective due to
the gap between the low-level visual features and the high
level semantic concepts [8]. Thus performance of CBIR
system is poor due to the semantics gap between the input
image and low level visual features [9]. The effect of
semantics gap is avoided by using relevance feedback
technique.
Relevance feedback is a powerful tool and online learning to
retrieve most relevant images. This strategy ask usertogive
some feedbacks on the results returned in the previous
query round and come up with a better resultbasedonthese
feedbacks. A variety of relevance feedback techniques
designed to bride the semanticsgapbetweenlowlevel visual
features and high level semantic concept of each image [7].
The general process of Relevance Feedback is as follows:
First user labels a number of relevant images as positive
feedback and a number of irrelevant images as negative
feedback from retrieved images. Then the CBIR system then
refines its retrieval procedure based on these labeled
samples. These processes carried out iteratively. RF
techniques are classified into two categories: that is query
movement and biased subspace learning. In this biased
subspace learning, all positive samples are alike and each
negative samples in negative in its own way [8].
2. LITERATURE SURVEY
In [1] Anelia Grigorova, Francesco G. B. De Natale, Charlie
Dagli, Thomas S. Huang, Life Fellow, presents a feature
adaptation techniques to retrieve more relevant images. It is
an effective feature space dimension reduction according to
user’s feedback, but also improves the image description
during the retrieval process by introducing new significant
features. FA-RF uses two iterative techniques to make use of
the relevance information that is query refinement and
featurere-weighting. FortheadaptationofacrossRFusesthe
descriptions of both relevant and irrelevant image, as wellas
their number and proportions. The query image is located
near to the boundary of the relevant cluster in the feature
space then the system containsfewrelevantimages.Thusthe
query refinement mechanism is useful to move the query
towards the middle of the cluster of relevant images in the
feature space. This FA-RF performs very well in terms of
capability in identifying most important features and
assigning them higher weights compared with classical
feature selection algorithms. Also maintain compact image
description. The main drawbacks are less efficient for large
databases. There is also needs an efficient feature extraction
algorithm.
In [2] Mohammed Lamine Kherfi andDjemelZiouproposeda
new RF framework that combines the advantages of using
both the positive example (PE) and the negative example
(NE). This method learns imagefeaturesandthenappliesthe
results to define similarity measures that correspond to the
user judgement. The use of the NE allows images undesired
by the user to be discarded, thereby improving retrieval
accuracy. This method tries to learn the weights the user
assigns to image features and then to apply the results
obtained for retrieval purposes. It also reduces retrieval
time. It clusters the query data into classes and model
missing data, and support queries with multiple PE and/or
NE classes. The main function of thismethodisthatitassigns
more importance to featureswith a highlikelihoodandthose
which distinguish well between PE classes and NE classes.
The drawbacks are small sample problem. Also the use of PE
is sufficient to obtain satisfactory results.
In [3] Dacheng Tao, Xiaoou Tang, Xuelong Li and Xindong
Wu, presents an Asymmetric BaggingandRandomSubspace
based Support Vector Machine (ABRS-SVM) to solve the
problems of SVM in image retrieval andoverfitting problem.

The bagging incorporates the benefits of both bootstrapping
and aggregation. In bootstrappingmultipleclassifierscan be
generated by training on multiple sets of samples that is
random sampling with replacement on thetrainingsamples.
Aggregation of the generated classifiersthenisimplemented
by majority voting. The bootstrapping is executed only on
the negative feedback samples because there are far more
negative feedback samples than the positive feedback
samples. Each generated classifier is trained on a balanced
number of positive feedback samples. The asymmetric
bagging strategy solves performance degradation of SVM
classifier. The small sample sized problem is solved byusing
Random Subspace based SVM. RSM performs the
bootstrapping in the feature space. The over fittinghappens
when the training set is relatively small compared with the
high dimensional feature vector. In order to avoid over
fitting, sample a small subset of features to reduce the
difference between the training data and the feature vector
length. Using this random samplingmethod,firstconstructa
multiple number of SVMs and then combine these SVMs to
construct a more powerful classifier. The main drawback of
this system is it does not handle unlabeled samples.
In [4] Ja-Hwung Su, Wei Jyun Huang, Philip S. Yu, Fellow,and
Vincent S. Tseng, proposed a Navigation Pattern based
Relevance Feedback (NPRF) achieve high efficiency and
effectiveness with the large scale image data. Also reduces
number of iterative feedbacks to produce refined search
results. The iterative feedbacks are reduced substantiallyby
using the navigation patterns discovered from the user
query log. This NPRF approach is divided into two
operations that is the online image retrieval and offline
knowledge discovery. NPRF Search makes use of the
discovered navigation patterns and three kinds of query
refinement strategies such asQueryPointMovement(QPM),
Query Reweighting (QR), and Query Expansion (QEX). The
query image is submitted to this system,andthenthesystem
first finds the most relevant images and returns it. This
process is called initial feedback. Next, the positive samples
picked up by the user is given to the image search phase
including new feature weights, new query points and user’s
intention. Navigation patterns with three search strategies
are included to find the desired images. For each user’s
browsing behaviours, offline operation for knowledge
discovery is triggered to perform navigation patternmining.
The main drawbacks of this system are image retrieval in
global feature space and results depends only on the
navigation pattern of users.
Figure 1: Workflow of NPRF Search [4]
In [5] Wei Bian and Dacheng Tao proposed a new
dimensionality reduction algorithm for relevance feedback
in the content based image retrieval is called Biased
Discriminative Euclidean Embedding (BDEE). The samples
in the original dimensional ambient space is transformed to
low level visual features to discover intrinsic coordinates of
an image. BDEE models both the interclass geometry and
interclass discrimination of each image. It does not ignore
the manifold structure of samples. BDEE is a subspace
learning method in which mapping vector is used to map
high dimensional space to low dimensional space.
Figure 2: Architecture of CBIR system
In the process of BDEE technique distance betweenpositive
samples and negative samples should be large and distance
between positive samples should be small. When a query
image is given to the system, first low level visual features
are extracted. Then all images in the database are sorted
based on the Euclidean distance. If the user is not satisfied
with the initial results then the Relevance Feedback process
is started. The user labels some top queryimagesaspositive
or negative samples. This RF model is trained and updated
based on BDEE algorithms. The advantages are reduces

under sampled problem, reduces computational complexity
and maintain the manifold regularization structure. Also
consider unlabeled samples for dimensionality reduction.
In [6] Yu-Chen Wang, Chin Chuan Han, Chen-Ta Hsieh, Ying-
Nong Chen, and Kuo-Chin Fan proposed a Feature Line
Embedding Biased Discriminant Analysis ( FLE-BDA) for
performance enhancement in relevance feedback scheme.It
maximizing marginbetween relevantandirrelevantsamples
at local neighborhood so that relevant images and query
image can be quite close, while irrelevant samples are far
away from relevant samples. In this subspace learning
method, find a linear transformationmatrixfromrelevantor
irrelevant images that is used in dimensionality reduction.
The retrieval process includes 1) A query image is inputted
to the IR system. After calculating the similarity values,
gallery images are ranked. 2) Users label the relevant or
irrelevant images according to their preference. 3) Then
user’s’ feedback is adopted to find a new transformation. 4)
The gallery images are re-ranked to obtain the retrieval
results in the next round. Two labels are assigned to the top
ranking images according to users’ preference. Feedback
with relevant or irrelevant labels represents users’
preference. The within class scatter is calculated from the
image samples with positive labels, while the between-class
scatter is calculated from those with negative labels. Based
on these assigned labels, the within class and between-class
weighted graphs are constructed for maximizing themargin
of relevant and irrelevant samples. Then new distance
between query and images are calculated. The advantages
are dimensionality reduction, solve singular problem in the
high dimensional space, increases generalization and
robustness using Laplacianregularization.Thedisadvantage
are computational complexity is very high due to the large
scale dataset.
In [7] Lining Zhang, Lipo Wang, and Weisi Lin [3] proposed
an conjunctive patches subspace learning(CPSL)method for
learning an effective semantic subspace by exploiting the
user historical feedback log data with the current data.CPSL
effectively integrate the discriminative information of
labeled log images, geometry information of labeled log
images and weakly similar information of unlabeled images.
For creating a reliable subspace,needto builddifferentkinds
of local patches for each image. Apart from other Relevance
Feedback techniques, Collaborative Image Retrieval system
integrates regular online RF schemes with an offline
feedback log data. From the figure, the CIR systems first
collect RF information from user which can be stored in an
RF log database. If user feedback log data is unavailablethen
the CIR system performs exactly like RF based CBIR system.
If the user RF information is available, the algorithm can
effectively exploit the user feedback log data. The image
retrieval can be done in less iteration than regular RF
schemes with the help of the user historical feedback log
data. The advantages are there is no need for the explicit
class label information for images in the dataset and also
consider local information of each image. The disadvantage
is increasing time complexity to take bothuserdata anduser
feedback log data.
Figure 3: Architecture of Log Based CBIR System
In [8] Lining Zhang, Hubert P. H. Shum and Ling Shao
proposed a discriminative semantic subspace analysis
(DSSA) method to bridge the gap between low level visual
features and high-level semantic concepts by exploiting the
training imageswithpairwiseconstraints.ThisDSSAmethod
effectively learn a reliable subspace from both labeled and
unlabeled images with similar and dissimilar pairwise
constraints without using any explicit class label
information. DSSA integrates the local geometry of labeled
similar images, the discriminative information between
labeled similar and dissimilarimages,andthelocal geometry
of unlabeled images. First the low level visual features are
first extracted then all images in the database are sorted
based on a predefined similaritymetric.Thesystemrequires
user to label some semantically similar and dissimilar
images as the positive and negative feedback samples,
respectively. Using these labeled similar and dissimilar
samples as the training data, RF model can be obtained
based on certain machinelearningtechniques.Thesimilarity
metric can thus be updated together with the RF model.
Then, all images are sorted based on the recalculated
similarity metric. If the user is satisfied with the refined
results, RF is no longer required and the system gives the
final results, which are the most semantically similarimages
with the query image.Otherwise,RFisperformediteratively.
The advantages of DSSA involve the local similar and
dissimilar pairwise constraints of feedback samples and do
not impose any label constraints on feedback samples. It
effectively finds most discriminative subspace compared
with classical supervised subspace analysis methods with
explicit class label information. It never meets the problem
of numerical computation.

Figure 4: Framework of RF based CBIR system
3. CONCLUSIONS
Content based image retrieval is a technique to retrieve
more relevant images. Retrieve similar images only is a
standing problem in digital image processing. The
performance of CBIR system is improved by introducing
relevance feedback techniques inthesystem.Several feature
modification and subspace learning based relevance
feedback methods are studied. Various systems use feature
modification of each image and tries to retrieve relevant
images. But these systems do not suitable for high
dimensional images. Several subspace learning relevance
feedback methods provides morerelevantimagescompared
with feature modification based methods. It also considers
local information of images and aims those similar images
close to but dissimilar images are far away from query
image. This paper focuses on the different relevance
feedback techniques in digital image processing.
REFERENCES
[1] Anelia Grigorova, Francesco G. B. De Natale, Charlie
Dagli, Thomas S. Huang, Life Fellow,”Content Based
Image Retrieval by Feature Adaptation and Relevance
Feedback,” IEEE Transaction on Multimedia, vol. 9, no.
6, October 2007.
[2] Mohammed Lamine Kherfi and Djemel Ziou “Relevance
Feedback for CBIR: A New Approach Based on
Probabilistic Feature Weighting With Positive and
Negative Examples,” IEEE Transaction on Image
Processing, vol. 15, no. 4, April 2006.
[3] Dacheng Tao, Xiaoou Tang, Xuelong Li and Xindong Wu,
“Asymmetric Bagging and Random Subspace for
Support Vector Machines Based RelevanceFeedback in
Image Retrieval ”, IEEE TransactiononPatternAnalysis
and Machine Intelligence, vol. 28, no. 7, July 2006.
[4] Ja Hwung Su, Wei Jyun Huang, Philip S. Yu, Fellow, and
Vincent S. Tseng, “Efficient Relevance Feedback for
Content Based Image Retrieval by Mining User
Navigation Patterns ”, IEEE Transaction on Knowledge
and Data Engineering, vol. 23, no 3, March 2011.
[5] Wei Bian and Dacheng Tao,“Biased Discriminant
Euclidean Embedding for Content Based Image
Retrieval ”, IEEE TransactionsonImageProcessing,vol.
19, no. 2, February 2010.
[6] Yu Chen Wang, Chin Chuan Han, Chen Ta Hsieh, Ying-
Nong Chen, and Kuo-Chin Fan, “Biased Discriminant
Analysis With Feature Line Embedding for Relevance
Feedback BasedImageRetrieval”,IEEETransactionson
Multimedia, vol. 17, no. 12, December 2015.
[7] Lining Zhang, LipoWang and Weisi Lin, “Conjunctive
Patches Subspace Learning With Side Information for
Collaborative Image Retrieval,” IEEE Transactions on
Image Processing, vol. 21, no. 8, August 2012.
[8] Lining Zhang, Hubert P. H. Shum and Ling Shao,
“Discriminative Semantic Subspace Analysis for
Relevance Feedback,” IEEE Transaction on Image
Processing, vol. 25, no. 3, March 2016.
[9] Y. Rui, T. S. Huang, M. Ortega, and S. Mehrotra,
“Relevance feedback: A power tool for interactive
content-based image retrieval,” IEEE Trans. Circuits
Syst. Video Technol., vol. 8, no. 5, pp. 644–655, Sep.
1998.

A Survey on Different Relevance Feedback Techniques in Content Based Image Retrieval

More Related Content

What's hot (19)

Similar to A Survey on Different Relevance Feedback Techniques in Content Based Image Retrieval (20)

More from IRJET Journal (20)

Recently uploaded (20)

A Survey on Different Relevance Feedback Techniques in Content Based Image Retrieval