Iaetsd a modified image fusion approach using guided filter

A Modified Image Fusion Approach Using Guided Filter
Ms. Deepa G. Raul1
, Prof. A. L. Renke2
Electronics and Telecommunication Department, Shivaji University, Kolhapur1
Maharashtra, India.
1dgraul.1989@gmail.com, 2amarrenke@hotmail.com
Abstract— A modified image fusion method using guided filter is proposed to combine images to give final fused image which contain
the information common in input images as well as present in either of them. The proposed method adopts simple two scale image
decomposition. Focusing on spatial context of images, guided image filter is used to preserve edge information which is uncommon in
other methods. The results drawn show the effectiveness of the proposed method. In this paper, on the basis of results, the proposed
method is compared with other two methods.
Keywords— Image fusion, Image decomposition, Guided filter, Edge preserving, Quality assessment.
I. INTRODUCTION
Among the various image processing techniques, the image fusion has a profound significance. Image fusion is nothing but
to commingle two or more images from the source in order to put together the information from them. The information may
imply here which is present in either of them and in both of them excluding noise. Images under consideration might be
multifocus images, multimodal images, multisensor images, multispectral images. Image fusion based on multiscale
decompositions and transformation like wavelet based image fusion [2] is proved to be successful. But to exploit consistency of
pixels, proposed image fusion using guided filter is featured. This makes use of spatial context to differentiate flat and edge
areas, which is used in preserving feature edges. This also includes simple two scale decomposition for which any desired
method can be selected. Also the input parameters of guided filter can be adjusted to achieve desired results of image fusion.
The paper is organized as follows section 2 gives related work. A brief description of guided filter in section 3, an overview
of fusion approach using guided filter in section 4, quality assessment measures in section 5, and simulation performance in
section 6 and section 7 concludes the paper.
II. RELATED WORK
As the image fusion has shown its importance in variety of applications like medical imaging, remote sensing, navigation,
etc. a tremendous work has been proposed. Considering the domain of image processing, the image fusion can be classified as
spatial image fusion and transform based image fusion. The transform based image fusion have been proposed such as discrete
wavelet transform based image fusion, discrete cosine transform based image fusion, stationary wavelet based transform fusion,
etc. The transform based image fusion techniques are famous methods but show the complexity of processing like lacking
translation invariance, insufficient edge preservation. While spatial image fusion techniques shows the ease of operation but fail
to use spatial properties. The spatial image fusion techniques have been proposed like averaging fusion, maximum selection
based image fusion, principal component analysis (PCA) based image fusion, intensity hue saturation (IHS) based image fusion,
etc. The image fusion using generalised random walks, Markov random fields are able to use the spatial content to the full
potential using global optimising approach. But these methods required more looping actions. Also global optimising approach
fails to control smoothing of weights. So the guided filter based image fusion adds up the features as follows:
1. Instead of decomposing image into multiple components, the image is separated into two components only by simple
processing like average filter.
2. The guided filter is local linear approach of filtering which makes full use of spatial context. Also, varying the value of the
parameters desired fusion results can be achieved.
III. GUIDED FILTER
Assuming, G and P be the guidance image and input image respectively. Also, the output of the filter can be denoted by O.
For each pixel i, the guided filter final output is local linear transform of the guidance image for window ωk which is centred at
pixel k.
Oi = akGi + bk, ∀i∈ ωk (1)
where, the coefficients ak and bk are considered to be constant in window ωk.
The cost function in window ωk is given to find coefficients ak and bk as follows:
E(ak, bk) = ∑ (i∈ωk
(akGi + bk − Pi)2
+ εak
2
) (2)
where, ε controls the value of ak which is also known as the blur degree.
The solution of the cost function considering it as linear regression model can be given by,
ak =
1
|ω|
∑ GiPi−i∈ωk
μkP̅k
σk
2+ε
(3)
bk = P̅k − akμk (4)
where, μk and σk
2
are the mean and variance of G in ωk,
|ω| gives the number of pixels in ωk ,
P̅k =
1
|ω|
∑ Pii∈ωk
, is the mean of P in ωk.
ISBN:978-1535061506
www.iaetsd.in
Proceedings of ICRMET-2016
©IAETSD 20161

Considering, all possible windows for the image, the output is given as
Oi =
1
|ω|
∑ (akk|iϵωk
Gi + bk) (5)
The average of the values for Oi is considered as its value might be different for different windows overlapping i.
Assuming, ∑ akk|iϵωk
= ∑ akk∈ωi
the symmetry of the window, rewrite above equation as
Oi = a̅iGi + b̅i (6)
where, a̅i and b̅i are the average of coefficients for all possible windows overlapping i.
The edge detection significantly depends on ε. If the edge is present in I which represent the structure of the guidance image,
the edge is transferred to the output image. For the flat region of the guidance image the output image will be average of input
in ωk. The structure of G i.e. for the edges present in the G, the output also shows edge.
Above explanation regarding gray scale or single channel can be extended to colour image by applying the filter separately to
each channel.
IV. A MODIFIED FUSION APPROACH USING GUIDED FILTER
The overall fusion approach can be represented in the three major parts as follows:
A. Image Decomposition
A simple average filtering is used to part the input images into the base layer and the detail layer. The base layer which is
direct output of the average filter is subtracted from the respective input image to get the detail layer of that image as shown in
figure 1. The major intensity variations can be observed in the blurred base layer while the detail layer gives the structure of the
image. So, the base layer is given as
Bn = In ∗ Z (7)
where, In is the nth input image, Z is the average filter.
The detail layer can be given as
Dn = In − Bn (8)
The average filter size should be carefully controlled to maintain image quality of the final fused image.
Fig.1 Two scale image decomposition
B. Guided Weight Maps
The saliency map Sn is obtained by applying the Gaussian filter on the absolute high pass image of nth image (Refer fig.2).
The saliency maps are compared with each other to get weight maps in the following way:
Pn
k
= {1 if Sn
k
= max(S1
k
, S2
k
, … . , SN
k
)
0 otherwise
(9)
where N is number of input images,
Sn
k
is the saliency value of the pixel k in the nth image.
Then the weight maps are guided by the input image in guided filter processing. The parameters are set to get suitable guided
weight maps for base and detail layers of respective input image.
C. Image Reconstruction
The guided weight maps (Wn
B
and Wn
D
) are used for weighted summation with respective image base layer and detail layer.
The result B̅ and D̅ i.e. fused base layer and fused detail layer are added together to get final fused image. (Refer fig.3)
B̅ = ∑ Wn
B
Bn
N
n=1 (10)
D̅ = ∑ Wn
D
Dn
N
n=1 (11)
Detail Layer
(D1)
Detail Layer
(D2)
Base Layer
(B1)
Base Layer
(B2)
Input Image 1
(I1)
Input Image 2
(I2)
Average Filtering
Average Filtering
Guided
Filtering
Guided
Filtering
Comparison
Saliency Measure
Saliency Measure
Input
Image2 (I2)
Input
Image1 (I1)
Saliency
Map1 (S1)
Saliency
Map2 (S2)
Weight
Map (P2)
Weight
Map (P1)
Base Weight
Map1 (W1
B
)
Detail Weight
Map1 (W1
D
)
Base Weight
Map2 (W2
B
)
Detail Weight
Map2 (W2
D
)
ISBN:978-1535061506
www.iaetsd.in
©IAETSD 20162

Fig.2 Guided weight maps
Fig.3 Image reconstruction
V. QUALITY ASSESSMENT MEASURES
Consider, A and B be the input images and F be the final fused image. The quality assessment measures used are explained in
brief as follows:
A. Peak Signal to Noise Ratio (PSNR)
It gives the easy measure of peak error taking into account mean squared error (MSE) between the input and the final fused
image, in decibels.
PSNR(dB) = 10log
N2
MSE
(12)
where, N is the maximum fluctuation in the data type. For example, N=255 for an 8 bit unsigned integer data type.
B. Normalized Mutual Information Metric (QMI)
According to the classic definition of mutual information, the results tend to bias towards the input image with higher value
of entropy. M. Hossny suggested modification as normalised version [9] which gives more relevant results. This metric gives
estimate of information which is transferred from input image to final fused image.
The normalised mutual information can be given as
QMI = 2 [
MI(A,F)
H(A)+H(F)
+
MI(B,F)
H(B)+H(F)
] (13)
where H(A), H(B) and H(F) are the marginal entropy of A, B and F respectively and MI(A, F) is the mutual information between
input image A and the fused image F.
MI(A, F) = H(A) + H(F) − H(A, F) (14)
where H(A, F) is the joint entropy between A and F and MI(B, F) is calculated similarly as MI(A, F).
C. Structural Similarity Metric (QY)
This metric gives the estimation of structural inormation transferred from input image to fused image which uses structural
similarity SSIM [8].
QY = {
λ 𝑤 SSIM(Aw,Fw) + (1 − λ 𝑤)SSIM(Bw,Fw), if SSIM(Aw, Bw|w) ≥ 0.75
max{SSIM(Aw, Fw), SSIM(Bw, Fw)}, if SSIM(Aw, Bw|w) < 0.75
(15)
where, w is the window size.
The weight λ 𝑤is given as:
λ 𝑤 =
v(Aw,)
v(Aw,)+v(Bw,)
(16)
where, v(Aw)and v(Bw) are the variance of input images A and B respectively.
D. Universal Image Quality Metric (QC)
This metric gives universal approach to measure the information transfer from input image to fused image irrespective of
viewing conditions. It uses universal image quality index (UIQI) [4].
Qc = μ 𝑤. UIQI(Aw, Fw) + (1 − μ 𝑤)UIQI(Bw, Fw) (17)
where, w is the window size.
The factor μ 𝑤 can given as
Fused
Image
Fused
base layer
Fused detail
layer
Base Weight Map1
(W1
B
)
Base Weight Map2
(W2
B
)
Detail Weight
Map1 (W1
D
)
Detail Weight
Map2 (W2
D
)
Weighted
Summing
Base Base
Layer Layer
(B1) (B2)
Detail Detail
Layer Layer
(D1) (D2)
Weighted
Summing
ISBN:978-1535061506
www.iaetsd.in
©IAETSD 20163

μ 𝑤 =
{
0, if
σAF
σAF+σBF
< 0
σAF
σAF+σBF
, if 0 ≤
σAF
σAF+σBF
< 1
1, if
σAF
σAF+σBF
> 1
(18)
where, σAF and σBF are the covariance between A, B and F.
E. Edge Information Metric (QE)
This metric gives the measure of the edge information [7] transfer from input images to fused image.
QE =
∑ ∑ QAF(i,j)wA(i,j)+QBF(i,j)wB(i,j)M
j=1
N
i=1
∑ ∑ (wA(i,j)+wB(i,j)M
j=1
N
i=1 )
(19)
where, N and M gives the width and height in terms of the number the pixels of the images respectively.
QAF(i, j) = Qg
AF
(i, j)Qα
AF
(i, j) (20)
where, Qg
AF
(i, j) and Qα
AF
(i, j) are the edge strength and orientation preservation values at pixel location (i, j)
respectively. QAF(i, j) and QBF(i, j) are weighted by wA(i, j) and wB(i, j) respectively. QBF(i, j) is calculated similarly as
QAF(i, j).
For all the metrics mentioned above, higher value means better performance of the image fusion.
VI.SIMULATION PERFORMANCE
The performance evaluation of the proposed system in Matlab environment is represented using the dataset of four pairs of
multifocus images as follows:
The guided filter parameters are set as:
For base layer: Window size (w): 7, Regularization parameter (ε): 0.3
For detail layer: Window size (w): 3, Regularization parameter (ε): 0.3
Also, the proposed guided approach results are compared with other two methods:
1. Discrete cosine transform based image fusion method
2. Discrete wavelet transform based image fusion method
A. For Multifocus Colour Input Image Pairs
(a) (b) (c) (d) (e)
Fig. 4 Performance comparison for Calendar input images
(a) Calendar Input Image 1. (b) Calendar Input Image 2. (c) DCT Method Output. (d) DWT Method Output. (e) Proposed Method Output.
(a) (b) (c) (d) (e)
Fig. 5 Performance comparison for Garden input images
(a) Garden Input Image 1. (b) Garden Input Image 2. (c) DCT Method Output. (d) DWT Method Output. (e) Proposed Method Output.
B. For Multifocus Gray Scale Input Image Pairs
(a) (b) (c) (d) (e)
Fig. 6 Performance comparison for Index input images
(a) Index Input Image 1. (b) Index Input Image 2. (c) DCT Method Output. (d) DWT Method Output. (e) Proposed Method Output.
(a) (b) (c) (d) (e)
Fig. 6 Performance comparison for Clock input images
ISBN:978-1535061506
www.iaetsd.in
©IAETSD 20164

(a) Clock Input Image 1. (b) Clock Input Image 2. (c) DCT Method Output. (d) DWT Method Output. (e) Proposed Method Output.
TABLE I. PERFORMANCE ASSESSMENT OVERVIEW FOR CALENDER INPUT IMAGES
Quality Metric DCT Method DWT Method Proposed Method
PSNR(dB) 15.28 15.68 21.50
QMI 0.91 0.88 0.92
QY 0.94 0.94 0.95
QC 0.80 0.88 0.89
QE 0.76 0.79 0.80
TABLE II. PERFORMANCE ASSESSMENT OVERVIEW FOR GARDEN INPUT IMAGES
PSNR(dB) 24.18 19.09 25.09
QMI 0.66 0.63 0.67
QY 0.86 0.90 0.91
QC 0.71 0.63 0.72
QE 0.52 0.57 0.59
TABLE III. PERFORMANCE ASSESSMENT OVERVIEW FOR INDEX INPUT IMAGES
PSNR(dB) 24.83 19.42 24.90
QMI 0.54 0.46 0.65
QY 0.96 0.97 0.97
QC 0.96 0.49 0.96
QE 0.83 0.74 0.84
TABLE 4. PERFORMANCE ASSESSMENT OVERVIEW FOR CLOCK INPUT IMAGES
PSNR(dB) 22.06 20.83 22.08
QMI 0.97 0.94 0.98
QY 0.87 0.92 0.99
QC 0.85 0.92 0.99
QE 0.75 0.77 0.79
Thus, the better results are observed with the proposed modified image fusion method using guided filter.
VII. CONCLUSION
The proposed method uses the multifocus image dataset in Matlab software. The result analysis shows how effectual the
proposed method of image fusion is. The initial decomposition step employs a simple average filtering with proper trade-off of
blurring. To use spatial consistency between pixels, guided filter is used to construct final weight maps. The parameters of the
guided filter are to be set to get desired results.
REFERENCES
[1] K. He, J. Sun, and X. Tang, “Guided image filtering,” in Proc. Eur.Conf. Comput. Vis., Heraklion, Greece, Sep. 2010, pp. 1–14.
[2] G. Pajares and J. M. de la Cruz, “A wavelet-based image fusion tutorial,” Pattern Recognit., vol. 37, no. 9, pp. 1855–1872, Sep. 2004.
[3] R. Shen, I. Cheng, J. Shi, and A. Basu, “Generalized random walks for fusion of multi-exposure images,” IEEE Trans. Image Process., vol. 20, no. 12,
pp. 3634–3646, Dec. 2011.
[4] Z. Wang and A. Bovik, “A universal image quality index,” IEEE Signal Process. Letters, vol. 9, no. 3, pp. 81–84, Mar. 2002.
[5] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process.,
vol. 13, no. 4, pp. 600–612, Apr. 2004.
[6] G. Qu, D. Zhang, and P. Yan, “Information measure for performance of image fusion,” Electron. Lett., vol. 38, no. 7, pp. 313–315, Mar. 2002.
[7] C. Xydeas and V. Petrovi´c, “Objective image fusion performance measure,” Electron. Lett., vol. 36, no. 4, pp. 308–309, Feb. 2000.
[8] C. Yang, J. Zhang, X. Wang, and X. Liu, “A novel similarity based quality metric for image fusion,” Inf. Fusion, vol. 9, no. 2, pp. 156–160, Apr. 2008.
[9] M. Hossny, S. Nahavandi, and D. Creighton, “Comments on ‘information measure for performance of image fusion’,” Electron. Lett., vol. 44, no. 18, pp.
1066–1067, Aug. 2008.
[10] M. Xu, H. Chen, and P. Varshney, “An image fusion approach based on markov random fields,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 12, pp.
5116–5127, Dec. 2011.
[11] VPS Naidu, “Discrete Cosine Transform-based Image Fusion”, Special Issue on Mobile Intelligent Autonomous System, Defence Science Journal, Vol.
60, No.1, pp.48-54, Jan. 2010.
[12] N. Ahmed, T. Natarajan and K.R.Rao, “Discrete Cosine Transform”, IEEE Trans. On Computers, Vol.32, pp.90-93, 1974.
[13] H Li, S Munjanath, S Mitra, “Multisensor Image Fusion Using the Wavelet Transform”, Graphical Models and Image Proc., Vol. 57, No. 3, 1995, pp
235-245.
[14] O. Rockinger, “Image sequence fusion using a shift-invariant wavelet transform,” in Proc. Int. Conf. Image Process., vol. 3, Washington, DC, USA, Oct.
1997, pp. 288–291.
[15] R.C. Gonzalez and R.E. Woods, Digital Image Processing, second ed. Prentice Hall, 2002.
ISBN:978-1535061506
www.iaetsd.in
©IAETSD 20165

Iaetsd a modified image fusion approach using guided filter

More Related Content

What's hot (20)

Viewers also liked (15)

Similar to Iaetsd a modified image fusion approach using guided filter (20)

More from Iaetsd Iaetsd (20)

Recently uploaded (20)

Iaetsd a modified image fusion approach using guided filter