Remote Sensing Image Fusion using PCNN Model Parameter Estimation by Gamma Distribution in Shearlet Domain Academic research paper on "Materials engineering"

CrossMark

Available online at www.sciencedirect.com

ScienceDirect

Procedía Computer Science 70 (2015) 304 - 310

4thInternational Conference on Eco-friendly Computing and Communication Systems (ICECCS)

Remote sensing image fusion using PCNN model parameter estimation by Gamma distribution in shearlet domain

Biswajit Biswasa*, Biplab Kanti Sena, Ritamshirsa Choudhuria

aUniversity of Calcutta, 92-A.P.C road, Kolkata and 700009, India

Abstract

Here the proposed approach deals with some adaptive parameters in pulse coupled neural network (PCNN) model which are highly suitable in image fusion. Initially, the source images are separately decomposed into multi-scaled and multi-directional bands by shearlet transform (ST). Later, the PCNN model is mapped between the decomposed low pass ST sub-bands which depends on Linking pulse response and coupling strength with regional statistics of ST coefficients. The process of different high pass ST sub-bands and utilization of singular value decomposition (SDV) have been discussed in details. Finally, we have obtained fusion results by the inverse shearlet transformation (IST). The experimental results on satellite images show that the proposed method has good performance and able to preserve spectral information and high spatial details simultaneously like the original source images. The objective evaluation criteria and visual effect illustrate that our proposed method has a better edge over the prevalent image fusion methods.

©2015 The Authors.PublishedbyElsevierB.V. Thisisanopenaccess articleunder theCCBY-NC-NDlicense (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-reviewunderresponsibilityoftheOrganizing Committee of ICECCS 2015

Keywords: Remote sensing image fusion, shearlet transform, pulse code neural network, single value decomposition, gamma distribution, spectral angle mapper

1. Introduction

Mainly remote sensing images are obtained from multiple satellite image sensors1. Now satellite imaging system produces high spatial PAN and multi-spectral MS images which are widely utilized in the arenas of remote sensing image analysis, feature extraction, modeling, image classification, target detection and recognition1,3. Generally, image fusion can be defined as the combination of visual information contained in any number of source images into a single fused image without any kind of spectral distortion or information loss3,4. In practice, the most image fusion method are performed at the pixel level. Most pixel-level image fusion schemes are Intensity-Hue-Saturation (IHS) technique1, Principal Components Analysis (PCA) approach1, Bravery Transform (BT)1, Gram-Schmidt technique (GST)2, Laplacian pyramid method (LP)1, Gaussian contrast pyramid2 and different statistical based schemes such as MS+PAN sharpening2, Bayesian approach3, Markov Random Fields approach3 as well as various soft computing

E-mail address: biswajit.cu.08@gmail.com

1877-0509 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the Organizing Committee of ICECCS 2015 doi:10.1016/j.procs.2015.10.098

approach1,3 etc. Based on multi-resolution theory, different pixel-level wavelet-based image fusion techniques like discrete wavelet transform (DWT)1,6, complex wavelet transform (CxWT)6,9, a trous wavelet transform (TDWT)1,9 and etc. have been developed which produces high quality fused images. As known , images fused by wavelets suffer from lack of spatial information, salient features and loss of geometrical information6,9. Also Multi-scale geometric analysis (MGA) is an efficient computational and special version of MSD model which can analysis the multi-resolution and geometric nature of a high dimensional signal6,9. Various MGA models have successfully been developed like ridgelet, curvelet, contourlet, bandlet, shearlet which are discussed in6,9. These MGA methodologies are highly effective to represent edge information than wavelet systems and are appropriate for extracting geometrical information from an image6,9. Shearlets is a new MGA methodology that equipped with a tight Parseval frame at various scales and directions, and optimally sparse in representing the geometric quality of an image such as edges9.

In this work, we have applied the benefits of ST and PCNN to implement a new fusion framework which can process and combine ST coefficients adeptly. PCNN have been effectively utilized in the proposed fusion scheme. The advantages of SVD have efficiently been utilized in this method to normalize the PAN image as compatible with MS image to avoid the spectral deformation. The low pass ST sub-bands are processed and the largest low pass ST sub-bands are fused by PCNN. The approach evaluates the related high pass ST sub-bands and the directional ST sub-bands of the images at different decomposition levels. Finally reconstruction of the fused image by inverse ST has been performed. Experimental results demonstrate the efficiency of the proposed fusion technique for analysis of remote sensing satellite images.

The rest of this paper is organized as follows. A brief introduction of ST is given in Section 2. Section 3 introduces PCNN. Techniques for construction and modification of low pass and high pass ST bands are illustrated given in Section 5. The fused results and discussions are briefed in Section 6. Finally, we conclude this paper in section 7.

2. Shearlet transform

Typically, ST have been developed based on an affine system with composite dilations9,10. Let us briefly discuss the continuous and discrete ST at the fixed resolution level j in the following subsections.

2.1. Continuous shearlet system

In dimension n = 2, the affine systems with composite dilations are the collections represented as9,10:

The elements of this system are known as composite wavelets if ¥ab(w) forms a Parseval frame or tight frame for f £ L2(R2). In this system, the dilations matrices Aj are the scale transformations, while the matrices Bl are the area-preserving geometric transformations, such as rotations and shear. As stated in the continuous ST, we have9,

^ab (w) = iWj,k,l (X) = |detA\j/2W (b1AX - k) : j, l £ Z, k £ Z2}

where Va

1. W(e) = W(ei, £2) = Wi(ei)W2(e2/ei);

2. Wi £ C"(M) suppwi C [-2, -1/2] U[1/2,2], where, Wi is continuous wavelet;

3. W2 £ C"(r) suppW2 C [-1,1], W2 > 0 But || y2|| = 1

(a) (b) (c)

Fig. 1: The structure of the frequency tiling by the ST: (a) The tiling of the frequency plane R2 induced by the ST, (b) The size of the frequency support of a ST Yjjjc, (c) The multi-scale and multi-directional decompositions of ST9.

For ya,s,k,a € R+, s € R+ and k € R+, for any f e L2(R2), is called shearlet, more specifically a collection of wavelets with different scales. Here, the anisotropic expansion matrix Us is associated with the scale transform and the shear matrix Va specify the geometric transformation. Mostly, a = 4 and s = 1. Where a,s, k are specified as scale transformations, shear direction and translation vector respectively.

2.2. The discrete shearlet system

The process of the discrete ST can be divided into two steps such as multi-scale subdivision and direction localiza-tion9,10. Figurel shows an example decomposition process using ST.

In this system, at any scale j, let f e L(ZN). Firstly, the Laplacian pyramid method is used to decompose an image fa- into low pass image fa and a high pass image fj with Nj = Nj-1/4, where fj- e L(ZN), fa € L(ZNj) and fj € L(ZN ). After decomposition, we estimate f on a pseudo-polar grid with the one-dimensional band pass filter

based on the signal components, which generate a matrix Dfj. Then, we apply a band-pass filter on matrix Dfb to reconstruct the Cartesian sampled values straightforwardly and also apply the inverse two-dimensional Fast Fourier Transform (FFT) to reconstruct the image9.

3. PCNN model

The components of the simple PCNN neurons form the receptive field, modulation domain and pulse generating domain7,8. The receptive field are given as follows7,8:

Fij [n] = Sij [n]; Lij [n] = e-aLLij [n - 1]+ Vt £ W^Yi [n - 1] (4)

Modulation domain is given as follow:

Uij[n]= Fij[n](1 + p Lij) (5)

and Pulse generating domain are given as follows:

Yij[n] = { 0 oSsc TiM ' Tij["]= e-aTTij[n - 1]+VTYij[n] (6)

where Fij is exterior outputs. Sij, Sij[n] are denote the input stimulus, n denotes the iterative steps and Lij represents the linking input. The parameter p denotes the connecting weight. wijlk denotes the synaptic links. VL, VT represents the magnitude scaling term of threshold potential as a normalization constant. Pij denotes the output pulse of a neuron whose value is either 0 or 1. Tij is the dynamic threshold. aT is a constant. Uij represents the firing map of the image which is related with the inner activity of the neuron. The fused images is obtained by using Uij. If Uij [n] > Tij [n]

LM SDV pcnn pronator estimation by gamma pdf PCNN -> PCNN firing map Combined low-pass

sub-bands based on

LP SDV - pcnn parameter estimation by gamme pdf - PCNN PCNN firing map firing maps 1ST

ST HM

Adaptive

weighted

factor

estimation

Modification of both high-pass sub-bands by adaptive weighted factor

Combined largegctfrem both hlglvpasa modified sub-bands

Fused Image

Fig. 2: Schematic diagram of PCNNST-based fusion algorithm.

4. Singular value decomposition (SVD)

Singular value decomposition (SVD) is a technique which orthogonally decomposes a given matrix into a singular value matrix which contains only a few non-zero values. Usually, SVD of an m x n matrix A is given by5:

A = UxZaVA

where the columns of the m x n matrix, UA are known as left singular vectors, the rows of the n x n matrix VA contain the elements of the right singular vectors, and the diagonal elements of the n x n diagonal matrix £A = diag(oi, o2, •••, on) are known as the singular values 5. Moreover, ai > 0 for 0 < i < q and ai = 0 if (q + 1) < i < n. In notational convenience, (o1 > o2 >•••> oq > 0)5.

5. Fusion schemes for shearlet coefficient

In this work, we have named the proposed fusion scheme as Pulse Code Neural Network in Shearlet Transform (PCNNST). The main steps of the PCNNST scheme are schematically demonstrated in Fig. 2. The role of all implemented processes are discussed as follows:

5.1. Application methodology ofPCNN infusion of low pass sub-bands

In this study, Lpan the high spatial ST PAN image which contains higher coefficient values than LMS image. In order to normalize the coefficient values of LPAN which be compatible with the coefficient values of LMS, the LPAN and LMS are modified by SVD. Applied SVD on both LPAN and LMS separately and determined a ratio parameter. The parameter is modified each singular value of LPAN and LMS. Normalization of ST coefficients of LPAN and LMS are performed as follows:

[Ums, £ms, Vms] = SVDms (Lms) , [Upan, £pan, Vpan] = SVDpan (Lpan)

where £MS, £Pan are singular values of LMS and LPAN respectively. To generate a normalized LPAN matrix from LPAN image, an adaptive factor, $ is computed by forming a ratio with £MS and £PAN and dividing it by a weighted factor W. This is represented as follows:

$ Is max (£s)

w * Ls (£s )

where w = y/( log(r * c)), r and c are row and columns of LPAN. Finally, we regenerate the normalized LPAN by using inverse SVD technique as shown below:

LPAN = VPAN * ( 2 * £PAN ) * UPANILMS = VMS * ((2 * Q*)£MS) * UMS

where LPAN and LMS are the normalized image. In order to combined of both normalized LMS and LPAN, PCNN is processed. In PCNN, receptive field, modulation domain and pulse generating domain for both LMS and LPAN as follows:

The receptive field:

FMS[n] = Sf[n], FlPAN [n] = SPfN [n], Lij [n] = e-aLLjn - 1]+Vt £WijUYa [n - 1] (11)

Modulation domain:

Uij[n] = max jn] (1 + PffD ,fPJan[n] (1 + P^Lj)) (12)

Pulse generating domain:

Yjn] = { 1 j Tj] , TijH = e-aTTij[n - 1]+VTYij[n] (D)

where pMS, PPAN of PCNN model are evaluated by regional statistics of LMS and LPAN. In this study, these parameters are estimated by using maximum likelihood estimation of generalized gamma distribution10. The parameter a is estimated by applying Kullback-Leibler distance (KLD) metric10. In mathematical convenience, the generalized gamma distribution X ~ GGD (a, p) with pdf

p(x; a,p ) = larm exp (14)

where r(■) is the Gamma function. The KLD between two PDFs is defined as:

KLD(p(x;0s) ||p(x;Qt))= i(p(x; ds) log dx (15)

J p (x; 0t)

The parameters of the proposed method were: Initialize internal activity Uij = 0, linking input Lij = 0, threshold Tij = 0. The maximum iterative time was 200 and the linking arrange parameter was 3. Magnitude scaling term of threshold potential VT = a * p, pJfS = ahS/w, pPAN = aPAN/w and Wij = ^--—, where LA is a

F j ' ' PAN/ 'J yT((i-LAx).2 + (j-LAy),2)'

weighted matrix defined by linking arrange parameter. 5.2. Fusion rule for high frequency shearlet coefficient

In PCNNST, a new decision map have been developed to improve the high pass ST sub-bands and an efficient fusion rule which combines improved high pass ST sub-bands automatically. Let Hl'k (x, y) be the high pass ST coefficient at the location (x, y) in the lth sub-band at the kth level and i = PAN, MS. For the current ST sub-band H^, let Qxh be the sum of Hl'k and other horizontal ST sub-bands Hm'k for the level k. It can be estimated by:

Q,h = £ £ H(H[-1,k,H<l'k)U (16)

k=11=1

where || ■ || is the Manhattan distance. Likewise, let Qt,v be the sum of H^ and all other vertical high pass ST sub-bands Hm'n for the different finer levels. It can be evaluated by:

Qi,v = £ £ H(Hl'k,HDH (17)

l,m=1 k,n=1

To determine the parameter Tt,h along horizontal and the parameter Tt,v along vertical high pass ST sub-bands for the present high pass ST sub-band Hl , we perform:

_ Ql,h _ Ql ,v ,.„,

Tlh = V (Ql,h+Qi,v)' Tt'v = V Qh+Qi,v)

Table 1: Quantitative comparison and result analysis in Fig. 3

Method AWLP GIHS PST PCNNST

RASE 20.5847 21.6973 17.7164 11.8230

ERGAS 1.5376 1.1942 1.3149 0.8961

SAM 0.758 0.876 0.618 0.627

Q4 0.9574 0.9589 0.9697 0.9841

qF/AB 0.7528 0.6594 0.6971 0.8706

The parameter t1 ,h is a relationship between Hlk and other neighboring high pass ST sub-bands in the same horizontal plane and is a relationship between H{ for the different vertical planes with its corresponding neighboring ST sub-bands. Then, estimation of the new coefficients H{new using parameters Tl,h and t,v, is given by:

HiL=Hl,k x^J i+Tfh+t2v (19)

Finally, the fused high pass ST sub-band coefficients Hl]k(x, y) is obtained by the following estimation:

HF (x, y) = { ^pAn^X y)l ifHPAN,new > HMkS,new; HMS,new(X y)otherwise (20)

6. Analysis of experimental results

Experimental simulation platform is MATLAB R2012b in the PC with the Intel (R) core (TM) 2 Quad CPU 2.4 GHz and 16 GB RAM. A variety of quality metrics are used for the assessment of the fusion performance quantitatively such as relative average spectral error (RASE)11-12, ERGAS211, spectral angle mapper (SAM)4-11, Q4 index11-12, and Qab/f 12. To evaluate the performance of PCNNST fusion of PAN and MS image, a valid experiment is implemented and the results are compared with four different typical fusion schemes : (1) Average Weighted Laplacian pyramid method (AWLP)1, (2) Generalized Intensity-Hue-Saturation(GlHS) method2, (4) MS+PAN Sharpening Technique (PST)14, respectively. To fuse PAN image with size 512 x 512 and MS image with size 512 x 512 the running time of the proposed technique have take average 5.975m13. The AWLP, and GIHS methods need less than 76.215s. The execution time of the PST method is about 1.535m. Compared with the above mentioned methods, the proposed method is more time consuming but rate of the algorithm can be significantly increased with graphics processing unit (GPU).

The performance results estimated by using RASE, ERGAS, SAM, Q4 index and QAB/F are shown in Table 1 of PCNNST and different fusion methods separately. The highest score in each row of Tables 1is demonstrated in bold. From the table 1, it can be noted that the PCNNST method consistently outperforms the other methods in both evaluation metrics. In addition to objective evaluation, we have also performed a visual comparison on images as shown in Fig. 3. The resulting fused images obtained from the AWLP, GIHS, PST and PCNNST respectively are shown in Fig. 3. Experiments show that PCNNST approach can resolve spectral distortion problems and successfully preserve the spatial information of a PAN image. The results of RASE and ERGAS demonstrate that the proposed method produces the best value in comparison to others. As to the SAM and QAB/F index, the AWLP, PST methods are provide good results, whereas GIHS suffers from spectral distortion. The PST method is offer improved quality of fusion. The proposed method PCNNST provides enhanced multi-spectral fused image with high spatial resolution and color information. Superiority of our method is demonstrated by comparing best SAM, RASE, ERGAS, Q4 and Qab/f index results.

7. Conclusion

In this paper, we present a new remote sensing satellite image fusion technique based on ST and PCNN. In our methodology, we first decompose the source image using ST and then apply the PCNN technique on those decomposed levels. We then select the best ST coefficients to preserve the spectral information. For the high frequency

(a) MS (b) PAN (c) AWLP

(d) GIHS (e) PST (f) PCNNST

Fig. 3: QuickBird MS, PAN image13, and the resulting images using different fusing methods. (a) MS;(b) PAN;(c) AWLP;(d) GIHS;(e) PST; (f)

PCNNST.

sub-band coefficients, a novel process is presented. The objective is to preserve the geometrical details and spectral information of the source images. All comparisons made demonstrate that the proposed method outperforms conventional fusion techniques in terms of both visual quality and quantitative evaluation criterion.

References

1. Wang Z, Ziou D, Armenakis C, Li D, and Li Q. A Comparative Analysis of Image Fusion Methods. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2005;43(6), p.1391-1402.

2. Zhou X, Liu J, Liu S, Cao L, Zhou Q. and Huang H., A GIHS-based spectral preservation fusion method for remote sensing images using edge restored spectral modulation, ISPRS(JPRS) 2014;88, p.16-27.

3. Chen S, Guo Q, Leung H, and Bosse E. A Maximum Likelihood Approach to Joint Image Registration and Fusion, IEEE TRANSACTIONS ON IMAGE PROCESSING 2011;20(5), p.363-1372.

4. Xu M, Chen H, and Varshney PK. An Image Fusion Approach Based on Markov Random Fields, IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2011;49(12), p.5116-5127.

5. Nasir H, Stankovic V, and Marshall S. Singular value decomposition based fusion for super-resolution image reconstruction, Signal Processing: Image Communication 2011.

6. Do MN and Vetterli M. The contourlet transform: An efficient directional multiresolution image representation, IEEE Trans. Image Proc 2005; 14(12), p.2091-2106.

7. Kong W, Zhang L, and Lei Y. Novel fusion method for visible light and infrared images based on NSST-SF-PCNN, Infrared Physics & Technology 2014;65, p.103-112.

8. Xiang T, Yan L, and Gao R. A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain, Infrared Physics & Technology 2015;69, p.53-6

9. Easley G, Labate D, Lim WQ. Sparse directional image representations using the discrete ST, Appl. Comput. Harmon. Anal 2008;25, p.25-46.

10. WangL, Li B, TianL. EGGDD: An explicit dependency model for multi-modal medical image fusion in shift-invariant ST domain, Information Fusion 2013.

11. Zhang Y. Methods for image fusion quality assessment-A review, comparison and analysis, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci 2008;XXXVII(B7), p.1101-1109.

12. Xydeas CS, Petrovic V. Objective image fusion performance measure, Electron; Lett 2000. 36, p. 308-309.

13. http://glcf.umiacs.umd.edu.