Scholarly article on topic 'Face recognition via edge-based Gabor feature representation for plastic surgery-altered images'

Face recognition via edge-based Gabor feature representation for plastic surgery-altered images Academic research paper on "Medical engineering"

CC BY
0
0
Share paper
OECD Field of science
Keywords
{""}

Academic research paper on topic "Face recognition via edge-based Gabor feature representation for plastic surgery-altered images"

O EURASIP Journal on

Advances in Signal Processing

a SpringerOpen Journal

RESEARCH Open Access

Face recognition via edge-based Gabor feature representation for plastic surgery-altered images

Chollette C Chude-Olisah1*, Ghazali Sulong1, Uche A K Chude-Okonkwo2 and Siti Z M Hashim1

Abstract

Plastic surgery procedures on the face introduce skin texture variations between images of the same person (intra-subject), thereby making the task of face recognition more difficult than in normal scenario. Usually, in contemporary face recognition systems, the original gray-level face image is used as input to the Gabor descriptor, which translates to encoding some texture properties of the face image. The texture-encoding process significantly degrades the performance of such systems in the case of plastic surgery due to the presence of surgically induced intra-subject variations. Based on the proposition that the shape of significant facial components such as eyes, nose, eyebrow, and mouth remains unchanged after plastic surgery, this paper employs an edge-based Gabor feature representation approach for the recognition of surgically altered face images. We use the edge information, which is dependent on the shapes of the significant facial components, to address the plastic surgery-induced texture variation problems. To ensure that the significant facial components represent useful edge information with little or no false edges, a simple illumination normalization technique is proposed for preprocessing. Gabor wavelet is applied to the edge image to accentuate on the uniqueness of the significant facial components for discriminating among different subjects. The performance of the proposed method is evaluated on the Georgia Tech (GT) and the Labeled Faces in the Wild (LFW) databases with illumination and expression problems, and the plastic surgery database with texture changes. Results show that the proposed edge-based Gabor feature representation approach is robust against plastic surgery-induced face variations amidst expression and illumination problems and outperforms the existing plastic surgery face recognition methods reported in the literature.

Keywords: Face recognition; Plastic surgery; Illumination normalization; Edge information; Gabor wavelets

1 Introduction

The much attention given to face recognition within the research and commercial community can be associated with its real-world application potentials in areas such as surveillance, homeland security, and border control. Among the most challenging tasks for face recognition in these application scenarios is the development of robust face recognition systems [1]. This implies that apart from recognizing faces under normal scenario, such systems should also be able to successfully handle issues arising from unconstrained conditions. The face recognition under unconstrained conditions results in faces which are termed here and throughout this paper as the unconstrained faces.

* Correspondence: razbygal@yahoo.com

1Faculty of Computing, Universiti Teknologi Malaysia (UTM), Skudai 81300, Malaysia

Fulllist of author information is available at the end of the article

Typically, unconstrained faces include faces that are subject to factors such as changes in illumination direction, pose, expression, and recently introduced variations due to plastic surgery [2]. The problem of pose, expressions, and illumination in face recognition has been addressed in a good number of literatures, some of which are [3-9]. However, there has been scanty literature on the recognition of surgically altered faces. Like changes in illumination direction, plastic surgery procedures induce intra-subject (face image versions of the same person) dissimilarity, which are impediments to robust face recognition. Such problem can be exacerbated when other conditions such as pose and expression are included. The main focus of this paper is to address the recognition problems that arise from conditions where the face is surgically altered.

To solve the problem of face recognition under unconstrained conditions, let us take a quick look at a typical face recognition system as shown in Figure 1. This system

Springer

© 2014 Chude-Olisah et al.; licensee Springer. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Figure 1 Architecture of a typical face recognition system. Dashed rectangles highlight the contribution points of this paper.

mainly consists of the face preprocessing, face representation, and face classification stages. Among these three stages, the face representation stage has been identified as a fundamental component in face recognition that is necessary for minimizing intra-subject variations as well as increasing inter-subject discrimination margin [10-12]. Over the past years, many face representation approaches such as eigenface [13], fisherface [14], Gabor [11], and local binary pattern (LBP) [15] have been introduced. The eigen-face and fisherface are categorized as global approaches, while Gabor [11] and LBP [15] are categorized as local approaches. In a study by Heisele et al. [16], a comparison between the global approaches and the local approaches shows that the local matching methods outperform the global matching methods in accurate face identification. Hence, in the ensuing discussions, emphasis will be on the local approach.

The local approach, LBP, describes a central point pixel by the changes in its neighboring pixels. According to Vu and Caplier [17], the LBP is basically a fine-scale descriptor that mostly captures small texture details. Some of the existing LBP-based descriptors are the monogenic-local binary pattern (M-LBP) [18], local binary pattern histogram Fourier features (LBP-HF) [9], and local Gabor binary pattern histogram sequence (LGBPHS) [8], which utilize image texture properties that LBP encodes. For faces altered by plastic surgery procedures that introduce texture variations on face images, the LBP-based descriptors fall short since they mostly encode texture properties of the face. Hence, the face recognition system that utilizes texture variant descriptors may not be hardy against faces altered by plastic surgery procedures.

On the other hand, Gabor descriptor captures salient visual properties such as spatial localization, orientation, selectivity, and spatial frequency characteristics [9]. Gabor typically encodes facial shape and appearance [17], which makes it robust against factors such as facial expression [4-6], mutilated faces [19], occlusion [7], and pose [3]. Gabor is also good for small sample size problem [11]. However, studies have shown that Gabor features are sensitive to gross changes in illumination direction [12]

and does retain some elements of small-scale textures [20]. Hence, compensating for the influence of illumination changes and texture changes for unconstrained faces is a necessary step towards a robust Gabor feature descriptor for surgically altered images.

Before addressing the illumination problem for the descriptor, one has to take into cognizance the type of face images that are input to the descriptor. Basically, in face recognition tasks, the inputs are the original gray-level (intensity) image. The intensity-based methods encode image texture properties [21]. The robustness of the texture-encoding approach in the recognition of surgically altered faces can be assessed by considering a typical scenario. A typical gross case of plastic surgery is the rhytidectomy, which is a procedure that changes the global appearance of a face. This surgery procedure fundamentally enhances facial skin texture from an aging state to a younger state, hence bringing about change in skin texture. In most cases, rhytidectomy is combined with some surgery procedures such as nose reshaping, eye lift, and jaw enhancement, which change the face appearance (increases intra-subject variation), but might not necessarily change the shape of the facial components. In other words, rhytidectomy is an embodiment of local and global appearance-changing surgery procedures, which may explain the challenges that the existing intensity-based recognition methods in the case of rhyti-dectomy [2] and subsequent works [22-25] faced. In contrast to the original gray-level image, the image edge information is an alternative because only valuable representation of the most significant details of the face is retained, which we presume is a good candidate for the recognition of surgically altered faces. However, as pointed out by Gao and Qi [26], edge information are insensitive to illumination changes but only to a certain extent. The question then is to what extent? The extent to which the edge information is insensitive to illumination is dependent on light distribution across the illuminated object. Hence, if we can eliminate or reduce the effect of the inconsistent lighting of the face, then the edge information will richly retain the shape of significant facial components, which may

minimize intra-subject variations induced by plastic surgery. And to the best of our knowledge, this alternative has not been explored in the recognition of surgically altered face images. Our work considers exploring the shape of the facial components to address the intra-subject dissimilarity problem due to plastic surgery.

In compensating for illumination variations, illumination normalization techniques are often employed. These techniques compensate for illumination variations while retaining image feature shape characteristics [27]. Some of the existing illumination normalization techniques include the histogram equalization (HE) [28], gamma correction (GC) [29], the logarithm transform (LT) [30], and the quotient image techniques [31]. The HE is normally used to make an image have a uniform histogram to produce an optimal global contrast in the image. However, HE may make an image that has uneven illumination turn to be more uneven. The LT works best at shadow regions of a given image [30]. For the quotient image-based techniques, it is known that they are dependent on the albedo (texture) [31]. Since the quotient image-based techniques comprise the ratio of albedo (texture) between a test face and a given face, edge information obtained from such techniques have the likelihood of containing many false edges [32]. The GC corrects the overall brightness of a face image to a pre-defined 'canonical form', which fades away the effect of varying lighting. But, the GC is still affected by some level of directional lighting as pointed out by [33].

We may note at this point that all the above mentioned illumination normalization techniques are used on gray-scale images. However, in recent times, most face images that are acquired and are available for face recognition task are color images. Studies have shown that illumination effect due to changes in light direction can be addressed in the color domain when the source color is known and constant over a scene [34,35]. According to Zickler et al. [34], red, green, and blue (rgb) color space transformations remove the effect of illumination direction without explicit specular/diffuse separation. This claim is also supported by the work of Finlayson et al. [35], where it is highlighted that the image dependencies due to lighting geometry can be removed by normalizing the magnitude of the rgb pixel triplets. Therefore, inspired by these studies, the proposition of illumination normalization steps that take advantage of color domain normalization to improve the performance of edge-based face recognition systems is desired.

1.1 Related works

Face recognition performance in plastic surgery scenarios for cases such as rhytidectomy (face and mid-face lift), rhinoplasty (nose reshaping), blepharoplasty (eye surgery), otoplasty (ear surgery), browlift, dermabrasion, and skin

peeling has been investigated [2,22-24,35-38]. Bhatt et al. [24,25] adopted non-disjoint face granulation approach where the granules are obtained from HE-normalized images. The features were then extracted using the extended uniform circular local binary patterns (EUCLBP) and scale invariant feature transform (SIFT). The performance of their method is significantly impacted by rhytidectomy procedure. In [23], a Gabor patch classifier that uses the rank-order list fused from equal and non-overlapping patches of surgically altered face images for discrimination was proposed. Aggarwal et al. [36] adopted a part-wise sparse representation approach for matching plastic surgery-altered faces. They employed the intensity characteristics of the principal component analysis (PCA)-based representation of the six facial components cropped from each face image. The facial components were then fused to determine the sparse representation error. A match is found if the probe sample produces smallest representation error to a test sample. In [37,38], the multimodal biometrics, such as holistic face information and the periocular region, were adopted. Then, the features were extracted using shape local binary texture and Gabor. In [22], the method of face analysis for commercial entities (FACE) was adopted. The FACE utilizes correlation index obtained from defined subregions between two images. By correcting for illumination problem using self-quotient image, an improved performance was obtained using the FACE method. With particular interest in rhytidectomy, it is worth pointing out that though the recognition results in [2,22] suggest that the algorithms have tried to address the challenges in face recognition in the event of rhytidectomy, there is still a significant scope for further improvement.

1.2 Our contributions and paper organization

In the following, we summarize the main contributions of this paper.

• We propose illumination normalization steps that reduce image dependency on illumination direction and control edge extraction sensitivity to illumination for the purpose of face recognition. The proposed illumination normalization steps are obtained from fusing RGB normalization (rgbN) method and the non-linear pixel power transform method termed GC for color images. We term this proposed steps the rgb-gamma encoding (rgbGE) technique.

• Under the assumption that the shape of the facial components might remain unchanged after surgery, we propose edge-based Gabor face representation for face recognition of surgically altered face images. The shape of significant facial components is retained by extracting gradient magnitude information from the rgbGE-normalized face image so as to minimize the intra-subject variations due to surgery.

• By means of experimental results, we show that the edge-based Gabor face representation approach performs significantly well in a simple nearest neighbor search face recognition framework. And the robustness of the proposed approach is investigated first with typically investigated problems such as pose, expression, and illumination problems and then plastic surgery.

The rest of this paper is organized as follows. In Section 2, the illumination normalization is presented. We present in Section 3 the proposed edge-based Gabor face representation approach. In Section 4, the face recognition experiment using the proposed face representation approach is presented. Finally, conclusions are drawn in Section 5.

2 Illumination normalization

As highlighted earlier, the changes in illumination condition results in false edges with respect to edge information extraction, and this has to be properly addressed. In this section, a brief review of the face reflectance model is firstly provided in such a manner that establishes a coherent basis for presenting the proposed illumination normalization technique. Subsequently, the proposed technique and the related step-by-step procedure for actualizing the technique are presented.

intensity, A is the wavelength of the light, Sc(A) is the spectral reflectance on a surface point c (where c is of spatial coordinates {x, y}). £(A) is the spectral power distribution of the incident light, and Ck(A) is the spectral sensitivity of the sensor. The terms wd and ws are the diffuse and specular terms of the incoming light, respectively. The first part of the right-hand side of (2) is the diffuse component, while the second part is the specular component.

For the color vectors {r, g, b}, (2) can be rewritten as

Wd(c) Dr(c) + Ws (c)Gr ■Jr(c) "

Tk (c) = Wd(c) Dg (c) H h Ws(c)Gg = J g(c)

Wd(c) Db(c) - h Ws(c)Gb _ Jb(c)

Equation 3 describes the intensity components, which comprise diffuse and specular reflections for an RGB image captured in uncontrolled lighting environment. Mathematically, the objective of the proposed normalization technique is to reduce/eliminate the dependency of Ik(c) on the factors ws, G„ Gg, and Gb. To achieve this objective, we employ the merits of RGB and GC normalizations. The RGB normalization will address the directional lighting effect, while the GC normalizes a face image to a predefined 'canonical form', which fades away the effect of illumination.

2.1 Face reflectance model

Light reflections from most surfaces are of two basic types, namely, the diffuse and specular reflections. The diffuse reflection defines the case where the incident light is reflected equally in all directions [39] and is well described by the Lambertian model [40]. The specular reflection for a smooth surface defines the case where the incident light is reflected in a mirror-like direction from the surface [41]. These reflections are often modelled using the Phong reflectance model [42].

To model a typical image captured using RGB camera sensor, we use the dichromatic reflection model described by Shafer [43], which includes the Lambertian term and the Phong's specular term. This model is given by

Ik (c) = Wd(c)y Sc(A) £(A) Ck (A) dA + ws(c)

J £(A) Ck (A) dA, k = r, g, b

= Wd (c) Dk (c) + ws(c)Gk,

(1) (2)

2.2 RGB normalization

The normalized RGB (Nrgb) is used in [35,44]. The Nrgb is expressed by [44]

ßk (c) =

Ik (c)

J r(c)+T g (c)+ Ib(c)!

where j¡k = {/>„ ¿6b} represents each color channel.

The computation of (4) results in the removal of intensity variations from the image, so that the RGB components of the image specify color only, and no luminance. This ensures that the normalized image becomes insensitive to changes in illumination direction.

2.3 Gamma correction

Gamma correction is a non-linear operation generally used to control image overall brightness. It is simply defined by the power law expression with respect to the input image /¡nput and output image Ioutput, as

T — Jr

T output = T input '

where Dk (c) = JSc(A) £(A) Ck (A) dA, Gk = J£(A) Ck

(A) dA , Tk = {Tr, Tg, Tb} is the color vector of image

where y is the exponent of the power function. Usually, the gamma value can be between the range [0,1] and is referred to as the encoding gamma. Gamma encoding, which is a form of non-linear transformation of pixels, enhances the local dynamic range of the images in dark

or shadowed regions while compressing it in bright regions and at highlights [45]. For a given image /(c), the gamma encoding transform is expressed as

G'(c)= 11=Y (c)

In essence, the transformation technique corrects the problem of non-uniform intensity, where too much bits belong to high intensities and too few bits to low intensities.

2.4 Fusion of Nrgb and GC

The proposed illumination normalization technique rgbGE fuses the merits of RGB normalization and GC described above in order to compensate for illumination problem. The steps in the fusion of Nrgb and GC techniques are presented as follows:

• Step 1: Separate the image 4(c) into the respective RGB color channels, Ir(c), Ig(c), and Ib(c).

Step 2: Obtain the square magnitude Im(c) of the separated RGB color channel images in step 1; thus,

Im (c) = V/12(c) + 1 g(c)+ 12(C)

Step 3: Normalize the images Ir(c), Ig(c), and Ib(c) in step 1 by the Im(c) (step 2) as follows:

1r(c) = 1g(c) = 1b(c) =

1 r(c) 1 m(c)

1 g (c)

1 m(c) 1 b(c) 1 m(c)

The essence of this step is to reduce the image intensity variation so that the RGB components of the image specify color only, and no luminance.

Step 4: Compute the gamma encoding transformation of the Im(c) in step 2.

Gm(c)= Im/Y(c), ye[0,1]

Step 5: Multiply the result of step 3 by Gm(c) as shown below:

f ;(c)= /r(c)Gm(c) f g(c)= I g(c)Gm(c) f b(c)= I b(c)Gm(c)

When the Ir(c), Ig(c), and Ib(c) in step 3 are recombined to form a color image, the resultant image is of low contrast. Step 5 is used to restore the contrast level to a level adequate for further processing.

Step 6: By concatenating the expressions in (10) along the third dimension, we obtain the rgbGE-normalized image; thus,

n (c) = fr(c) ®3fg(c) ®3fb(c),

where Wk(c) is the rgbGE-normalized image, and ® 3 symbolizes concatenation along the third dimension.

By this transformation, the illumination problem in the original image is compensated, and edge information obtained from ^k(c) will have little or no false edges. It is important to note that the working principle of the rgbGE is based on the dichromatic reflectance model, which does not consider the ambient component. Hence, the performance of the technique will be

Figure 5 Edge-based Gabor magnitude representation of a sample face image.

significantly affected by outdoor captured images that mostly have ambient lighting components. More also, it should be noted that the variation in the degree of illumination problem across all images in a given database also affects the performance of rgbGE. Basically, the rgbGE performs well only when the degree of illumination variation across all images in a database is insignificant.

The main idea behind the proposed illumination normalization technique is to minimize the intra-subject variation due to illumination as well as skin texture differences. In Figure 2, example images (images of a person) from three datasets are used to illustrate the performance of the proposed illumination normalization technique under varying lighting conditions (but without any ambient component). A number of images for each subject with varying degree of lighting problem are used in the illustration. It can be seen from Figure 2 that the

normalized images of the subjects appear more similar to each other.

3 Edge-based Gabor face representation

As can be observed in image 1 of Figure 2b, the pre-surgery face images (the images on the first two columns) and the post-surgery images (the image on the third column) show a great amount of skin texture changes. Such differences between the images of the same subject are likely to impact on the face recognition accuracy. A plausible solution is to exploit the face information that are not likely to be affected by plastic surgery. Hence, we exploit the shape of the facial components, i.e., the shape of the eyes, nose (nostrils), eyebrow, and mouth that do not change after plastic surgery procedures. We put forward that this frame of reference serves as a platform for constructing robust and efficient feature descriptors for recognizing surgically altered face images. Under these assumptions, we utilize edge information, which are dependent on the shapes of the significant facial components of the face to address the intra-subject variations due to plastic surgery procedures. The basic idea of the proposed edge-based Gabor face representation approach is aimed at mitigating the intra-subject variations induced by plastic surgery procedures. This is achieved via computing the edge gradient magnitude of the illumination-normalized image. Applying Gabor wavelet on the resultant edge gradient magnitude image accentuates on the uniqueness of significant facial components, which enlarges the discrimination margin between different person face images. These processes are discussed below.

3.1 Edge gradient magnitude computation

Let the grayscale version of the illumination-normalized image ^k(c) be denoted as ftc). The edge information g

Figure 6 Overview of the stages taken into consideration for describing a face image. The stages enclosed by the broken line highlight the proposed EGM descriptor.

(c) of the image W(c) is obtained via the computation of the gradient magnitude of the image; thus [46],

g (c) = yW W (c))2 + W (c))2, (12)

where dx W = (W® ®S and dy W = (V® jy) ®S denote partial derivatives, with S as smoothening Sobel filter function.

The false edges in the gradient magnitude image g(c) are substantially reduced when the rgbGE normalization technique is employed. This can be observed in Figure 3a.

In Figure 3, the gradient of the rgbGE normalized face images (three images of a subject) is compared with the original image without correction and with various illumination normalization methods such as LT, HE, and GC. It can be seen from the figure that the gradient of the rgbGE face images shows less facial appearance differences in comparison to the other methods. In subsequent subsections, the Gabor encoding process is given in detail.

3.2 Gabor wavelets

Gabor wavelets (kernels and filters) have proven useful in pattern representation due to their computational properties and biological relevance [3,7,11,19]. It is a powerful tool that provides spatial domain and frequency domain information on an object. The Gabor kernels can be expressed by [47]

^v(c) = e(-IM|2W2M -e-°2'2],

where ^ and v define the orientation and scale of the Gabor kernels, respectively, c = (x, y), II . II denotes the norm operator. The term l^tV is defined as [11]

W = lvef*", (14)

where lv = lmax/sf and = rc^/8. lmax is the maximum frequency, sf is the spacing factor between kernels in the frequency domain [47], and a is a control parameter for the Gaussian function.

Figure 8 Original sample faces from the GT dataset.

The family of self-similar Gabor kernels in (13) is generated from a mother wavelet by selecting different center frequencies (scales) and orientations. In most cases, the Gabor wavelets at five scales v e {0, ..., 4} and eight orientations ^ e {0, ..., 7} are used [11,19]. This paper uses Gabor kernels at five scales and eight orientations with the following parameters: a = 2n, lmax = n/2, sf = \/2 [11,19] as shown in Figure 4. The edge image g(c) is convolved with a family of Gabor kernels at five scales and eight orientations; thus,

0„,v(c)= g(c)* f^c), (15)

where * denotes the convolution operator, and 0^>v(c) is the corresponding convolution result at different scales v and orientations

Applying the convolution theorem, each 0^>v(c) from (15) can be derived via the fast Fourier transform (FFT) [11]:

0„,v(c) = 5-1{ 5 {g (c)}5{ f^c)} }> (16)

where 5 and 5-1 denote the Fourier transform and its inverse, respectively.

The Gabor wavelet representation of the edge image g(c) is shown in Figure 5, where only the magnitude responses

fi 0.96

0 •43

O 0.94

- Gabor with original gray-level faces

- Gabor with gradient magnitude faces

Figure 7 Face recognition performances using Gabor with original gray-level face images and gradient magnitude faces.

oitlgKiPF]

Figure 9 Rhytidectomy plastic surgery sample faces, pre-surgery (top row), post-surgery (bottom row).

of 0^v(c) is used to construct the Gabor feature. Having computed 0^>v(c), the augmented feature vector, namely the edge-based Gabor magnitude (EGM) feature matrix zn is obtained by concatenating each 0^>v(c) already downsampled by a factor p, where p = 64 becomes 0ftv(c) and normalized to zero mean and unit variance. By so doing, the augmented EGM feature matrix z(p) encompasses every possible orientation selectivity, spatial locality, and frequency of the representation result; thus,

0<W* 0<W*07,4,/kJ

where T is the transpose operator, O^vjk are the respective J x K downsampled image matrices with orientation ^ and scale v, and Zrq are the elements of the R x Q EGM feature matrix. The procedure for obtaining the EGM feature is clearly illustrated in Figure 6.

3.3 Dimensionality reduction and discriminant analysis

The EGM features are of high dimensional space, such that Z(p) e R^, where N is the dimensionality of the

vector space. To address the dimensionality problem and still retain the discriminating information for identifying a face, we apply the two-stage (PCA + LDA) approach [14,48]. Each same person face is defined as belonging to a class. Let «1, «2, •", and N1, N1, •••, NL denote the classes and the number of images within each class, respectively. Let M1, M1, •••, ML and M be the mean values of the classes and the grand mean value. The within-class scatter matrix Sa and the between-class scatter matrix Sb are defined as [14,48]

Sa = ¿P(A) £{(r(p)-Mi)(r(p)-m;)7|A} /=1

Sb = ]T P(A-)(Mi-M) (Mi-M)7

(18) (19)

where Y(p) is the most expressive feature of the original data z(p) obtained with a PCA step so that LDA is implemented in the PCA subspace [14]. P(A) is the probability of the ith class, and L denotes the number of classes.

The LDA derives a projection matrix A that maximizes the Fisher's discriminant criterion:

J (A) = argmax

|ASb AT

(A) |ASaAT

The Fisher's discriminant criterion is maximized when A consists of the eigenvectors of the matrix S"xSb [48].

Sr/SbA = AA,

where A and A are the eigenvector and eigenvalue matrices of S^1Sb, respectively. The two-stage (PCA + LDA) dimensionality reduction approach is employed to maximize the between-class variations and minimize the within-class variations of the projected face subspace.

For validation purpose, the face recognition performance of Gabor, i.e., with the original gray-level face images, and the EGM, i.e., with the gradient magnitude face images, are shown in Figure 7.

It can be observed that the use of gradient magnitude image improved the performance of the Gabor descriptor significantly compared to using the original gray-level face images. At this point, it is important to note that the illumination normalization technique that can be used with the EGM include any of the existing normalization techniques discussed in this work. For simplicity, and hence forth, we use the acronym EGM-rgbGE to represent the EGM that employs the rgbGE illumination normalization technique, EGM-HE to represent the EGM that employs the HE illumination normalization technique, and EGM-GC to represent the EGM that employs the GC illumination normalization technique.

4 Face recognition experiment

In this section, the results of the proposed EGM-based face recognition method on a plastic surgery database [2], the Georgia Tech (GT) face database [49], and the Labeled Faces in the Wild (LFW) database [50] are presented. The details of the datasets and the experimental setups for the face recognition experiment are provided.

Figure 11 Recognition rate for EGM-based face recognition method with different illumination normalization methods on GT database.

We show through simulations the result of the proposed method compared with the existing face recognition methods.

4.1 Datasets and experimental setup

4.1.1 Georgia Tech dataset

The Georgia Tech face database [49] contains of 750 color images of 50 subjects, some of which where captured during different sessions. These images comprise variations in illumination direction, scale, pose, and expression; see sample images in Figure 8. The images were manually cropped and resized to size 128 x 128. The database is partitioned into training and testing sets. The number of training and test images is selected to resemble a real-time scenario where only one image per person is tested on a large database in which there exist numerous images of the same person.

4.1.2 Plastic surgery dataset

The plastic surgery dataset [2] contains frontal face images of plastic surgery-altered faces, which vary by scale and small expression, small pose, and majorly by plastic

Table 1 Recognition performance comparisons of the proposed EGM-based face recognition method on Georgia Tech face database

Methods Rank-1 (%) Equal error rate

EGM without normalization 94 2.12

EGM-HE 92 1.8

EGM-GC 92 2.0

EGM-LT 94 2.0

EGM-rgbGE 98 0.34

The EGM-rgbGE is compared with different illumination normalization methods. HE, histogram equalization; GC, gamma correction; LT, logarithm transform; rgbGE, proposed normalization technique.

Table 2 Recognition rate comparisons with some existing methods on GT face database

Methods Rank-1 (%) EER Comments

Majumdar et al. [52] 86.5 n/a n/a

Maturana et al. [53] 92.57 n/a No face alignment

Naseem [54] 92.86 n/a No face alignment

Geng and Jiang [55] 97.43 n/a With face alignment

Li et al. [56] 96.9 n/a With face alignment

Wouter et al. [19] 98.80 n/a With face alignment

Proposed 98.00 0.34% No face alignment

n/a, not applicable.

Table 3 Recognition performance comparisons of the proposed EGM-based face recognition method on LFW Database

Methods Rank-1 (%) Equal error rate (%)

EGM-woc 76.68 16.67

EGM-HE 70.00 16.68

EGM-GC 76.67 16.67

EGM-rgbGE 76.33 16.33

woe, without correction; HE, histogram equalization; GC, gamma correction; rgbGE, proposed normalization technique.

surgery procedures. There are only two images (pre-surgery and post-surgery) per subject available. The dataset consists of different plastic surgery procedures such as rhinoplasty (nose surgery), blepharoplasty (eyelid surgery), brow lift, skin peeling, and rhytidectomy (face and mid-face lift surgery). According to Bhatt et al. [24,25], global surgeries like rhytidectomy severely impacted on their recognition algorithm. To investigate this impact, we used the rhytidectomy plastic surgery dataset, which we arranged to combine full-face lift and mid-face lift surgery images. The rhytidectomy dataset consists of two images per subject. There are 321 subjects; hence, the total number of images is 642. The samples of the rhytidectomy images from the dataset are shown in Figure 9.

Motivated by the fact that a face can be identified from its mirror image [51], we include a mirror version of the pre-surgery images in the database in order to increase the sample size of each subject. Hence, there are a total of 963 face images cropped and resized to size 128 x 128. For this database, we consider two scenarios s1 and s2 in the experimental setup. In s1, out of the 321 subjects, we used 128 subjects for training and performance is evaluated on the remaining subjects, where the pre-surgery images are used as gallery and post-surgery images are used as the probe. In s2, pre-surgery images of all the subjects

are used for training and gallery while the post-surgery images are used as the probe.

4.1.3 Heterogeneous dataset

This database consists of both plastic surgery images and other images that are randomly selected from other databases. The main idea behind the use of this database structure in performance analysis of the EGM is to consider a typical real-world scenario of training-testing where the system is unaware of any plastic surgery cases. In this dataset, the images of 321 subjects with three images per subject are added to the plastic surgery database used above. This brings the total number of subjects to 642, with every subject having three images. For this database, we consider two scenarios s1 and s2 just like in the plastic surgery database above. In s2, the pre-surgery images of all the subjects are used for training and gallery, while the post-surgery images are used as the probe.

4.1.4 Labeled Faces in the Wild (LFW) dataset

The LFW database [50] is a well-known database with gross pose, expression, and illumination problems. Out of all the subjects in the database, we choose subjects with up to three images per subject. Hence, our dataset consists of 869 subjects with images resized to size 128 x 128. For this database, there is only one scenario where 869 images are used as test set, while the training and gallery consist of the remaining images. The LFW sample face images are shown in Figure 10.

4.2 Experimental results

Here, we evaluate the proposed method on all the datasets presented above. In all the experimental setups presented above, the gamma value used in the illumination correction is chosen as y = 0.75, but the value of y can be user defined. We used a simple nearest neighbor classifier with the cosine distance measure in order to measure the similarity between two images. Experimental

Table 4 Recognition rate comparison with existing methods on plastic surgery database: a case of rhytidectomy

Methods Rank-1 (%) EER Comments

Proposed 75.3 to 89.64 7.78% to 3.61% No face alignment

Granular approach [24,25] 71.76 n/a With face alignment

GPROF [23] 86.68 n/a With face alignment

FACE [22] 74.00 17.00 With face alignment

n/a, not applicable.

results are obtained for datasets with illumination correction and without illumination correction. The evaluation is of two categories. In the first category, the proposed method is evaluated on the GT face database and LFW database. The performance of the EGM in the case where the proposed rgbGE illumination normalization technique is used is compared to where GC, LT, and HE illumination normalization techniques are used for the GT. This performance comparison is also extended to the case where GC and HE illumination normalization techniques are used for the LFW. The results of the experiment on the GT face database are shown in Table 1 and Figure 11. A comparison of the recognition accuracy of the proposed EGM-rgbGE with existing methods on GT database is shown in Table 2. The results of the experiment on the LFW database are shown in Table 3 and Figure 12.

For the second category, we conduct an experiment on a rhytidectomy plastic surgery database. In Table 4 and Figure 13, the rank-1 recognition performance of the proposed EGM-rgbGE-based face recognition method on rhytidectomy plastic surgery database is compared to the existing face recognition methods. The recognition performance of the EGM-rgbGE on the heterogeneous database is shown in Figure 14.

From the results in Figure 11 and Table 1, it can be observed that the EGM-LT did not improve on the rank-1 recognition rate (RR) of EGM without correction. This shows that it does not address the presence of false

edges induced by illumination as is evident from Figure 3. However, it improved in terms of equal error rate (EER). On the other hand, the RR performances of EGM-HE and EGM-GC degraded in comparison with EGM without correction. In this case, such degradation may be associated with the introduction of more false edges to the edge information. The EGM-rgbGE-based face recognition method is observed to be more insensitive to performance degradation factors such as illumination, pose, and expression with 98% RR and 0.34% EER.

In Table 2, the performance of the proposed EGM-based face recognition method compared with some existing face recognition methods for the GT database is shown. The performance of the face recognition methods considered are sorted by their rank-1 recognition rates. The performance of EGM-rgbGE in terms of RR with no face alignment is significantly higher than those of all other methods with no face alignment. For the methods with face alignment, the EGM-rgbGE performed better except for the method by Wouter and Peter [19]; of course with face alignment, the EGM-rgbGE will perform better. In the case of LFW, the performance of the proposed method performs equally well as the existing results in the literature. The reader is referred to [57] for results on LFW database. We note at this point that the goal of the current paper is to demonstrate the efficiency of the proposed descriptor based face recognition method, not to compete in the LFW challenge.

In Figure 13, the proposed EGM-rgbGE method performs better in the s2 scenario than in s1 scenario. This performance difference may not be unconnected with the method of training and evaluation. In both s1 and s2 scenarios, the recognition rates are better when rgbGE normalization is used than when it is not incorporated into the EGM method. Table 4 shows the performance comparison of the EGM-rgbGE method with the existing methods on rhytidectomy plastic surgery database. The EGM-rgbGE rank-1 recognition rate of over 89% is a difference of 17.48% in comparison to granular approach

Figure 13 EGM recognition performance for plastic surgery database. The experimentalscenarios s1 and s2 are observed on the basis of with and without rgbGE illumination normalization technique.

--- 1- -

A ___A -♦"Wc/EGM-rgbGE-sl —wc/EGM-rgbGE-s2 woc/EGM-sl -b-woc/EGM

Figure 14 EGM recognition performance for heterogeneous database. The experimentalscenarios s1 and s2 are observed on the basis of with and without rgbGE illumination normalization technique.

[24,25], 15.24% in comparison to FACE [22], and 2.56% in comparison to GROF [23]. However, it should be noted that face alignment was employed to obtain the result in the GPROF method, while the EGM-rgbGE result is without face alignment. The RR and EER results of the different experimental setups and scenarios for the plastic surgery database are shown in Table 5.

The result of the experiment on the heterogeneous database (a combination of the plastic surgery-altered images and non-surgery images) for training-testing, whereby the face recognition system is set to resemble a real-world scenario, unaware of any plastic surgery image, is shown in Figure 14. The experimental results in the two scenarios show how the proposed method, which considers the shape of facial components, performs across surgery and non-surgery images. Again, the proposed EGM-rgbGE method performs better in scenarios si and s2 in comparison with other existing methods. In both si and s2 scenarios, the recognition rates are better when the rgbGE normalization is used than when it is not incorporated into the EGM method. As can be observed, the result of the experiments in the case of heterogeneous database outperforms that of the experiments in the case of only-plastic-surgery database. This performance variation is expected since the average performance will be expectedly higher due to the inclusion of non-plastic surgery images in the database with lesser recognition challenges than purely plastic surgery

Table 5 Recognition performance of the proposed EGM-based face recognition method for the plastic surgery database

Experimental setup Scenario RR (%) EER (%)

wc/EGM-rgbGE s1 75.38 7.78

s2 89.64 3.61

woc/EGM-rgbGE s1 69.23 10.79

s2 87.23 3.73

we, with correction; woe, without correction.

database. The RR and EER results of the different experimental setups and scenarios for the heterogeneous database are shown in Table 6.

5 Conclusion

We have presented the proposed edge-based Gabor feature representation approach for appearance representation of faces altered by plastic surgery procedures. The proposed face representation approach exploited the shape of significant facial components to address the intra-subject dissimilarity problem due to plastic surgery procedures. The face recognition experiment using the proposed face representation approach considered other unconstrained conditions, which included usual pose, illumination, and expression problems. Comparative experimental results on the GT and LFW databases were provided to show the robustness of the representation approach under usually experimented-on problems and then on the rhytidectomy plastic surgery database. The results indicate that the proposed face representation approach in a simple nearest neighbor search face recognition framework (with no face alignment) performed better than the previously reported methods in the literature on rhytidectomy plastic surgery database. In relation to the GT database, the proposed method performed better than the existing methods, except in comparison with one particular method, in which prior face alignment on the database was carried out, and 0.88% performance

Table 6 RR and EER performance comparisons of the proposed EGM-based face recognition method for the heterogeneous database

Experimental setup Scenario RR (%) EER (%)

wc/EGM-rgbGE s1 81.16 4.31

s2 93.93 0.91

woc/EGM-rgbGE s1 80.19 4.95

s2 90.97 1.72

wc, with correction; woc, without correction.

difference was reported. However, with face alignment on the database, it is believed that the proposed method will greatly outperform all methods reported so far in the literature. On the LFW database, our method provides up to 76.33% recognition rate. In future work, we will integrate texture insensitive and illumination invariant processes into a single processing step and investigate its performance with other plastic surgery procedures and cases of gross illumination problems.

Competing interests

The authors declare that they have no competing interests. Acknowledgements

The authors thank the Universiti Teknologi Malaysia for the International DoctoralFellowship (IDF) scholarship.

Author details

1Faculty of Computing, Universiti Teknologi Malaysia (UTM), Skudai 81300, Malaysia. 2Wireless Communication Center, Universiti Teknologi Malaysia (UTM), Skudai 81300, Malaysia.

Received: 30 July 2013 Accepted: 21 May 2014 Published: 5 July 2014

References

1. W Zhao, R Chellappa, PJ Phillips, A Rosenfeld, Face recognition: a literature survey. ACM Comput Surv 35(4), 399-458 (2003)

2. R Singh, M Vatsa, H Bhatt, S Bharadwaj, A Noore, S Nooreyezdan, Plastic surgery: a new dimension to face recognition. IEEE Trans Inf Technol Biomed 5, 441-448 (2010)

3. Z Zheng, F Yang, W Tan, J Jia, J Yang, Gabor feature-based face recognition using supervised locality preserving projection. Signal Processing 87, 2473-2483 (2007)

4. BA Olshausen, DJ Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607-609 (1996)

5. R Rao, D Ballard, An active vision architecture based on iconic representations. Artif Intell 78, 461-505 (1995)

6. B Schiele, JL Crowley, Recognition without correspondence using multidimensional receptive field histograms. Int J Comput Vis 36, 31-52 (1996)

7. B Kepenekci, FB Tek, GB Akar, Occluded face recognition based on Gabor wavelets. Proc ICIP 1, 293-296 (2002)

8. W Zhang, S Shan, W Gao, X Chen, H Zhang, Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. Proc ICCV 1, 786-791 (2005)

9. T Ahonen, J Matas, C He, M Pietikainen, Lecture notes in computer science: rotation invariant image description with local binary pattern histogram Fourier features, in Image Analysis, ed. by A-B Salberg, JY Hardeberg,

R Jenssen (Springer, Berlin, 2009). vol. 5575, pp. 61-70

10. NS Vu, A Caplier, Face recognition with patterns of oriented edge magnitudes. Proc ECCV 6311, 313-326 (2010)

11. C Liu, H Wechsler, Gabor feature based classification using the enhanced Fisher linear discriminant model for face recognition. IEEE Trans Image Process 11 (4), 467-476 (2002)

12. Y Adini, Y Moses, S Ullman, Face recognition: the problem of compensating for changes in illumination direction. IEEE Trans Pattern Anal Machine Intell 19, 721-732 (1997)

13. M Turk, A Pentland, Eigenfaces for recognition. J Cogn Nuerosci 3, 71-86 (1991)

14. P Belhumeur, J Hespanha, D Kriegman, Eigenfaces vs fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell

19, 711-720 (1997)

15. T Ahonen, A Hadid, M Pietikainen, Face recognition with local binary patterns. Proc ECCV 3021, 469-481 (2004)

16. B Heisele, P Ho, J Wu, T Poggio, Face recognition: component-based versus global approaches. Comput Vis Image Understanding 91, 6-12 (2003)

17. NS Vu, A Caplier, Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans on Image Processing 2, 1352-1365 (2011)

18. L Zhang, L Zhang, Z Guo, D Zhang, Monogenic-LBP: a new approach for rotation invariant texture classification, in Paper presented at the 17th IEEE international conference on image processing (ICIP) (Hong Kong), pp. 2677-2680. 26-29 Sept 2010

19. JR Wouter, HN Peter, Robust face recognition algorithm for the identification of disaster victims (SPIE, Burlingame, CA, USA). 03 Feb 2013, vol. 8655. doi:10.1117/12.2001634

20. BS Manjunath, WY Ma, Texture feature for browing and retrieval of image data. IEEE Trans Pattern Anal Machine Intell 12, 55-73 (1996)

21. C Liu, H Wechsler, An integrated shape and intensity coding scheme for face recognition. Proc IJCNN 5, 3300-3304 (1999)

22. M Marsico, M Nappi, D Riccio, H Wechsler, Lecture notes in computer science: robust face recognition after plastic surgery using local region analysis, in Image Analysis and Recognition, ed. by M Kamel, A Campilho (Springer, Berlin, 2011), pp. 191-200. vol. 6754

23. X Liu, S Shiguang, C Xilin, Lecture notes in computer science: face recognition after plastic surgery: a comprehensive study, in Computer Vision, ed. by KM Lee, Y Matsushita, JM Rehg, Z Hu (Springer, Berlin, 2013), pp. 565-576. vol. 7725

24. H Bhatt, R Singh, M Vatsa, Evolutionary granular approach for recognizing faces altered due to plastic surgery, in Paper presented at the IEEE international conference on automatic face and gesture recognition and workshops (AFGR) (Santa Barbara, CA, USA), pp. 720-725. 21-25 March 2011

25. H Bhatt, S Baradwaj, R Singh, M Vatsa, Recognizing surgically altered face images using multi-objective evolutionary algorithm. IEEE Trans Inf Forensics Security 8, 89-100 (2013)

26. Y Gao, Y Qi, Robust visual similarity retrieval in single model face databases. Pattern Recognition 39, 1009-1020 (2005)

27. W Chen, M Er, S Wu, Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. IEEE Trans System, Man, Cybern B 36, 458-466 (2006)

28. SM Pizer, EP Amburn, Adaptive histogram equalization and its variations. Comput Vis Graph, Image Process 39, 355-368 (1987)

29. S Shan, W Gao, B Cao, D Zhao, Illumination normalization for robust face recognition against varying lighting conditions, in Paper presented at the IEEE international workshop on analysis and modeling of faces and gestures (AMFG) (Nice, France, 17 Oct 2003), pp. 157-164

30. S Marios, BK Kumar, Lecture notes in computer science: illumination normalization using logarithm transforms for face authentication, in Audio- and Video-Based Biometric Person Authentication, ed. by J Kittler, MS Nixon (Springer, Berlin, 2003), pp. 549-556. vol. 2688

31. X Xie, WS Zheng, J Lai, C Yuen, Y Suen, Normalization of face illumination based on large- and small-scale features. IEEE Trans Image Processing 20, 1807-1821 (2011)

32. O Arandjelovic, Gradient edge map features for frontal face recognition under extreme illumination conditions (Paper presented at BMVC, Surrey, England), pp. 1-11.3-7 Sept 2012

33. R Al-Osaimi, M Bennamoun, A Mian, Illumination normalization for color face images. Proc ISCV 4291, 90-101 (2006)

34. T Zickler, P Mallick, J Kriegman, N Belhumeur, Color subspaces as photometric invariants. Int J Comput Vision 79,13-30 (2008)

35. G Finlayson, B Schiele, J Crowley, Comprehensive colour normalization. Proc ECCV 1,475-490 (1998)

36. G Aggarwal, S Biswas, JP Flynn, KW Bowyer, A sparse representation approach to face matching across plastic surgery, in Paper presented at the IEEE conference on applications of computer vision, ed. by (Breckenridge, CO, USA), pp. 113-119. 9-11 Jan 2012

37. NS Lakshmiprabha, J Bhattacharya, S Majumder, Face recognition using multimodal biometric features, in Paper presented at the IEEE conference on image information processing, ed. by (Himachal, Pradesh). 3-5 Nov 2011

38. NS Lakshmiprabha, J Bhattacharya, S Majumder, A novel face recognition approach using multimodal feature vector. Int J Adv Eng Tech 3(1), 477-486 (2012)

39. L Cook, K Torrance, A reflectance model for computer graphics. ACM T Graphic 1, 7-24 (1982)

40. S Mallick, T Zickler, Lecture notes in computer science: specularity removal in images and videos: a PDE approach, in Computer Vision, ed. by A Leonardis, H Bischof, A Pinz (Springer, Berlin, 2006), pp. 550-563. vol. 3951

41. K Nayar, K Ikeuchi, T Kanade, Surface reflection: physical and geometrical perspectives. IEEE Trans Pattern Anal Mach Intell 13,611 -634 (1991)

42. BT Phong, Illumination for computer generated pictures. Comm ACM 18, 31 1-317 (1975)

43. S Shafer, Using color to separate reflection components. Color Res Appl 10, 210-218 (1985)

44. R Tan, R Ikeuchi, Separating reflection components of textured surfaces using a single image. IEEE Trans Pattern Anal Machine Intell 27, 178-1935 (2005)

45. X Tan, B Triggs, Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6), 1635-1650 (2010)

46. J Zhu, W Zheng, L Lai, Lecture notes in computer science: complete gradient face: a novel illumination invariant descriptor, in Biometric Recognition, ed. by J Lai, Y Wang, X Chen, PC Yuen, W-S Zheng, Z Su (Springer, Berlin, 2012), pp. 17-24. vol. 7701

47. M Lades, J Vorbruggen, J Buhmann, L Lange, C Malsburg, RP Wurtz, W Konen, Distortion invariant object recognition in the dynamic link architecture. IEEE Trans Comput 42, 300-311 (1993)

48. D Swets, J Weng, Using discriminant eigenfeatures for image retrieval. IEEE Trans Pattern Anal Machine Intell 18, 831-836 (1996)

49. Georgia Tech Face Database. (2007). http://www.anefian.com/research/ face_reco.htm. Accessed 2 Feb 2013

50. G Huang, T Manu Ramesh, E Learned-Miller, Labeled Faces in the Wild: a database for studying face recognition in unconstrained environments, Technical report (University of Massachusetts, Amherst, 2007)

51. K Etemad, R Chellappa, Discriminant analysis for recognition of human face images. JOSA A 14(8), 1724-1733 (1997)

52. A Majumdar, A Bhattacharya, Face recognition by multi-resolution curvelet transform on bit quantized facial images, in Paper presented at the international conference on computational intelligence and multimedia applications (ICCIMA) (Sivakasi, Tamil Nadu), pp. 209-213. 13-15 Dec 2007

53. D Maturana, D Mery, A Soto, Face recognition with local binary patterns, spatial pyramid histograms and naive Bayes nearest neighbor classification, in Paper presented at the international conference of the Chilean Computer Science Technology (SCCC) (Santiago, Chile), pp. 125-132. 10-12 Nov 2009

54. I Naseem, R Togneri, M Bennamoun, Linear regression for face recognition. IEEE Trans Pattern Anal Machine Intell 32, 2106-2112 (2010)

55. C Geng, X Jiang, Fully automatic face recognition framework based on local and global features. Mach Vision Appl 24, 537-549 (2013)

56. B Li, W Liu, S An, A Krishna, T Xu, Face recognition using various scales of discriminant color space transform. Neurocomputing 94, 68-76 (2012)

57. LFW, Results. (1999). http://vis-www.cs.umass.edu/lfw/results.html. Accessed 17 Jan 2014

doi:10.1186/1687-6180-2014-102

Cite this article as: Chude-Olisah et al.: Face recognition via edge-based Gabor feature representation for plastic surgery-altered images. EURASIP Journal on Advances in Signal Processing 2014 2014:102.

Submit your manuscript to a SpringerOpen journal and benefit from:

7 Convenient online submission 7 Rigorous peer review 7 Immediate publication on acceptance 7 Open access: articles freely available online 7 High visibility within the field 7 Retaining the copyright to your article

Submit your next manuscript at 7 springeropen.com