2. Estimation of Lighting Environment for Exposing Image Splicing Forgeries

2.2 Lighting Model

B_{α}_{i}_{,β}_{j} ≈

K

X

k=1

γ_{i,j,k}uk (2.3)

whereγi,j,kis the weighting factor for each eigenvectoruk. Therefore, using Equations (2.2) and (2.3),Zcan be approximated as

Z≈X

k

bkuk (2.4)

where

bk =

M

X

i=1 N

X

j=1

w(αi, βj)γi,j,k (2.5)

These basis vectors, being the eigenvectors of the covariance matrix, point to the directions of variation in the set of boundary images. As the boundary images are of the same object captured under different LEs, the dominant variations in these images are because of lighting only. Therefore, these basis vectors capture the lighting variations in the set of boundary images.

The first few eigenvectors form a low-dimensional lighting model. Figure 2.2a shows the first six eigenfaces computed from a set of front pose face images of a male individual. These eigenfaces can be interpreted as faces lit from front, side, top/bottom, extreme side, corner, and extreme corner [64]. This lighting model is used in the later sections to estimate the LEs from any test face image.

2.2.1 Theoretical Analysis of the Low-dimensional Lighting Model

As already defined,Bα,β(θ, φ) denotes the intensity of a single pixel in the boundary image
B_{α,β}. Assume that the surface in the image is Lambertian and convex, the point light source
is at infinity, the camera response is linear, and the surface albedo is uniform. Under these
assumptions, B_{α,β}(θ, φ) becomes proportional to the irradiance due to a point light source at
(α, β).

A low-dimensional lighting model can be constructed by applying the PCA on the set of
boundary images. First, an observation matrixQ, which contains all the observations, is created
by uniformly sampling the light source positions (αp, βp) and the surface normal coordinates
(θi, φi) of the boundary imageB_{α,β}. The matrixQhas the form

2. Estimation of Lighting Environment for Exposing Image Splicing Forgeries

(a)

(b)

Figure 2.1: Examples of images of a single subject under different light sources from (a) Ex- tended Yale B database [4] and (b) Multi-PIE [5] dataset.

Qip =Bαp,βp(θi, φi) (2.6) where i indexes the pixels in the boundary image, and p indexes the light source positions.

To find the principal components oreigenimages, we have to compute the eigensystem of the
covariance matrixT = (Q− µ1)(Q−µ1)^{T}, where 1 is the all-ones matrix and µis the mean
irradiance obtained by averaging over the pixels and all the light sources. Each element ofTis

TH-2553_136102029

2.2 Lighting Model

computed as

Ti j =X

p

(B_{α}_{p}_{,β}_{p}(θi, φi)−µ)(B_{α}_{p}_{,β}_{p}(θj, φj)−µ) (2.7)
Note that µ is same for all the pixels. This is because of the assumption that the surface is
convex and the light sources are uniformly sampled over the entire sphere.

The SH domain representation ofBα,β(θ, φ) is given by [69], [70]

Bα,β(θ, φ)=

∞

X

l=0 l

X

m=−l

AˆlLl,m(α, β)Yl,m(θ, φ) (2.8)

whereY_{l,m}is themth SH of orderl, L_{l,m}is the corresponding SH coefficient of the LE and ˆAl is
the normalisedlth order SH coefficient of the reflectance function. In [69] and [70], the authors
showed that ˆAl decays rapidly for Lambertian surfaces, and 99% energy of the Lambertian
reflection is captured by the first 9 SH coefficients of the LE up tol=2.

Plugging Equation (2.8) in Equation (2.7), we get
Ti j = (( ˆA)^{2}_{0}−4πνµ^{2})Y_{0,0}(θi, φi)Y_{0,0}(θj, φj)+

∞

X

l=0 l

X

m=−l

( ˆAl)^{2}Ylm(θi, φi)Ylm(θj, φj) (2.9)
where ν is the number of images taken with different light source positions. Equation (2.9)
is the analytic form of the elements of the covariance matrix T. The PCA is used for finding
the principal components, which are the eigenvectors ofTcomputed by solving the following
eigensystem:

Tu=λu (2.10)

where λis the eigenvalue. Representing the eigenvector in terms of SHs and substituting the value ofTi j from Equation (2.9), we get

X

p,q

Ml,m:p,qcp,q =λcl,m (2.11)

wherecl,mis the coefficient corresponding tomth SH of orderlof the eigen vectoru, and Ml,m:p,q =X

j

Yl,m(θj, φj)Yp,q(θj, φj) (2.12) Assuming that the light source samples are infinitely dense, the summation operation in

2. Estimation of Lighting Environment for Exposing Image Splicing Forgeries

Equation (2.12) becomes integration over the angular coordinates (θ, φ) given by

M_{l,m:p,q} =Z π
θ=0

Z 2π φ=0

Y_{l,m}(θ, φ)Y_{p,q}(θ, φ) sinθdθdφ (2.13)
Here, the term Ml,m:p,qcaptures the orthogonality between various SHs. When the image pixels
correspond uniformly to the entire sphere of surface normals, the SHs will be orthogonal to
one another. Thus, the eigenvectors will simply be the SHs themselves, and the first 9 eigen-
vectors will capture 99% of irradiance. However, in the case of a single image, where only
the front-facing surface normals are visible, the pixels will be distributed over the upper hemi-
sphere only. While the orthonormality of SHs guarantees that none of their linear combination
can have norm 0, Ramamoorthy [71] showed that the norm of certain linear combinations come
very close to 0 when the domain of integration is restricted to the upper hemisphere. There-
fore, the eigenvectors will be the linear combinations of SHs with the same value of m, i.e.,
m = 0, m = 1 and m = −1. SHs corresponding to m = ±2, i.e., Y4 andY8, are not affected
by this rearrangement and become eigenvectors alone. Because of this intermingling of SHs,
fewer eigenvectors will now capture most of the irradiance. The first six eigenvectors now cap-
ture about 98% of the irradiance, and thereby proving the empirical low-dimensional model
proposed by Hallinan [64].

2.2.2 Computation of Low-dimensional Lighting Model

To create the lighting model, a set of front pose face images of a single individual captured under different point light sources is collected. The face parts are cropped from these images and geometrically aligned so that all the images have identical eye locations. Then, the PCA is applied to this set to get the principal components. The principal components are the eigen- vectors of the covariance matrix of the set of face images represented as vectors. As reported in [64] and [63], we have also observed that the eigenvectors corresponding to the first few significant eigenvalues are sufficient to capture the lighting variations in the image set. Hence, these eigenvectors can be used as a lighting model to estimate the LE from a test face image.

Let F_{1},F_{2}, ...,FZ be Z face images in the image set, each of size M× N. Each face Fi is
rearranged as a vectorIi of dimensionMN. The PCA is applied on this set to find the orthonor-
TH-2553_136102029