2. Estimation of Lighting Environment for Exposing Image Splicing Forgeries
2.2 Lighting Model
Bαi,βj ≈
K
X
k=1
γi,j,kuk (2.3)
whereγi,j,kis the weighting factor for each eigenvectoruk. Therefore, using Equations (2.2) and (2.3),Zcan be approximated as
Z≈X
k
bkuk (2.4)
where
bk =
M
X
i=1 N
X
j=1
w(αi, βj)γi,j,k (2.5)
These basis vectors, being the eigenvectors of the covariance matrix, point to the directions of variation in the set of boundary images. As the boundary images are of the same object captured under different LEs, the dominant variations in these images are because of lighting only. Therefore, these basis vectors capture the lighting variations in the set of boundary images.
The first few eigenvectors form a low-dimensional lighting model. Figure 2.2a shows the first six eigenfaces computed from a set of front pose face images of a male individual. These eigenfaces can be interpreted as faces lit from front, side, top/bottom, extreme side, corner, and extreme corner [64]. This lighting model is used in the later sections to estimate the LEs from any test face image.
2.2.1 Theoretical Analysis of the Low-dimensional Lighting Model
As already defined,Bα,β(θ, φ) denotes the intensity of a single pixel in the boundary image Bα,β. Assume that the surface in the image is Lambertian and convex, the point light source is at infinity, the camera response is linear, and the surface albedo is uniform. Under these assumptions, Bα,β(θ, φ) becomes proportional to the irradiance due to a point light source at (α, β).
A low-dimensional lighting model can be constructed by applying the PCA on the set of boundary images. First, an observation matrixQ, which contains all the observations, is created by uniformly sampling the light source positions (αp, βp) and the surface normal coordinates (θi, φi) of the boundary imageBα,β. The matrixQhas the form
2. Estimation of Lighting Environment for Exposing Image Splicing Forgeries
(a)
(b)
Figure 2.1: Examples of images of a single subject under different light sources from (a) Ex- tended Yale B database [4] and (b) Multi-PIE [5] dataset.
Qip =Bαp,βp(θi, φi) (2.6) where i indexes the pixels in the boundary image, and p indexes the light source positions.
To find the principal components oreigenimages, we have to compute the eigensystem of the covariance matrixT = (Q− µ1)(Q−µ1)T, where 1 is the all-ones matrix and µis the mean irradiance obtained by averaging over the pixels and all the light sources. Each element ofTis
TH-2553_136102029
2.2 Lighting Model
computed as
Ti j =X
p
(Bαp,βp(θi, φi)−µ)(Bαp,βp(θj, φj)−µ) (2.7) Note that µ is same for all the pixels. This is because of the assumption that the surface is convex and the light sources are uniformly sampled over the entire sphere.
The SH domain representation ofBα,β(θ, φ) is given by [69], [70]
Bα,β(θ, φ)=
∞
X
l=0 l
X
m=−l
AˆlLl,m(α, β)Yl,m(θ, φ) (2.8)
whereYl,mis themth SH of orderl, Ll,mis the corresponding SH coefficient of the LE and ˆAl is the normalisedlth order SH coefficient of the reflectance function. In [69] and [70], the authors showed that ˆAl decays rapidly for Lambertian surfaces, and 99% energy of the Lambertian reflection is captured by the first 9 SH coefficients of the LE up tol=2.
Plugging Equation (2.8) in Equation (2.7), we get Ti j = (( ˆA)20−4πνµ2)Y0,0(θi, φi)Y0,0(θj, φj)+
∞
X
l=0 l
X
m=−l
( ˆAl)2Ylm(θi, φi)Ylm(θj, φj) (2.9) where ν is the number of images taken with different light source positions. Equation (2.9) is the analytic form of the elements of the covariance matrix T. The PCA is used for finding the principal components, which are the eigenvectors ofTcomputed by solving the following eigensystem:
Tu=λu (2.10)
where λis the eigenvalue. Representing the eigenvector in terms of SHs and substituting the value ofTi j from Equation (2.9), we get
X
p,q
Ml,m:p,qcp,q =λcl,m (2.11)
wherecl,mis the coefficient corresponding tomth SH of orderlof the eigen vectoru, and Ml,m:p,q =X
j
Yl,m(θj, φj)Yp,q(θj, φj) (2.12) Assuming that the light source samples are infinitely dense, the summation operation in
2. Estimation of Lighting Environment for Exposing Image Splicing Forgeries
Equation (2.12) becomes integration over the angular coordinates (θ, φ) given by
Ml,m:p,q =Z π θ=0
Z 2π φ=0
Yl,m(θ, φ)Yp,q(θ, φ) sinθdθdφ (2.13) Here, the term Ml,m:p,qcaptures the orthogonality between various SHs. When the image pixels correspond uniformly to the entire sphere of surface normals, the SHs will be orthogonal to one another. Thus, the eigenvectors will simply be the SHs themselves, and the first 9 eigen- vectors will capture 99% of irradiance. However, in the case of a single image, where only the front-facing surface normals are visible, the pixels will be distributed over the upper hemi- sphere only. While the orthonormality of SHs guarantees that none of their linear combination can have norm 0, Ramamoorthy [71] showed that the norm of certain linear combinations come very close to 0 when the domain of integration is restricted to the upper hemisphere. There- fore, the eigenvectors will be the linear combinations of SHs with the same value of m, i.e., m = 0, m = 1 and m = −1. SHs corresponding to m = ±2, i.e., Y4 andY8, are not affected by this rearrangement and become eigenvectors alone. Because of this intermingling of SHs, fewer eigenvectors will now capture most of the irradiance. The first six eigenvectors now cap- ture about 98% of the irradiance, and thereby proving the empirical low-dimensional model proposed by Hallinan [64].
2.2.2 Computation of Low-dimensional Lighting Model
To create the lighting model, a set of front pose face images of a single individual captured under different point light sources is collected. The face parts are cropped from these images and geometrically aligned so that all the images have identical eye locations. Then, the PCA is applied to this set to get the principal components. The principal components are the eigen- vectors of the covariance matrix of the set of face images represented as vectors. As reported in [64] and [63], we have also observed that the eigenvectors corresponding to the first few significant eigenvalues are sufficient to capture the lighting variations in the image set. Hence, these eigenvectors can be used as a lighting model to estimate the LE from a test face image.
Let F1,F2, ...,FZ be Z face images in the image set, each of size M× N. Each face Fi is rearranged as a vectorIi of dimensionMN. The PCA is applied on this set to find the orthonor- TH-2553_136102029