Figure (d) shows a fake image, while (e) and (f) are the DPHs of the two people, and the correlation between the two DPHs is 0.73. Figure (d) shows a falsified image of the same dataset, while (e) and (f) are the DPHs of the two people, and the correlation between the two DPHs is 0.39.
- Splicing Forgery
- Copy-Move Forgery
- Retouching Forgery
- Content-Removal Forgery
A famous example of fake retouching is shown in Figure 1.3a), where the photo of former American football star O. J. Simpson was darkened and the lighting was modified by Time magazine to give him a more menacing appearance. Yezhovm, who is seen standing to the left of Stalin in the authentic image shown in Figure 1.4a), is removed in the forged image shown in Figure 1.4b) and the missing pixels are filled from the surrounding pixels using image painting techniques.
- JPEG Compression-based Methods
- Noise-based Methods
- Lighting Environment-based Methods
- Illumination Colour-based Methods
- Resampling-based Methods
- Camera sensor-based Methods
- Deep learning-based data-driven methods
In , the misalignment of the 8×8 block grid is used to locate splicing and copy-move counterfeits. In , the resampled images are detected by computing the Radon transform  of the derivative of image pixels.
Research Motivation and Problem Statement
Therefore, methods based on lighting environment and lighting colors are more reliable in detecting perfectly curated realistic-looking fusion forgeries. Therefore, there is scope for research in the lighting environment and image forensics based on lighting colors.
Research Contributions and Thesis Organization
Experimental results on two standard splicing data sets, DSO-1 and DSI-1, demonstrate the effectiveness of the proposed method. The experimental results show the effectiveness of the proposed method compared to the prior art.
Related Work and Research Gap
Low-dimensional lighting subspace
More importantly, they proved that the Lambertian reflection acts as a low-pass filter, and the first 9 SH coefficients are sufficient to capture 99% of the irradiance. Later, Ramamoorthi  provided a theoretical connection between the SH subspace and the eigensubspace through the analytic PCA construction, thus proving that the first 5–6 eigenvectors are sufficient to capture 98% of the illumination variations in the face.
Theoretical Analysis of the Low-dimensional Lighting Model
First, an observation matrixQ, containing all the observations, is created by uniformly sampling the light source positions (αp, βp) and the surface normal coordinates (θi, φi) of the boundary imageBα,β. Because of this mixing of SHs, fewer eigenvectors will now capture most of the radiation.
Computation of Low-dimensional Lighting Model
When the image pixels correspond uniformly to the entire sphere of surface normals, the SHs will be orthogonal to each other. Thus, the eigenvectors will simply be the SHs themselves, and the first 9 eigenvectors will capture 99% of the irradiance.
Estimation of Lighting Environment
Using these Leigenfaces, we create the low-dimensional illumination model WL as shown in equation (2.16). A face imageFaf dimension M×N Output: LC vectorΩ. i) Retrieve the low-dimensional lighting modelWL from Algorithm 2.1. ii) ConvertF to a grayscale image of dimension M×And rearrange as an MN-dimensional vectorI. iii) Projection onto the L-dimensional subspace using equation (2.17) to obtain the LC vectorΩ.
If the two levels Fi and Fj come from two different LEs, the distance between Ωi and Ωj will be large. Therefore, the image will be considered split if the maximum distance between all the pairs is more than a predefined threshold.
Experimental Results and Analysis
- Lighting Model Computation
- Lighting Environment Estimation
- Classification of consistent and inconsistent LEs
- Performance on non-frontal face images
- Performance in Splicing Detection
The ROC curves of the three methods on the Yale B and Multi-PIE datasets are shown in Figure 2.8a and 2.8b, respectively. We use the illumination model, created using the front pose face images (ie, the six separate faces shown in Figure 2.2a).
On the other hand, all the pairwise distances in the authentic image (Figure 2.13(b)) are below the threshold value, as can be seen in Table 2.6. The maximum distance d for the authentic image was found to be 0.10, which is less than rth.
This method estimates the luminance color from different parts of an image using DRM . Riess and Angelopoulou  proposed to create a new image, called luminance map (IM), using the luminance colors estimated from the input image.
Dichromatic Reflection Model (DRM)
Similar to , also here, the features are classified in a pairwise manner by concatenating similar features of the two face IMs calculated from the same type of IMs, converted to the same color space. According to tone-neutral interface reflection, the spectral power density of the interface reflection is the same as that of the illumination source.
Dichromatic Plane Histogram (DPH)
Under uniform illumination, the dichromatic planes of two different colored surfaces in the same scene intersect at the illumination color. This is because the illumination color is common to both the dichromatic levels estimated from the two surfaces.
To calculate the DPH from all the faces present in the image, as in Chapter 2, the faces are manually cropped. In the case of a bona fide image, the HDs of all faces will be similar since they come from the same lighting conditions.
Analysis of Some Famous Forged Images
The DPHs calculated from the two individuals in the authentic image are almost equal, as shown in Figure 3-9. On the other hand, the DPHs calculated from the two persons in the falsified image are different from each other, as shown in Figure 3-10.
The method proposed in Chapter 3 extracts an illumination color-related histogram-based feature from the facial parts to detect the split forgery. Once trained, the CNN part of the siamese network is used to extract features from the face IMs.
Related Work and Research Gap
Illumination Colour Estimation
With the GGE hypothesis, the light color was estimated as the pth Minkowski norm of the derivative of the image pixels. The body reflection component contains the object color information, while the interface reflection contains the illumination color information.
Illumination Map (IM)-based Image Forensics
Recently, Pomariet al. proposed a deep learning-based method that removes the need to extract handcrafted features from the IMs. Based on the above literature survey, the following open problem is identified: The existing IM-based methods either extract handcrafted features or deep features using a pre-trained network from the IMs.
Overview of the Method
On the other hand, if they come from two different images, their IMs have different functions. We consider a pair of face IMs to be a genuine pair if they come from the same lighting source (i.e. the same image) and a matched pair if they come from two different lighting sources.
IM Computation and Face Extraction
Also, the automated methods often include non-face areas in the detected face frame. For these reasons, in the proposed method, we manually select the bounding boxes around the faces in the images.
Face-IM Pair Classification for Splicing Detection
- Siamese Network for Feature Learning
- Network Architecture
- Feature Extraction
- Splicing Detection using an SVM
The number of superpixels in face regions also decreases with reduced input size. Although the Siamese network learns to distinguish between authentic and artificially generated conjoined face pairs present in the training set, the difference in concatenated artificial face-IM pairs is more compared to conjoined face pairs present in images united in real life. .
Experiments and Results
Training the siamese network
The network trained on GGE face IMs achieved a classification accuracy of 95.31% on the GGE face IM pairs of the test set. On the IIC face-IM pairs of the test set, the network trained on IIC face IMs achieved an accuracy of 97.44%.
- Performance on DSO-1 dataset
- Performance on DSI-1 dataset
- Comparison to the state-of-the-arts
- Robustness to JPEG compression
We then test the performance of features extracted from GGE-IM, IIC-IM, and raw face images using CNNs trained on IIC-IM. We have performed a series of experiments to see the robustness of the proposed method for JPEG compression.
Chenet al. proposed the first DL-based method for image processing operation detection. The image editing software contains a large number of image editing tools such as Adobe Photoshop and GIMP.
Proposed Manipulation Detection Method
- Siamese Convolutional Neural Network for Image Editing Operation
- Network Architecture
- Distance Layer
- Manipulation detection using One-shot Classification
If the image is classified as manipulated, we check if it has been manipulated using one of the editing operations present in the test gallery. We want to check whether the editing in the gallery is applied to the test image patch or not.
Forgery Localization and Detection
Therefore, we first establish the class of the reference patch, i.e., whether it is authentic or counterfeit, and then detect counterfeit patches accordingly. After the reference patch class is set, counterfeit patches are detected according to the following two cases:
Manipulation Detection Results
- Discrimination of Different Image Editing Operations
- Detection of different Image Editing Operations
- Generalization to Unseen Manipulations
- Dependence of Generalization Accuracy on Number of Train-
- Detection of Unknown Manipulations
In the first experiment, we test the ability of the proposed method to detect/distinguish different image processing operations. This shows the generalization ability of the proposed method to unknown types of image editing operations.
Selection of Hyper-parameters of the Proposed CNN
The method achieves the average accuracy of 98.12% and 99.40% using the fixed SRM filters and the constrained convolution filters, respectively. In the case of dual tamper detection, the method achieves the average accuracy of 92.01% and 93.64% using the fixed SRM filters and the constrained convolution filters, respectively.
Forgery Localization and Detection Results
We have calculated F1 score and Matthew's correlation coefficient (MCC)  to quantitatively assess the counterfeit localization ability of the proposed method. We have also conducted an experiment to show the effectiveness of the proposed method in the image-level forgery task.
One stream of the network learns the high-level manipulation-related features, while the other streams TH. For example, Zhou et al. showed the effectiveness of the fusion of the high-level and low-level features, learned from the training examples, in an R-CNN framework.
- Image-Stream Encoder-Decoder (ISED)
- Noise-Stream Encoder-Decoder (NSED)
- Feature Concatenation and Prediction Layer
We employ the ResNet  architecture instead of the VGGNet  architecture used in the encoder for SegNet. The low-level artifacts present in forged images, i.e. the inconsistencies in the noise level and traces related to different image processing operations are more affected by the image modification operation compared to high-level artifacts.
Pre-training on Synthetic Dataset
In this work, we used the split images of the synthetic dataset created by Bappy et al.  from datasets from DRESDEN , COCO  and NIST16. We trained the proposed network on this synthetic dataset by using 90% for training and 10% for validation.
Fine-tuning and Evaluation on Standard Forgery Datasets
These results quantitatively demonstrate the superior performance of the proposed method over LSTM-EnDec on these datasets. We have also conducted an experiment to show the performance of the proposed method on pristine images.
Finally, we have checked the performance of the network when training with the weighted cross entropy loss instead of the cube loss. The images in the last two columns are predictions of the proposed network using the weighted cross entropy and the cube losses on IFS data sets, respectively.
Rocha, “Exposing digital image forgeries by illumination color classification,” IEEE Transactions on Information Forensics and Security, vol. Farid, "Exposing Digital Forgeries in Complex Lighting Environments," IEEE Transactions on Information Forensics and Security, vol.