FUSION OF MULTIMODAL BIOMETRICS
JYOTSANA GROVER
DEPARTMENT OF ELECTRICAL ENGINEERING INDIAN INSTITUTE OF TECHNOLOGY DELHI
OCTOBER 2014
© Indian Institute of Technology Delhi (IITD), New Delhi, 2014
FUSION OF MULTIMODAL BIOMETRICS
by
JYOTSANA GROVER
DEPARTMENT OF ELECTRICAL ENGINEERING Submitted
in fulfillment of the requirements of the degree of DOCTOR OF PHILOSPHY
to the
INDIAN INSTITUTE OF TECHNOLOGY DELHI
OCTOBER 2014
CERTIFICATE
This is to certify that the thesis titled“Fusion of Multimodal Biometrics”being submitted by Ms. Jyotsana Grover to the Department of Electrical Engineering, Indian Institute of Technology, Delhi, for the award of the degree of Doctor of Philos- ophy, is a record of bonafide research work carried out by her under our guidance and supervision. In our opinion, the dissertation has reached the standards fulfilling the requirements of the regulations relating to the degree.
The results contained in this dissertation have not been submitted to any other uni- versity or institute for the award of any degree or diploma.
Prof. M. Hanmandlu Prof. H. M. Gupta
Supervisor Supervisor
Electrical Engineering Department Electrical Engineering Department Indian Institute of Technology, Delhi Indian Institute of Technology, Delhi
New Delhi-110016 New Delhi-110016
ACKNOWLEDGEMENTS
It is my pleasure to record deepest gratitude to my supervisors, Prof. M. Hanmandlu and Prof. H. M. Gupta for giving me this opportunity to work under their supervision and for making my research a beautiful journey. I am thankful to their consistent encouragement and constant advice.
I am thankful to IIT Delhi authorities for providing me the necessary facilities for the smooth completion of my work. I would like to give special thanks to my SRC members for their valuable suggestions and feedback during my research.
I would also like to thank all my friends at IIT Delhi for their constant support and encouragement throughout my PhD days. Most importantly, I would like to express my heart-felt gratitude to my parents and my closed ones. None of this would have been possible without the love and patience of my family.
I would like to thank my thesis examiners for their valuable suggestions for improving the thesis.
Finally, my greatest regards to the Almighty for bestowing upon me the courage to face the complexities of life and complete this thesis.
Jyotsana Grover
i
ABSTRACT
The thesis considers the multimodal biometric fusion of different instances of finger- knuckle-prints and also of multiple representations of the palmprints. The instances of finger-knuckle-prints (FKPs) consist of left index, left middle, right index, and right middle FKPs whereas the multiple representations of palmprints include its multispec- tral bands such as Near-Infra-Red (NIR), Red, Blue, and Green. The thesis considers four types of fusion methodologies, viz., feature level, score level, rank level, and deci- sion level. In addition we have devised error level fusion.
To accomplish the multimodal biometric fusion, we have proposed new textural fea- tures: topothesy-fractal dimension, Hanman Transform, and structure function based transform features for FKPs and multispectral palmprints. To evaluate these features we have proposed T-norm based classifier whose performance measured in terms of Equal Error Rates (EER) is at par with that of SVM. Moreover these features outper- form the existing features like Gabor filter using the proposed classifier.
As part of feature level fusion, the filter based feature selection is achieved using new generalized fuzzy entropy. It is shown to be more effective than the commonly employed wrapper and embedded techniques.
Score level fusion is accomplished using the T-norms as these norms take care of the uncertainty associated with the scores and this approach is found to be more efficient than the existing classification based and the likelihood ratio based approaches for the score level fusion. This fusion has been extended to hybrid fusion by combining with the feature level fusion.
The rank level fusion is improvised by using the entropy function and in addition some non linear functions like logarithmic, sine hyperbolic inverse and sigmoid are also
iii
investigated for the rank level fusion. We have also adapted the Choquet integral for the rank level fusion as it takes care of the overlapping information between the modal- ities. A hybrid learning algorithm called Reinforced hybrid Bacterial Foraging Particle Swarm Optimization is devised to learn the fuzzy densities and the interaction factor needed in the Choquet integral.
In order to take care of the uncertainty in the decisions (accept/reject), the adap- tive fuzzy decision level fusion is formulated. Further, the hybridization of score level fusion and adaptive fuzzy decision level fusion is also carried out with a view to reduce the complexity of finding the best fusion rule when the number of modalities increases.
The error level fusion is proposed using the Choquet integral whose role in this is to fuse the error rates of different modalities by taking care of the overlapping information present among the modalities.
All the fusion methodologies have been tested on publicly available databases of FKPs and multispectral palmprints and their effectiveness has been ascertained using the proposed features and classifier.
iv
Table of Contents
Acknowledgements . . . i
Abstract . . . iii
Table of Contents . . . v
List of Figures . . . ix
List of Tables . . . xiii
Acronyms . . . xvii
1 Introduction 1 1.1 Multimodal biometric systems . . . 4
1.1.1 Need for the multimodal biometric systems . . . 5
1.2 Levels of fusion . . . 5
1.3 Literature Survey on Fusion . . . 7
1.4 Motivation for the work in the thesis . . . 13
1.5 Issues in Multimodal Biometrics . . . 14
1.6 Organization of the Thesis . . . 15
2 Extraction of biometric textural features 17 2.1 Introduction . . . 17
2.1.1 The existing features of FKPs . . . 17
2.1.2 The Existing features of palmprints . . . 19
2.2 Motivation . . . 21
2.3 Databases used . . . 22
2.4 Extraction of the proposed texture features . . . 22
2.4.1 An Introduction to Information Sets . . . 23
v
2.4.2 Fractal Parameters . . . 28
2.4.3 Structure function based transform features . . . 31
2.5 Hanman Classifier based on triangular norms . . . 32
2.5.1 Triangular norms (T-norms) . . . 32
2.5.2 Classifier based on T-norms . . . 33
2.5.3 Algorithm for classifier . . . 35
2.6 Experimental results . . . 35
2.7 Conclusions . . . 44
3 Feature Level Fusion 47 3.1 Introduction . . . 47
3.1.1 Earlier works on the feature selection . . . 48
3.2 Motivation . . . 49
3.3 Review of the existing fuzzy entropies . . . 50
3.4 Proposed fuzzy entropy . . . 51
3.5 Fuzzy entropy based feature selection . . . 55
3.6 Experimental Results . . . 57
3.7 Conclusions . . . 62
4 Score Level Fusion 63 4.1 Introduction . . . 63
4.2 Literature Survey . . . 63
4.3 Motivation for using T-norms for the score level fusion . . . 65
4.4 Score level fusion using T-norms . . . 66
4.5 Hybrid of Feature level and Score level fusions . . . 67
4.6 Experimental Results . . . 67
4.6.1 Results of the score level fusion . . . 67
4.6.2 Results of Hybrid fusion . . . 75
4.7 Conclusions . . . 77
vi
5 Rank Level Fusion 79
5.1 Introduction . . . 79
5.2 Literature Survey . . . 80
5.3 Motivation . . . 81
5.4 Approaches for the Rank Level Fusion . . . 82
5.4.1 Highest Rank Method . . . 82
5.4.2 Borda Count . . . 82
5.4.3 Weighted Borda Count . . . 83
5.5 The Proposed Rank level fusion methods . . . 83
5.5.1 Entropy based function for the rank level fusion . . . 85
5.6 Experimental Results . . . 86
5.7 Conclusions . . . 94
6 Adaptive Fuzzy Decision Level Fusion 95 6.1 Introduction . . . 95
6.1.1 Literature survey on Decision level fusion . . . 96
6.2 The Framework for Adaptive Decision Level Fusion . . . 98
6.3 Adaptive Decision Level Fusion . . . 99
6.4 Motivation for the adaptive fuzzy decision level fusion . . . 101
6.5 The Relevant Fuzzy Logic Concepts . . . 102
6.5.1 Fuzzy parameters . . . 102
6.5.2 Total Distance Criterion . . . 102
6.6 Adaptive Fuzzy Bayesian Decision Framework . . . 103
6.7 Adaptive Fuzzy Decision Level Fusion . . . 104
6.8 Hybrid of Score level and the adaptive fuzzy Decision level fusion methods107 6.9 Experimental Results . . . 109
6.10 Conclusions . . . 116
7 Fusion using the Choquet integral 119 7.1 Introduction . . . 119
7.1.1 Existing works on Choquet Integral . . . 120
vii
7.1.2 Motivation . . . 120
7.2 Properties of a Fuzzy Integral . . . 121
7.2.1 The q-measures . . . 122
7.2.2 Finding the q-measures from the fuzzy densities . . . 122
7.2.3 The Choquet Integral . . . 123
7.3 Rank level fusion . . . 124
7.4 Error level fusion . . . 124
7.5 Reinforced Hybrid Bacterial Foraging-Particle Swarm Optimization (BF- PSO) . . . 126
7.6 Experimental Results . . . 130
7.7 Conclusions . . . 133
8 Conclusions and Suggestions for Future Research 135 8.1 Concluding Remarks . . . 135
8.2 Contributions of the thesis . . . 137
8.3 Limitations of the thesis research . . . 138
8.4 Suggestions for future research . . . 138
References 141 Appendix 161 A Feature Selection 161 B Properties of entropy function 163 Research Publications . . . 169
Biodata of the Author . . . 171
viii
