Masked LFW (MLFW) Database
-
Motivation
-
Welcome to Masked LFW (MLFW) database, a renovation of Cross-Age LFW (CALFW) database.
As more and more people begin to wear masks due to current COVID-19 pandemic, existing face recognition systems may encounter severe performance degradation when recognizing masked faces. To figure out the impact of masks on face recognition model, we build a simple but effective tool to generate masked faces from unmasked faces automatically, and construct a new database called Masked LFW (MLFW) based on Cross-Age LFW (CALFW) database. The mask on the masked face generated by our method has good visual consistency with the original face. Moreover, we collect various mask templates, covering most of the common styles appeared in the daily life, to achieve diverse generation effects. Considering realistic scenarios, we design three kinds of combinations of face pairs. The recognition accuracy of SOTA models declines 5%-16% on MLFW database compared with the accuracy on the original images. There are three motivations behind the construction of MLFW benchmark as follows:
1.Establishing a relatively more difficult database to evaluate the performance of masked face verification so the effectiveness of several face verification methods can be fully justified.
2.Beyond the age gap, MLFW considers that two faces with the same identity wear different masks and two faces with different identities wear the same mask, which further emphasizes both the large intra-class variance and the tiny inter-class variance simultaneously.
3.Maintaining the data size, the face verification protocol which provides a 'same/different' benchmark while maintaining the same identity as that in calfw. Therefore, MLFW can be easily applied to evaluate the performance of masked face verification.
-
Examples of masked faces
-
MLFW is constructed by adding mask to the images in CALFW with perturbation for achieving diverse generation effects.
We dedicate to maintain the protocols, dataset size, and the identities in each fold of CALFW database in order to encourage fair and meaningful comparisons. You can find more information about standard CALFW protocol in Cross-Age LFW (CALFW).
We expect MLFW could promote algorithms to make reliable verification judgement, and close the large gap between the reported performance on benchmarks and performance on real world tasks.
Baseline Results
We select multiple SOTA deep face recognition methods that have achieved top performance on major benchmark databases.
COMPARISON OF VERIFICATION ACCURACY (%) ON DIFFERENT DATABASES USING SOTA DEEP FACE RECOGNITION MODELS.
| Method | LFW7 | SLLFW8 | TALFW9 | CPLFW10 | CALFW11 | MLFW (ours) |
|---|---|---|---|---|---|---|
| Private-Asia, R50, ArcFace1 | 99.50% | 98.00% | 69.97% | 84.12% | 91.12% | 74.85% |
| CASIA-WebFace, R50, CosFace2 | 99.50% | 98.40% | 49.48% | 87.47% | 92.43% | 82.87% |
| VGGFace2, R50, ArcFace3 | 99.60% | 98.80% | 55.37% | 91.77% | 93.72% | 85.02% |
| MS1MV2, R100, Arcface4 | 99.77% | 99.65% | 64.48% | 92.50% | 95.83% | 90.13% |
| MS1MV2, R100, Curricularface5 | 99.80% | 99.70% | 69.32% | 93.15% | 95.97% | 90.60% |
| MS1MV2, R100, SFace6 | 99.82% | 99.68% | 64.47% | 93.28% | 95.83% | 90.63% |
- Qingzhong Wang, Pengfei Zhang, Haoyi Xiong, and Jian Zhao. Face. evolve: A high-performance face recognition library. arXiv preprint arXiv:2107.08621, 2021.
- Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z. Li. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.
- Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pages 67–74. IEEE, 2018.
- Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4690– 4699, 2019.
- Yuge Huang, Yuhan Wang, Ying Tai, Xiaoming Liu, Pengcheng Shen, Shaoxin Li, Jilin Li, and Feiyue Huang. Curricularface: adaptive curriculum learning loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5901–5910, 2020.
- Yaoyao Zhong, Weihong Deng, Jiani Hu, Dongyue Zhao, Xian Li, and Dongchao Wen. Sface: sigmoid-constrained hypersphere loss for robust face recognition. IEEE Transactions on Image Processing, 30:2587–2598, 2021.
- Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. 2008.
- Weihong Deng, Jiani Hu, Nanhai Zhang, Binghui Chen, and Jun Guo. Fine-grained face verification: Fglfw database, baselines, and human-dcmn partnership. Pattern Recognition, 66:63–73, 2017.
- Yaoyao Zhong andWeihong Deng. Towards transferable adversarial attack against deep face recognition. IEEE Transactions on Information Forensics and Security, 16:1452–1466, 2020.
- Tianyue Zheng and Weihong Deng. Cross-pose lfw: A database for studying cross-pose face recognition in unconstrained environments. Beijing University of Posts and Telecommunications, Tech. Rep, 5, 2018.
- Tianyue Zheng,Weihong Deng, and Jiani Hu. Cross-age lfw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197, 2017.