Cross-Pose LFW (CPLFW) Database
Motivation
-
Attention! We updated the positive/negative lists and baselines for CPLFW in September 19th,2018. Please use the new list to do experiments.
Welcome to Cross-Pose LFW (CPLFW) database, a renovation of Labeled Faces in the Wild (LFW), the de facto standard testbed for unconstraint face verification.
Labeled Faces in the Wild (LFW) database has been widely utilized as the benchmark of unconstrained face verification and due to big data driven machine learning methods, the performance on the database approaches nearly 100%. However, we argue that this accuracy may be too optimistic. Besides different illuminations, occlusions and expressions, cross-pose face is another challenge in face recognition yet LFW does not pay much attention on it. Thereby we construct a Cross-Pose LFW (CPLFW) which deliberately searches and selects 3,000 positive face pairs with pose difference to add pose variation to intra-class variance. Negative pairs with same gender and race are also selected to reduce the influence of attribute difference between positive/negative pairs. We evaluate several deep learning methods on the new database. Compared to the accuracy on LFW, the accuracy drops about 15%-20% on CPLFW. There are three motivations behind the construction of CPLFW benchmark as follows:
1.Establishing a relatively more difficult database to evaluate the performance of real world face verification so the effectiveness of several face verification methods can be fully justified.
2.Continuing the intensive research on LFW with more realistic consideration on pose intra-class variation and fostering the research on cross-pose face verification in unconstrained situation. The challenge of CPLFW emphasizes pose difference to further enlarge intra-class variance. Also, negative pairs are deliberately selected to avoid different gender or race. CPLFW considers both the large intra-class variance and the tiny inter-class variance simultaneously.
3.Maintaining the data size, the face verification protocol which provides a 'same/different' benchmark and the same identities in LFW, so one can easily apply CPLFW to evaluate the performance of face verification.
For more information, please read the technical report： Cross-Pose-LFW.pdf Comparison with LFW
-
Pose distribution comparison
According to the figures, the yaw distribution of images in CPLFW is more average. Also, pose difference of most positive pairs in LFW is less than 40 degrees while that of most positive pairs in CPLFW is larger. This confirms the existence of pose variation in intra-class variance of CPLFW.
Positive pairs comparison
CPLFW is collected by crowdsourcing efforts to seek the pictures of people in LFW with pose difference as large as possible on the Internet. Compared to LFW, the positive pairs in CPLFW contain obvious pose difference.
Compared to LFW, the negative pairs in CPLFW have same gender and race, which reduces the influence of attribute difference between positive pairs and negative pairs in face verification.
We dedicate to maintain the protocols, dataset size, and the identities in each fold of LFW database in order to encourage fair and meaningful comparisons. You can find more information about standard LFW protocol in Labeled Faces in the Wild (LFW).
We expect CPLFW could promote algorithms to make reliable verification judgement, and close the large gap between the reported performance on benchmarks and performance on real world tasks.
Baseline Results
We select three SOTA deep face recognition methods that have achieved top performance on major benchmark databases: LFW, IJB-A and MegaFace..
COMPARISON OF VERIFICATION ACCURACY (%) ON LFW AND CPLFW USING FOUR SOTA DEEP FACE RECOGNITION MODELS.
| Method | LFW | CPLFW | Centerface1 | 98.75% | 77.48% |
|---|---|---|
| SphereFace2 | 99.27% | 81.40% |
| VGGFace23 | 99.43% | 84.00% |
| ArcFace4 | 99.82% | 92.08% |
| HUMAN-Individual | 97.27% | 81.21% |
| HUMAN-Fusion | 99.85% | 85.24% |
COMPARISON OF 10-FOLD VALIDATION ERROR (%) OF THREE SOTA DEEP FACE RECOGNITION MODELS. THE INCREASE OF ERROR IS ALSO ENUMERATED WHEN TRANSFERRING FROM LFW TO CPLFW.
| Method | LFW | CPLFW | Centerface1 | 1.17 | 22.52 ( ↑ 1925%) |
|---|---|---|
| SphereFace2 | 0.65 | 18.60 ( ↑ 2862%) |
| VGGFace23 | 0.49 | 16.00 ( ↑ 3265%) |
- A discriminative feature learning approach for deep face recognition. In European Conference on Computer Vision, Springer, 2016, pp. 499–515.
- Deep hyperspherical learning. In NIPS, 2017, pp. 3953–3963.
- Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman. Vggface2: A dataset for recognising faces across pose and age. arXiv preprint arXiv：1710.08092, 2017.
- Arcface: Additive angular margin loss for deep face recognition. arXiv preprint arXiv：1801.05599, 2018.