Face Recognition

Face recognition is a category of computer vision technology that automatically identifies or verifies a person by analysing the geometric and textural features of a human face. The system typically captures an image or video frame, detects the face within it, normalises the orientation and lighting, extracts a numerical representation known as a face embedding, and compares that embedding to one or more enrolled templates to produce an identity match or verification decision.

Technical foundations

Modern face recognition systems are built on deep learning, particularly convolutional neural networks (CNNs) trained on large labelled image datasets. The pipeline generally has four stages: face detection, alignment, feature extraction, and matching. Detection locates faces in an image, often using single-shot detectors such as MTCNN or RetinaFace. Alignment normalises pose and scale using facial landmarks. Feature extraction transforms the aligned face into a fixed-length vector (commonly 128 to 512 dimensions) trained with metric learning objectives such as triplet loss, ArcFace, or CosFace. Matching computes cosine similarity or Euclidean distance between embeddings and applies a threshold to decide identity.

Breakthrough architectures include DeepFace (2014), FaceNet (2015), VGGFace, and the ArcFace family, which pushed verification accuracy on the Labelled Faces in the Wild (LFW) benchmark past 99 percent. Subsequent research has focused on robustness to pose, illumination, age, occlusion, and presentation attacks.

Verification versus identification

Face recognition has two distinct operating modes. Verification (one-to-one matching) confirms whether a claimed identity matches the live face, typical of login or eKYC flows. Identification (one-to-many matching) searches a gallery to determine who a face belongs to, used in surveillance, watch-list screening, and missing-person investigations. The two modes have different error profiles: verification trades off false acceptance and false rejection rates, while identification is additionally constrained by gallery size and rank-N accuracy.

Liveness detection and presentation attacks

Because static photographs, printed masks, and replayed videos can spoof naive recognisers, production systems pair recognition with liveness detection, also called presentation attack detection (PAD). Active liveness asks the user to blink, smile, or turn their head. Passive liveness analyses texture, depth, micro-movements, or infrared signal without user interaction. ISO/IEC 30107 defines the standard taxonomy for presentation attacks.

Applications

| Domain | Example use | |---|---| | Banking and fintech | eKYC onboarding, transaction authentication | | Mobile devices | Phone unlock (Face ID, equivalent Android implementations) | | Border control | Automated immigration gates | | Retail and access | Office entry, event check-in, age estimation | | Law enforcement | Suspect identification, missing person search |

Accuracy, bias and governance

Public benchmarks such as the NIST Face Recognition Vendor Test (FRVT) report continuous accuracy gains, with top algorithms now exceeding 99.8 percent verification accuracy at controlled false-match rates. However, NIST and academic studies have repeatedly documented demographic differentials, with higher error rates for darker-skinned, female, elderly, and very young subjects. Mitigation involves more representative training data, fairness-aware loss functions, and operational thresholds calibrated per population.

Regulators worldwide have introduced laws restricting public-sector and real-time biometric surveillance, with the European Union AI Act, Illinois BIPA, and various national data protection laws imposing consent, purpose-limitation, and retention requirements on face data.

See Also

References

References

1. Schroff, F., Kalenichenko, D., and Philbin, J. (2015). FaceNet: A Unified Embedding for Face Recognition and Clustering. CVPR.
2. Deng, J. et al. (2019). ArcFace: Additive Angular Margin Loss for Deep Face Recognition. CVPR.
3. National Institute of Standards and Technology. Face Recognition Vendor Test (FRVT) Ongoing. NIST.
4. Bank Negara Malaysia. (2020, updated 2024). Policy Document on Electronic Know-Your-Customer (e-KYC).
5. ISO/IEC 30107-3. Information technology — Biometric presentation attack detection.