A practical face-recognition system may have only one enrollment image for each person. It must recognize that person later despite changes in lighting, expression, pose, camera quality, or appearance.

This is a one-shot learning problem:\[ \text{one enrolled example} \longrightarrow \text{recognize future examples} \]

Training an ordinary image classifier from one example per identity is generally ineffective. A better approach is to train a reusable similarity function on a much larger collection of identities and then apply it to newly enrolled people.

The One-Shot Recognition Problem

Suppose an organization stores one reference image for each of four employees:\[ \mathcal{D} = \{ (x_1,\text{person}_1), (x_2,\text{person}_2), (x_3,\text{person}_3), (x_4,\text{person}_4) \} \]

When a new face image \(x_q\) is captured, the system must determine whether it belongs to one of these people.

The problem is difficult because each identity has only one stored example. The system must tolerate natural differences between the enrollment and probe images.

For example, the same person may appear with:

A different facial expression
Different lighting
A different camera angle
Different clothing or accessories
A changed hairstyle
Lower image quality
Partial occlusion

At the same time, the system must reject an unknown person who is not enrolled.

Why an Ordinary Classifier Is a Poor Fit

A direct classification approach might use a neural network with one output for each enrolled person and another output for unknown:\[ x \rightarrow \operatorname{ConvNet}(x) \rightarrow \operatorname{softmax} \]

For four enrolled people:\[ \hat{y} \in \mathbb{R}^{5} \]

The outputs would correspond to:

Person 1
Person 2
Person 3
Person 4
Unknown

This approach has several problems.

Too Little Data per Identity

A large neural network cannot learn the full visual variation of one person from only one image.

It may memorize irrelevant details such as:

Background
Lighting
Image crop
Clothing
Camera characteristics

Memorizing the enrollment photograph is not the same as learning identity.

Changing Output Classes

If a fifth person is enrolled, the classifier needs another output:\[ 5+1=6 \]

The output layer changes whenever a person is added or removed.

Retraining the model after every enrollment is inefficient and operationally inconvenient.

The Unknown Class Is Poorly Defined

“Unknown” does not represent one coherent category. It represents every person who is not enrolled.

A classifier cannot observe all possible unknown identities during training. Treating this enormous open set as one ordinary class is therefore problematic.

Closed-Set Assumption

A standard softmax classifier is naturally suited to closed-set classification, where every input is assumed to belong to one of the training classes.

Real face-recognition systems often require open-set behavior:\[ \text{known identity} \quad\text{or}\quad \text{unknown} \]

This requires a rejection rule rather than unconditional classification.

Learning a Similarity Function

Instead of training one classifier over the enrolled identities, learn a function:\[ d(x_i,x_j) \]

that measures the difference between two face images.

The desired behavior is:\[ d(x_i,x_j)\text{ small} \]

when the images show the same person, and:\[ d(x_i,x_j)\text{ large} \]

when they show different people.

A verification rule is:\[ \text{same identity} \quad\text{if}\quad d(x_i,x_j)\leq\tau \]

and:\[ \text{different identities} \quad\text{if}\quad d(x_i,x_j)>\tau \]

where \(\tau\) is a calibrated threshold.

Embedding-Based Similarity

The similarity function is usually implemented with an embedding network:\[ f_\theta(x)\in\mathbb{R}^{d} \]

The network converts each face into a numerical representation.

The distance between two faces can then be:\[ d(x_i,x_j) = \left\| f_\theta(x_i)-f_\theta(x_j) \right\|_2 \]

Other choices include:

Squared Euclidean distance
Cosine distance
A learned pairwise comparison function

The model is trained so that identity-related information determines position in the embedding space.

Shared Embedding Space

A useful embedding space should satisfy:\[ \operatorname{id}(x_i) = \operatorname{id}(x_j) \quad\Longrightarrow\quad f_\theta(x_i)\approx f_\theta(x_j) \]

and:\[ \operatorname{id}(x_i) \neq \operatorname{id}(x_j) \quad\Longrightarrow\quad f_\theta(x_i)\text{ and }f_\theta(x_j)\text{ are separated} \]

This representation is not tied to the particular people enrolled in one application.

The network learns general facial similarity from a larger representation-training dataset, while the local enrollment database can contain previously unseen identities.

Verification

Face verification performs a one-to-one comparison.

The input consists of:

A probe image \(x_q\)
A claimed identity \(k\)

Retrieve the enrolled template:\[ e_k=f_\theta(x_k) \]

Compute the probe embedding:\[ e_q=f_\theta(x_q) \]

Then calculate:\[ d_k = \|e_q-e_k\|_2 \]

The decision is:\[ \text{accept} \quad\text{if}\quad d_k\leq\tau \]

Otherwise:\[ \text{reject} \]

This answers:

Does the probe match the claimed identity?

Identification

Face identification performs a one-to-many search.

Suppose the gallery contains:\[ \mathcal{G} = \{e_1,e_2,\ldots,e_K\} \]

For a probe embedding \(e_q\), calculate:\[ d_k = \|e_q-e_k\|_2 \]

for every enrolled identity.

The closest candidate is:\[ k^* = \underset{k}{\operatorname{argmin}} \; d_k \]

In an open-set system:\[ \hat{y} = \begin{cases} k^*, & d_{k^*}\leq\tau\\ \text{unknown}, & d_{k^*}>\tau \end{cases} \]

This answers:

Which enrolled identity is the closest acceptable match, or is the person unknown?

Example

Suppose the distances between a probe and four enrolled templates are:\[ (8.4,\;0.32,\;7.1,\;9.0) \]

If:\[ \tau=0.6 \]

the second identity is accepted because:\[ 0.32\leq0.6 \]

Now suppose an unknown person produces:\[ (7.2,\;6.8,\;8.1,\;7.5) \]

The minimum is:\[ 6.8 \]

Since:\[ 6.8>0.6 \]

the system returns unknown.

The numerical scale is model-dependent; these values are only illustrative.

Why This Solves One-Shot Enrollment

When a new person joins the system:

Capture an enrollment image.
Detect and align the face.
Compute its embedding.
Store the embedding with the new identity.

Formally:\[ e_{\text{new}} = f_\theta(x_{\text{new}}) \]

The network does not need a new output unit, and its parameters do not need to be retrained.

The gallery simply changes from:\[ \{e_1,\ldots,e_K\} \]

to:\[ \{e_1,\ldots,e_K,e_{K+1}\} \]

This makes embedding-based recognition much more scalable than an identity-specific softmax classifier.

One-Shot Enrollment Does Not Mean One-Shot Model Training

The deployed system may enroll each new person using one image, but the embedding network itself is usually trained on a much larger dataset.

Representation training generally requires:

Many different identities
Multiple images for many identities
Variation in pose, lighting, expression, and image quality
Carefully constructed positive and negative relationships

The distinction is:

Model Training

Learn a general similarity representation from substantial data.

Enrollment

Add a new identity using one or a few examples without retraining the model.

The “one shot” applies to enrolling a new identity, not necessarily to training the complete neural network from one example.

Training the Similarity Function

Several objectives can train an embedding network.

Contrastive Loss

Contrastive loss operates on pairs:\[ (x_i,x_j,y) \]

where \(y\) indicates whether the identities match.

It pulls matching embeddings together and pushes nonmatching embeddings apart.

Triplet Loss

Triplet loss operates on:\[ (A,P,N) \]

where:

\(A\) is an anchor.
\(P\) shows the same identity.
\(N\) shows a different identity.

It enforces:\[ d(A,P)+\alpha\leq d(A,N) \]

where \(\alpha\) is a margin.

Binary Pair Classification

A Siamese network can compute two embeddings and pass their difference into a classifier:\[ \hat{y} = P(\text{same identity}\mid x_i,x_j) \]

Classification-Based Metric Learning

The embedding network can also be trained over a large set of training identities using a classification objective designed to create well-separated embeddings.

After training, the original classification head is removed, and the learned embedding is used for new identities.

Threshold Selection

The threshold \(\tau\) controls the balance between security and convenience.

A smaller distance threshold is stricter:

Fewer impostors are accepted.
More genuine users may be rejected.

A larger threshold is more permissive:

More genuine users are accepted.
More impostors may be accepted.

The threshold should be selected on representative validation data.

It should not be chosen only because it worked for another model, dataset, or environment.

Verification Metrics

Overall accuracy is often insufficient. Important metrics include:

False-Accept Rate

\[ \operatorname{FAR} = \frac{ \text{impostor attempts accepted} }{ \text{total impostor attempts} } \]

False-Reject Rate

\[ \operatorname{FRR} = \frac{ \text{genuine attempts rejected} }{ \text{total genuine attempts} } \]

True-Accept Rate

\[ \operatorname{TAR} = 1-\operatorname{FRR} \]

The appropriate operating point depends on the application’s risk requirements.

Identification Becomes Harder as the Gallery Grows

In one-to-many recognition, each probe is compared with many identities.

If one nonmatching comparison has false-match probability \(q\), a simplified independence model gives the probability of at least one false match among \(K\) comparisons as:\[ 1-(1-q)^K \]

For small \(q\), this is approximately:\[ Kq \]

This is only an intuition because real comparisons are not perfectly independent. Nevertheless, it illustrates why a threshold suitable for verification may be too permissive for a large identification gallery.

Identification performance should be evaluated at the intended gallery size.

Precomputing Gallery Embeddings

Enrolled images do not change frequently, so their embeddings can be computed once:\[ e_k=f_\theta(x_k) \]

At recognition time, calculate only the probe embedding:\[ e_q=f_\theta(x_q) \]

Then compare \(e_q\) with the stored gallery.

This avoids repeatedly processing every enrollment image through the convolutional network.

For a large gallery, approximate nearest-neighbor search can further accelerate matching.

Multiple Enrollment Images

Although one-shot enrollment is possible, several enrollment images often improve robustness.

For identity \(k\), store:\[ e_{k,1},e_{k,2},\ldots,e_{k,n_k} \]

The system can:

Compare against every template
Keep the best match
Average similarity scores
Build a normalized mean embedding
Learn a template-aggregation function

A normalized mean template is:\[ \bar{e}_k = \frac{ \sum_{r=1}^{n_k}e_{k,r} }{ \left\| \sum_{r=1}^{n_k}e_{k,r} \right\|_2 } \]

Multiple images can cover different poses and capture conditions.

Face Detection and Alignment

The embedding comparison assumes that the input contains a properly extracted face.

A practical pipeline is:

input image
    ↓
face detection
    ↓
facial landmark estimation
    ↓
face alignment and cropping
    ↓
embedding network
    ↓
similarity comparison

input image
    ↓
face detection
    ↓
facial landmark estimation
    ↓
face alignment and cropping
    ↓
embedding network
    ↓
similarity comparison

Poor detection or alignment can produce a large distance even for the correct person.

The embedding model and preprocessing pipeline should be evaluated together.

Liveness Is a Separate Requirement

A small embedding distance means that the captured face resembles the enrolled identity. It does not prove that a live person is present.

A photograph, screen replay, mask, or synthetic image might produce a strong identity match.

Security-sensitive systems may require separate presentation-attack detection:\[ \text{accept} = \text{identity match} \land \text{liveness accepted} \]

Additional safeguards may include rate limiting, audit logs, device checks, or another authentication factor.

Common Mistakes

Training a New Classifier for Every Gallery

This makes enrollment expensive and does not use the main advantage of metric learning.

Treating the Closest Identity as Automatically Correct

An open-set system must reject the probe when even the closest match is not sufficiently similar.

Choosing the Threshold on Training Data

Threshold calibration should use separate data representative of deployment.

Assuming One Enrollment Image Is Always Sufficient

One-shot enrollment is possible, but additional high-quality images often improve reliability.

Ignoring Gallery Size

False-identification risk can increase as more identities are searched.

Storing Embeddings Without Protection

Face embeddings are sensitive biometric templates and should receive strong access, encryption, retention, and auditing controls.

Confusing Enrollment Data with Representation-Training Data

The system may enroll a person from one image only because the embedding model has already learned general face similarity from much broader data.

Key Takeaway

One-shot face recognition is handled by learning a reusable similarity function rather than training a new identity classifier for every enrolled person.

A neural network produces an embedding:\[ f_\theta(x) \]

and face difference is measured by:\[ d(x_i,x_j) = \left\| f_\theta(x_i)-f_\theta(x_j) \right\|_2 \]

Verification accepts a claimed identity when:\[ d(x_i,x_j)\leq\tau \]

Identification finds the closest enrolled embedding and returns it only if the match satisfies the threshold.

This design allows a new identity to be added by storing one new embedding, without changing the network architecture or retraining the model.