Drop a screenshot of a profile picture into a classic reverse image search and you'll often get nothing back. That's not a bug. It's a consequence of what those engines are designed to match: images, not people.

If you're building anything that deals with user photos, verification or trust and safety, it's worth understanding why the two approaches behave so differently.

Approach 1: Perceptual Hashing

Most traditional reverse image search starts from some form of perceptual hashing. Unlike a cryptographic hash, a perceptual hash is designed so that visually similar images produce similar fingerprints.

How it works

A typical pipeline, such as aHash, dHash or pHash, looks like this:

• Downscale the image to a tiny grid, for example 32x32
• Convert it to greyscale
• Optionally apply a DCT and keep the low-frequency coefficients
• Threshold the values into a 64-bit fingerprint
• Compare fingerprints with Hamming distance

Where it breaks

Perceptual hashes tolerate compression and minor resizing well. They struggle with:

• Aggressive crops: the global structure changes
• Mirroring and rotation: most simple hashes are not invariant to them
• Heavy filters or overlays: shifts in tone and added UI elements
• A different photo of the same person: a completely different fingerprint

That last case is fundamental. A hash describes an image, so two different photos of one person are, by design, unrelated.

Approach 2: Face Search

Face search changes the unit of comparison from the image to the face. When you search by face, the pipeline usually has four stages.

1. Detection

A detector (RetinaFace, MTCNN or similar) finds faces and returns bounding boxes plus landmarks: eyes, nose, mouth corners. Everything outside the box is discarded, which immediately removes borders, backgrounds and screenshot chrome.

2. Alignment

Using the landmarks, each face is warped with an affine transform to a canonical pose and size, typically 112x112 with the eyes at fixed coordinates. This makes the next stage far less sensitive to tilt, scale and framing.

3. Embedding

A CNN or vision transformer trained with a margin-based loss (ArcFace-style objectives are common) maps the aligned face to a vector, often 512 dimensions, L2-normalised. The training objective pulls embeddings of the same identity together and pushes different identities apart, so cosine similarity becomes a proxy for "same person".

4. ANN vector search

Exact nearest-neighbour search over millions of vectors is too slow, so production systems use approximate nearest neighbour indexes:

• HNSW graphs for low-latency, high-recall queries
• IVF partitioning combined with product quantisation to save memory
• A similarity threshold to drop weak matches before ranking

The result is a ranked list of faces, from indexed public sources, closest to the query.

Why This Matters in Practice

Because comparison happens in embedding space, the cases that break hashing often survive: a cropped selfie, a mirrored image, a filtered profile picture, even a different photo of the same person. That's why a tool that lets you search by face can spot a dating profile using borrowed photos, or a fake account impersonating someone, when a hash-based lookup returns nothing.

Trade-offs to keep in mind

• False positives: lookalikes exist; thresholds need tuning
• Quality sensitivity: blur, occlusion and extreme angles degrade embeddings
• Bias: model accuracy can vary across demographic groups, so evaluate on diverse data

The Responsible-Use Part

Face embeddings are biometric data. If you build or use this kind of system:

• Index only publicly available sources
• Avoid retaining uploaded images longer than necessary
• Design for verification and safety use cases: catfish checks, impersonation reports, searching your own face
• Never support tracking or exposing private individuals

Takeaway

Hashes answer "where is this image?". Embeddings answer "where is this face?". Pick the one that matches the question you're actually asking.