How Browser-Based AI Face Analysis Actually Works

You upload a selfie; one second later a bar chart tells you that you are 72% deer and 18% cat. What actually happened in that second? No server received your photo. No human designed a rule like "big eyes equal rabbit." Something genuinely interesting occurred inside your browser tab, and it is worth understanding — both because the technology is elegant and because knowing how it works explains both its capabilities and its limits. This is the plain-language tour.

Neural Networks: Pattern Learners, Not Rule Followers

Classical software follows rules a programmer wrote. Machine learning inverts this: you show a model thousands of labeled examples, and it discovers the rules itself. The model is a neural network — millions of simple numerical units organized in layers, each connection carrying a weight that training gradually adjusts. Show the network an image labeled "fox face," let it guess, measure how wrong it was, nudge every weight slightly in the direction that would have made it less wrong, and repeat at massive scale. Nobody ever tells the network what a fox face is. It converges on its own internal, statistical answer.

For images specifically, the workhorse is the convolutional neural network (CNN). Early layers of a CNN learn to detect tiny primitives — edges, corners, gradients. Middle layers combine those into textures and parts: an arc, an eye-ish blob, the boundary of a jaw. Late layers combine those into whole-image concepts. This hierarchy — pixels to edges to parts to concepts — loosely mirrors how biological vision is organized, and it is why CNNs revolutionized image recognition in the early 2010s.

Transfer Learning: Standing on a Giant's Shoulders

Training a good CNN from scratch requires millions of images and serious computing budgets. Almost no small application does this, and neither did we. Instead, tools like this test use transfer learning: start with a network already trained on millions of general photographs, keep the layers that learned universal visual skills (edges, textures, shapes — skills useful for recognizing anything), and retrain only the final layers on your specific categories.

The pre-trained base in our case is MobileNet, a family of networks Google designed specifically to be small and fast enough for phones and browsers — it trades a sliver of accuracy for being dozens of times lighter than research-grade giants. On top of that base, the AnimalFace AI classifier was trained with Teachable Machine, Google's tool for exactly this workflow: supply curated example images for each category — ten animal face archetypes — and it retrains MobileNet's final layers until the network separates them. The result is a compact model file: MobileNet's general visual competence, specialized into one narrow skill.

TensorFlow.js: Why Your Photo Never Leaves

Here is the architecturally important part. Historically, AI inference happened on servers: your photo was uploaded, a data center ran the model, an answer came back — and your face was now on someone else's computer, subject to their retention policies and their breaches. TensorFlow.js eliminates the round trip. It is Google's open-source machine learning library for JavaScript, and it runs the actual neural network math inside your browser, using your device's own GPU through WebGL for speed.

When you open the test, your browser downloads the model — the trained weights, a few megabytes, the same file for everyone. Your photo then flows through that local copy: decoded into pixels, resized to the model's input size, normalized into numbers, and pushed through the layers, all in device memory. The direction of travel is the whole privacy story: the model comes to your photo; your photo never goes to the model's home. There is no server-side code path that could receive your image, which is a much stronger guarantee than a policy promising not to look.

What the Percentages Mean

The network's final layer produces one score per category, converted by a function called softmax into positive numbers summing to 100%. So "72% deer, 18% cat, 10% others" does not mean your face is 72% deer-shaped; it means the model's learned representation of your photo sits much closer to its deer cluster than its cat cluster. It is a similarity report against training examples, expressed as confidence.

This also explains the behaviors every user notices. Different photos of you produce different results because lighting, angle, and expression genuinely change the pixel patterns — a wide smile activates features the model associates with dog and rabbit training images; heavy shadow across the brow activates wolf-ish ones. The model has no concept of "you," only of this image. Trying several photos and watching the distribution shift is honestly the most instructive way to feel what the model responds to.

Honest Limits

Understanding the pipeline makes the limits obvious rather than mysterious. The model knows only its training data: it inherits whatever biases live in the curated examples, and faces unlike anything it saw during training get less meaningful scores. It reads impressions, not truths — as our first-impressions article details, no image classifier can measure personality, because personality is not in the pixels. And confidence is not correctness: softmax will cheerfully produce a decisive-looking 85% on an out-of-distribution photo of your cat. These aren't flaws in one test so much as the honest boundaries of the entire technology — worth knowing, and entirely compatible with the tool being a great deal of fun.