We hid the scales before testing calorie estimates.
A weighed meal is a useful reference. A visible scale is an answer leak. We prepared 179 public photos, tested a challenger and found food-matching errors worth fixing.
Gemini 3.1 Flash Lite stays. Mistral Large 4.0 did not meet our requirements for a switch. We also corrected food-database matches shared by both models. The remaining results show why recognition, portion size and nutrition matching need to be measured separately.
First, remove the answer from the picture
DFood-TJ supplies public hospital-meal photos with weighed food and table-derived nutrition under CC0. We masked visible scale and weight cues without changing food pixels, excluded unsafe overlaps and duplicate photos, and kept a hash of every original, prepared image and exact model input.
The final set contains 61 development photos and 118 confirmation photos, separated by participant. Public data may have appeared in training, and repeated hospital menus can cross participant groups. We call this a confirmation split, not guaranteed unseen food.
A plausible total can hide a wrong match
The pilot's “coleslaw, no dressing” query selected coleslaw dressing. Steamed buns selected hamburger rolls. The larger development audit found black rice mapped to rice-and-beans, ordinary beef mapped to bacon, and a dressed salad mapped to dressing alone.
We fixed those identities and checked the entire saved USDA candidate pools. We also reject pure fat and offal rows when the query asks for ordinary meat. If the database has no suitable row, the existing model nutrition estimate handles it.
These are correctness fixes, not proof of a large accuracy gain. On the first 12 photos, correcting the matches slightly reduced average calorie error but moved one more photo outside the 25% tolerance. An overly dense food match can accidentally cancel an underestimated portion.
The model comparison
Both models ran the same production analysis code on all 61 development photos. Mistral used reasoning off for recognition and nutrition. After correcting the matcher, we replayed the saved responses and bought only newly needed nutrition estimates.
| Model | Completed | Calories within 25% | Mass within 25% | Model time p95 | Cost / completed meal |
|---|---|---|---|---|---|
| Gemini 3.1 Flash Lite | 61/61 | 33/61 | 62.3% | 4.11 s | $0.00076 |
| Mistral Large 4.0 | 60/61 | 28/61 | 52.5% | 9.93 s | $0.00283 |
All 61 attempts stay in the accuracy denominators, including Mistral’s recognition timeout. Cost includes all attempts and a reservation for the timeout’s unknown charge. Time sums model stages, reuses their recorded timings and excludes cold USDA searches, database work and hosted overhead.
Before the second matching correction, the calorie counts were 33/61 for Gemini and 29/61 for Mistral; afterward, 33/61 and 28/61. The paired calorie difference has a 95% participant-cluster interval from -21.2 to +3.8 percentage points. That is insufficient evidence to replace Gemini, and Mistral also missed the fixed speed, cost, macro and portion gates.
The frozen confirmation check
We then froze the model, prompts, matcher, input encoding, analysis and spending limit. Gemini alone scanned each of the 118 confirmation photos twice. No confirmation result was used to tune this version.
235/236 scans completed. 113/236 were within 25% of the source calorie reference: 47.9% (95% participant-cluster interval 40.9 to 55.8%). Measured mass was within 25% on 62.7% of attempts.
Mean absolute calorie error among completed scans was 88.6 kcal. Median model-stage time was 2.72 seconds, p95 3.67 seconds. Conservative provider cost per completed meal was $0.00077.
Each photo contributes the mean of its two repeats. The interval resamples 34 participant groups, preserving their photos and repeats. Grouping by the source menu names instead gives a wider calorie interval of 30.5% to 64.3%, showing how repeated menus affect uncertainty. The single failure came from an upstream rate limit during nutrition estimation; it was not retried away. These observations do not establish future service reliability.
What these labels can and cannot tell us
The weights are measured. Calories and macros come from food tables and prepared-dish labels, not laboratory assays. Some lists omit visibly plausible components. Eighteen confirmation photos have source calorie values that differ by more than 25% from a simple 4/4/9 macro calculation; that is a warning flag, not an automatic correction or exclusion.
We retain those photos in the main result. Removing those 18 pre-flagged photos raises calorie agreement to 56.0% across the remaining 200 attempts, still counting the failed scan. This is a sensitivity check, not a replacement headline. The median estimated mass was 85.6% of measured mass. Whole-meal mass and calorie-density errors can cancel, so a close calorie total alone does not prove that the ingredients or portions are correct.
What changes for the product
The corrected food matcher is live. The model and prices stay the same. We are keeping portion estimation, visible versus inferred cooking fats, and ambiguous reference recipes as explicit follow-up questions. A blanket portion multiplier or a more expensive model has not earned its place.
For someone logging food, review the identified ingredients and portions, especially mixed dishes and cooking fats. For developers, the API exposes the model, matcher version and food nutrition sources so an estimate can be traced and corrected.
Results and method
Download the aggregate results and sensitivity analyses and frozen protocol, amendments and model decision. All model replies are cached internally with exact request, model, provider, image and per-food provenance; rescoring does not require another paid recognition call.
KcalSnap’s own product evaluation, with no private customer photos. Claude Opus 5.5 independently challenged the code, method and interpretation; KcalSnap owns the decisions. Sources: DFood-TJ dataset and dataset paper. Earlier: our Nutrition5k comparison.