Jev Images Tested Against Three Vision LLMs

The short answer

Jev 1.14 answered 2,158 questions about the screenshots and got none wrong. Claude Haiku 5.5 also got none wrong. Gemini 3.5 Flash Lite missed 8 and GPT-6 Luna missed 12. When the two of them were wrong, they were sure of it.

Jev took about half a second per screenshot, against 1.4 to 3.9 seconds for the chat models. It also cost the least: 18 cents per 1,000 screenshots, against 26 cents for Haiku and Luna and 48 for Flash Lite.

Three questions per screenshot, dense pages, 21 October 2026

Wrong answers count every page and every run. Time is the median on the dense pages. Cost is for new screenshots, without the discount one provider gave on repeat images.

On this job, reading text in a screenshot, Jev is as accurate as the best cheap vision model we tried. It is faster and costs less. The money is small for every model, though. Reading 10,000 screenshots costs $1.81 on Jev and $4.81 on the dearest model here.

Every earlier Jev test was on text

An AI search tracker asks the same few questions of every screenshot. Is my brand in the answer? Who is listed first? Is something covering the page? Each answer is closed: yes or no, or one of a short list. Jev was built for questions like these.

We already knew how Jev does on text. It held up on 108 claims checked against their sources, and it is 1.7 times faster than a cheap chat model on one decision. Images are new, and TypeSafe's own documentation says they are not yet up to its standards to call ready for production.

The one public test we found ran 30 photos through Jev 1.14 and 1.13. It compared the two versions with each other and with no other model. Its author found 1.14 matched fewer photos to their own headlines than 1.13 did (15 of 30 against 23). Nobody had put Jev beside a vision LLM on the same images.

How we built the test

We wrote 60 buyer questions in 12 categories, such as "best running shoes for beginners" and "best CRM for small business". A separate model, Gemini 2.5 Flash Lite, wrote an answer to each in the shape AI search uses. Each answer has a one-line intro, five to seven picks with a short reason, and a closing line. That model took no part in the test.

We built each answer into a search results page in Chrome and took a screenshot. Because we wrote the page, we know which brands are in the answer, which comes first and whether a pop-up is open. The answer key came from the page text. No person or model read the picture to make it.

Every screenshot got the same three questions, worded the same way for every model:

1. Does the AI answer name this brand? Count only the panel labelled AI answer, not ads, sponsored cards, sidebars or pop-ups.
2. Which brand does the AI answer list first? Pick one of four. One of the four is a brand from an ad.
3. Is a pop-up covering part of the page?

All three questions went in one call per screenshot. Each model answered with a probability. Jev returns one by design, and we asked the chat models for one. We counted an answer as right when it landed on the correct side of 50 percent.

Three kinds of page

Clean pages alone would not tell the models apart, so we made the pages harder in two steps.

On the dense and small-print pages, the brand we asked about sat in one of four places, split evenly. It was in the list of picks, inside another pick's reason, only in an ad, or nowhere on the page. We left Jev 1.13 off the small-print pages, because it had matched 1.14 on every answer up to then.

Where the models went wrong

Jev 1.14, Jev 1.13 and Claude Haiku 5.5 answered every question right on all three page types. Gemini 3.5 Flash Lite and GPT-6 Luna slipped on the dense pages, and Luna slipped more on the small print.

Screenshots with all three answers right

Clean pages ran once; dense and small-print pages ran three times each.

The 20 mistakes

• A brand named only inside another pick's reason, read as not named: Flash Lite 7 times, Luna 4 times.
• A sponsored card with a Shop now button, read as a pop-up: Luna 7 times.
• A brand that appeared only in an ad, read as named: Flash Lite once.
• A brand that was nowhere on the page, read as named: Luna once.

Every one of those 20 answers was confident. Each sat at 0, 0.99 or 1.0 on the wrong side. A threshold would not have caught them. Jev never had a wrong answer to be sure of. Nineteen in twenty of its answers sat at 0.98 or higher, and its least sure answer, at 0.84, was still right.

The gaps are small: 8 and 12 mistakes in 2,158 answers. On the dense pages the difference between Jev and Flash Lite held up when we resampled the screenshots. So did the gap to Luna on the small print. The other gaps could be chance.

A brand named in passing is still named

Most of the chat models' misses came from one sentence pattern. A pick's reason ended with "Often compared with Altra." Asked whether the answer named Altra, Flash Lite said no. Altra is not a pick, but the answer names it.

A tracker should count that mention. Being named beside a rival is a real presence in the answer, and a count that drops it under-reports you. It also shows that the wording of your question decides your number. Ask whether the answer names your brand, or whether it recommends it, and decide which you want before you start counting.

Speed per screenshot

Jev answered in about half a second on all three page types. The chat models took three to seven times as long.

Median time per screenshot, three questions in one call

Wall-clock time, including the network. Jev was called on TypeSafe's own endpoint. The chat models went through a third-party router, which adds to their times.

On the dense pages Jev was 2.8 times faster than Flash Lite, 5.4 times faster than Haiku and 7.4 times faster than Luna. For a tracker that runs overnight, none of these times matter much. A check that runs while someone waits will feel the gap between half a second and three.

Cost per screenshot

Models bill by the token, a small chunk of text or image. Jev counts more tokens per image than the chat models do, and still costs the least. It counted about 4,300 input tokens for a dense screenshot with three questions. The chat models counted 1,260 to 2,190 for the same image. Jev's price is $0.042 per million input tokens, and it does not charge for output.

Cost per 1,000 new screenshots, three questions each

First-pass prices. On our second and third runs, GPT-6 Luna's price fell to a fifth because the provider discounted the repeated image prompt. A tracker sends each screenshot once, so we priced every model on new images.

Jev is 1.4 times cheaper than Haiku and Luna and 2.7 times cheaper than Flash Lite. At any of these prices, the model is not where a tracker spends its money. Capturing the pages, storing them and running the browser cost far more than the 18 to 48 cents per 1,000 reads.

Is Jev 1.14 better at images than 1.13?

Not on our pages. We ran both versions on the clean and dense screenshots, 1,618 questions each. They gave the same answers and counted the same tokens for every image, so they cost the same. Jev 1.14 was 43 milliseconds faster at the median on clean pages and 20 on dense ones.

Box questions do need 1.14. A Box question asks where something is and returns its edges as fractions of the image. We tried one: we asked for the AI answer panel on a dense page. Each edge of the box came back within 1.2 percent of the panel's real edge, at a confidence of 0.77. One try is not a test, so we report it and nothing more.

A router can drop the image without an error

Send image calls to TypeSafe's own endpoint, with the header TypeSafe-Danger-Zone-Images set to true. Then check the token count on the first call. A screenshot should add thousands of input tokens. If it adds none, the model is answering blind.

What this test does not cover

Which one to use

If you are reading text in screenshots at volume, Jev 1.14 is the fastest and cheapest of the four, with no loss in accuracy on our pages. Claude Haiku 5.5 matched it on accuracy and is the one to fall back to when Jev is down. Gemini 3.5 Flash Lite and GPT-6 Luna were right more than 99 percent of the time, but their few misses came with full confidence.

One caution carries over from our text tests: a perfect score on one set of pages is a measurement of those pages. Before you gate anything on Jev, label 50 to 100 of your own screenshots by hand and count how often it agrees. When your labels and the model disagree, check the labels first. Ours were wrong once in this study.

Frequently Asked Questions

Keep reading