Jev, Gemini Flash Lite or Haiku: which fast model can triage a French inbox?

I built a fake inbox for a sales rep at a French SaaS company and asked three fast AI models, Jev, Gemini Flash Lite and Haiku, to triage it. The aim was to find out which one a sales or support team could rely on, at what price, and how much the context and rules around the model change the result.

Why test it in French

Sales and support teams often receive more email than they can read carefully. Part of every morning goes into working out what needs an answer today, what needs an action and what can wait, and the same problem exists on a smaller scale in anyone’s personal inbox.

Sorting is the first step, and it is the one that suits a fast model. A more capable model can then gather the context and draft the reply for the emails that matter, once something quicker has decided which ones those are. At the volume a support team handles, that first pass runs on every single email, so its price per call and its response time matter as much as its accuracy.

A team in France, Germany or Spain that wants to deploy this kind of tool needs to know how it copes with its own language, its phrasing and the way it writes dates.

Three fast models

Jev has drawn a lot of attention recently. It is the first model from TypeSafe, built for one job: making quick, narrow decisions that software can use directly. You give it a piece of text and a question with a fixed list of possible answers, and it picks one and returns a probability for each option.

TypeSafe’s own documentation is clear about where it stands. English is its main training language, and other languages are handled “but not equally well”, with an explicit recommendation to test on your own content first. A separate page on its known weak points lists, among others, reading instructions very literally and comparing dates, two things email triage depends on.

The other two models I compared Jev against are the fast, low-price versions of general-purpose models: Gemini 3.5 Flash Lite from Google and Claude Haiku 4.5 from Anthropic. They write text by design, so to keep the comparison fair I made them answer in the same fixed format as Jev, with the same list of choices.

The test: a fake inbox

I built an inbox of 239 fake emails for an imaginary sales rep at a French SaaS company: client questions, prospects chasing a proposal, internal requests, invoices, newsletters, cold outreach and a few phishing attempts. I fixed the expected answer for each email first. A model from OpenAI then wrote the email, and a second one read it without seeing the answer to check it led to the same answer. Using a provider outside the three under test meant none of them sorted text written in its own style, and I settled every disagreement by hand.

Each model had to make five decisions per email:

• Type: a conversation, an automatic notification, a newsletter or unsolicited prospecting
• Expected action: reply, act (pay, sign, fill in a form) or nothing to do
• Urgency: today, within 48 hours, this week, or it can wait
• Deadline: the date by which something has to be done, if there is one
• Suspicious: whether it looks like phishing or a scam

Some emails were written to be hard on purpose: a follow-up that only says “I’m following up on my message below”, with the actual request further down the thread, a deadline given as “before our committee on Thursday morning”, an “urgent” subject line on something that can wait, or a message where the rep is only in copy. Forty of them ask for something without giving any deadline, and I set their urgency myself from reading the text, because that is the case where a model has to judge on its own.

Every email was sorted twice by each model, once with the email alone and once with the context a CRM would provide: a short profile of the rep and a record for each known contact, with the relationship, the account and any deal in progress. I used twelve emails to adjust the wording of the questions, so the scores below cover the other 227.

Results

With the CRM context and a few sorting rules applied in code (more on both in the next section), the three models land within ten points of each other on accuracy and far apart on price and speed.

An email counts as sorted correctly only when all five decisions are right. Response time is the time within which 95% of answers came back.

Gemini sorts the most emails correctly. Jev costs a quarter as much as Gemini and answers almost instantly, which is what it was built for. Haiku is the most expensive and by far the slowest, for a score in between the other two.

Context and code do most of the work

Asked to sort the emails on their own, the three models get between 61% and 66% of them right. Two additions take them to between 81% and 90%.

The first is context. Knowing that the sender is a client with a deal about to close, or a prospect with no deal attached, changes what an email asks for and how soon it needs an answer. Overall, Gemini gains the most from it, Jev a little and Haiku nothing at all. The gap is widest on the forty requests with no deadline, where the share of urgencies Gemini gets right doubles from 38% to 75%.

The second is a handful of rules applied in code once the model has answered, identical for all three:

• a thread the rep has already answered needs nothing more
• a suspicious email needs nothing, however urgent it claims to be
• urgency follows from the deadline: today or tomorrow means today, two days means within 48 hours, up to a week means this week
• an email that needs nothing can wait

In the end, each email goes through the same four steps:

Email + CRM context     rep profile, contact records
        ↓
Model's raw triage      five decisions
        ↓
Rules in code           already answered, suspicious, urgency from the deadline
        ↓
Final triage            what the scores measure

Each rule in code only draws the consequence of something the model has already said or the inbox already shows, and together they add between 14 and 22 points depending on the model.

Writing them is where most of the time should go, and it has little to do with AI. I wrote these quickly for the test. For a real team, it means understanding how salespeople actually handle their inbox, what counts as urgent for them and what they can ignore, and turning that into one shared definition that every model is measured against.

Where each model falls short

In its final triage, Jev still asks for a reply too often. On 19 emails that need nothing, it tells the rep to answer, most of them cold outreach ending with a polite “if you are interested”. This fits the literal reading its documentation warns about, where a question at the end of an email reads as a request. The error is visible, and one more rule could probably remove most of it.

Haiku invents deadlines. On 9 of the 40 requests with no date, it gives one anyway, and most of the time the urgency that follows is wrong too. An invented deadline is the error a sales team forgives least, because it pushes the wrong email to the top of the list with a date that looks authoritative, and a team that catches a tool making things up stops trusting its other answers.

Phishing detection held up well: of the 10 attempts, Gemini caught all of them and Jev and Haiku missed two each.

Vague emails are hard for all three. When an email states its deadline, the models find it in almost every case and the code derives the urgency. When it gives none, or leaves the request in an earlier message of the thread, the model has to judge the urgency on its own, and that is where most of the remaining urgency errors come from. On the forty requests with no deadline, Jev and Gemini get 75% of urgencies right, Haiku 57%.

What this test can and cannot tell you

The final scores rest on one complete pass per model. Earlier runs on previous versions of the inbox gave similar results, but the scores may still vary from one run to another. The set-up weighs as much as the model. Without context and rules, Jev came first and Gemini last, and with them the order reversed.

What this test offers instead is a setting closer to real use. Public benchmarks measure general abilities, mostly in English. This one measures a single business task, in French, with the context a CRM holds and the triage rules a sales team would apply. The numbers are best read as a trend, and any team should run the same kind of test on its own emails before choosing.

Which one I would pick

Gemini Flash Lite is the one I would start with. It sorts the most emails correctly, gains the most from context and rules, and its price and speed are low enough to run on every incoming email before a more capable model prepares the reply.

Jev becomes the better choice when volume makes cost and speed critical, at a quarter of Gemini’s price and a third of a second per email. It would need more work on its questions and context first, starting with the replies it asks for too often.

The two can also work together. Taking the type, the action and the phishing check from Gemini, and the urgency and deadline from Jev, sorts 92% of emails correctly, the best result of the test, for $0.72 per 1,000 emails. It also avoids depending on a single model.

Haiku is the one I would leave aside: the most expensive, the slowest, and the only one that makes up deadlines.

The next step is a round on my own inbox, with the models’ names hidden while I judge their answers. Any team working outside English should do the same: test in its own language, on its own emails, before trusting a ranking, this one included.