A U.S. enterprise IT managed-services provider gives its enterprise clients a portal with one view of all their services, plus an AI assistant they can question in plain language. Behind every answer the assistant gives sits one small decision. That decision ran on a general-purpose LLM: too expensive to run on every question, too unreliable to trust, and too slow for users to wait for.

With Plurai, the team turned that decision into a custom small language model (SLM) in 20 minutes. Three numbers changed the product:

• 97% lower inference cost than the live setup.
• 82% fewer failures and policy violations in the assistant's answers.
• 7x faster responses, from 1.0 second to 0.15 seconds.

The answers also became consistent. Ask the same question twice and you get the same answer, which frontier models often fail to deliver.

And the first SLM was only the start. Within 2 weeks, the team had built 100+ SLMs on Plurai, including a data-leakage guardrail, a prompt-injection detector and a tool router.

‍

The problem: a slow, costly decision behind every answer

When a client asks the assistant something, the first job is to work out which parts of the portal the question is about. The portal is organized into dashboard tiles covering the network, billing, orders, support and mobility. A question can point to one tile, several or none, and many tiles look alike.

Get the tiles wrong, and everything downstream is wrong too. And because this step runs before every answer, its cost and speed are paid on every single question.

‍

Three options, none good enough

There were three ways to improve that decision, and each came with a catch.

Keep the general-purpose LLM. Their live setup, Claude Haiku without reasoning, answered in 1.0s and cost $2.87 per 1,000 calls. But it resolved only 69% of requests.

Turn on reasoning. Reasoning-enabled models reached 92%. But each call then took 2.8–5.0 seconds and cost $2.44–$5.27 per 1,000 calls, which is too slow for a live conversation and too expensive for a step that runs on every question. Frontier models also don't give the same verdict every time; they flip between runs.

Build a custom model. A fine-tuned SLM can be accurate, fast and cheap. But building one meant a data science and ML team creating and curating thousands of labeled examples, training the model and optimizing it for inference. That is months of work and dedicated people for every model. And it is never a one-time project: an enterprise needs hundreds of these models, and each one has to be updated continuously.

‍

Enter Plurai vibe post-training: a custom SLM from a prompt or a few examples, in 20 minutes

Plurai removes the catch in the third option. With vibe post-training, any engineer can build a custom SLM from a prompt or about 10 examples, the same input they would write for a general-purpose LLM. Plurai does the rest: it generates the training data, fine-tunes the SLM in minutes and serves it in production. It works through Plurai's web UI or directly in the IDE with the Plurai Claude Code plugin. For this team, the first SLM went from task description to a served model in 20 minutes.

‍

How it works

Instead of asking one model to weigh every tile at once, Plurai breaks the decision into simple yes/no questions, one per tile. A small model answers each question, and a few lines of code combine the answers into the final set of tiles. Every answer can be traced back to the questions behind it.

To make the SLM robust, Plurai generates the rest of the training data with BARRED (ICML 2026), its published method for generating synthetic training data, including hard cases such as near-duplicate tiles. A strong frontier model labels the hard cases, but only while the SLM is being built. In production, every request runs on the SLM alone, served as a lightweight adapter on a shared 4B base model. Plurai's latest research on this approach will be published soon.

Plurai also optimizes how its models run in production. Techniques such as Activated LoRA (aLoRA), token compression and smart decoding let many task-specific SLMs share a single GPU without slowing down. Activated LoRA lets every adapter reuse the same cached view of the conversation, so each SLM processes only its own short task. In Plurai's paper Hundreds of Guardrails, One GPU, 100 adapters on one GPU evaluated a 5,000-token conversation in 285ms on average, 32x faster than standard LoRA (9,053ms).

‍

The results: three numbers that changed the product
‍

1. 97% lower cost

The SLM costs $0.076 per 1,000 calls, against $2.87 for the live setup. That is 97% less, or 38x cheaper. Reasoning-enabled frontier models resolve 3 points more requests than the SLM (92% vs. 89%), but at 32–69x its cost and 19–33x its latency.

Benchmark against the team's live setup, 5 models

The Plurai SLM resolves 89% of requests at $0.076 per 1,000 calls: 38x cheaper than the live setup (97% lower cost) and 69x cheaper than Claude Haiku with reasoning. Cost axis is log scale.

‍

2. More requests resolved, 82% fewer failures

Quality improved in two separate ways.

More requests resolved. The SLM resolves 89% of requests, picking the right tiles, against 69% for the live setup. That is 20 more requests in every 100 that the assistant can carry through to an answer. When the assistant starts from the right part of the portal, everything after it has a chance to be right too.

82% fewer failures and policy violations. Resolving more requests only helps if the answers can be trusted. Two kinds of bad answers matter here:

1. Failures: answers that are wrong. Examples: resolving the wrong entity or tiles (a question about an unpaid invoice answered from the orders dashboard), answering what isn't grounded in the client's data instead of declining, or returning aggregated numbers that don't add up.
2. Policy violations: answers that break the provider's rules. Examples: leaking internal tool names or schema in a response, outputs that violate the provider's policy on content, tone or language, or injected instructions in user input that reach the agent unflagged.

Against the live setup, these failures and violations dropped by 82%. The assistant now does more, and gets it wrong far less often.

‍

3. 7x faster

The SLM answers in 0.15 seconds, against 1.0 second for the live setup. Getting comparable accuracy from a general-purpose model would have meant turning on reasoning, at 2.8–5.0 seconds per decision. And that wait comes before the assistant even starts composing its answer.

‍

Beyond the numbers: consistent answers, run after run

Resolving more requests is one part of trust. Giving the same answer every time is another. A client who asks the same question twice and sees two different sets of tiles stops trusting the assistant, even if both answers are plausible.

Frontier models are not built for this. In Plurai's research on the same kind of tile-matching task, frontier models changed 26–37% of their answers across five identical runs. Plurai's approach changed 5–7%, so it flipped 5–7x fewer answers than gpt-5.5. Each answer is assembled from simple yes/no questions, so it can also be traced back to exactly why the assistant gave it.

Plurai research on a tile-matching task, five runs per model

‍

From one SLM to hundreds

The tile decision was the first SLM. Once the team saw that any engineer could build one in minutes, the approach spread across the assistant to the guardrail, routing and classification decisions that had all been running on a general-purpose LLM. Among the SLMs now in production:

• A data-leakage guardrail that blocks internal tool names and schema before a response goes out.
• A prompt-injection detector that flags injected instructions in user input before they reach the agent.
• A tool router that sends each request to the right tool, the same kind of decision as the tile resolver.

Plurai's serving stack keeps that scale affordable. Each SLM ships as a lightweight adapter on a shared base model, and Activated LoRA lets them all read the same conversation once. Hundreds of task-specific models run side by side on a single GPU, without the latency or cost of hundreds of separate deployments.

‍

What it changed for the product and its users

Speed was the difference between an assistant users waited for and one they used. A pause of one to five seconds before every answer is long enough for people to stop asking. At 0.15 seconds, the tile decision disappears from the user's wait, and the assistant answers at the pace of a conversation.

Speed got users to try the assistant; reliability and consistency got them to rely on it. Adoption of the assistant grew significantly across the provider's users [PLACEHOLDER: add a number, such as weekly active users or questions per user, or a customer quote].

Cost tells the same story at scale. At 1 million questions a day, for example, this one decision would drop from about $86,000 to about $2,300 a month. Across 100 decisions like it, that is the difference between roughly $8.6 million and $230,000 a month. More usage no longer means a matching jump in the AI bill.

‍

Key takeaways for enterprises building agents

This story isn't only about one assistant. It shows how enterprise agents get from prototype to production at a cost that scales.

1. An agent runs on hundreds of small decisions, not one big model. Routing, entity resolution, guardrails and classification sit behind every answer an agent gives. When each one runs on a general-purpose LLM, the whole agent becomes slower, more expensive and less consistent.
2. Purpose-built SLMs are the production layer for those decisions. Here, the SLM cut inference cost by 97%, failures by 82% and latency 7x against the general-purpose model it replaced. And it answers consistently, flipping 5–7x fewer answers than a frontier model.
3. Latency is an adoption metric. Every second a hidden decision adds is a second users wait before they see an answer. Cutting that step 7x is what turned this assistant into one people use.
4. Post-training is now an engineering task, not an ML project. What took a data science and ML team months per model now takes any engineer about 20 minutes, from a prompt or a few examples. That is what lets one SLM become hundreds, and keeps each one up to date as the agent evolves.
5. Scale doesn't multiply the infrastructure bill. Hundreds of SLMs share a single GPU as lightweight adapters. Adding the next guardrail or router takes minutes, not a new deployment.

The enterprises that win with agents won't be the ones calling the biggest model on every step. They'll be the ones that turn every high-volume decision into a fast, reliable, purpose-built model, and keep improving it.

Running a general-purpose LLM on narrow, high-volume decisions? See what they would cost as SLMs. 

‍

Get started

Vibe post-train any open-source model for your task in about 20 minutes at app.plurai.ai, or straight from your IDE with the Plurai Claude Code plugin. Want to see what your agent's decisions would cost as SLMs? Book a demo.

Further reading from Plurai:

• Hundreds of Guardrails, One GPU
• BARRED (ICML 2026)
• Lessons from deploying thousands of LoRA guardrails in production
• Best small language models (SLMs) for evaluation and guardrails