Fine-tuning, RL, and what actually works.

A chatbot is tuned to produce a good single response. An agent is tuned to complete multi-step tasks: choosing tools, passing correct arguments, recovering from errors, and knowing when it is done. That means training data is made of full trajectories rather than isolated prompt-response pairs, and success is measured by whether the task finished correctly - not by how fluent one message sounds. Agent fine-tuning therefore leans heavily on simulated environments and outcome-based rewards.

Parameter-efficient methods like LoRA and QLoRA train a small set of adapter weights on top of a frozen base model. They are cheaper, faster, and let you serve many specialized adapters from one base model, which makes them the default for most enterprise workloads. Full fine-tuning updates every weight and can squeeze out extra quality on large, stable datasets, but costs more to train, store, and serve. Start with LoRA and move to full fine-tuning only when evaluation shows a measurable gap.

Continual learning means an agent keeps improving after deployment instead of staying frozen at the version you shipped. Production signals - failed tasks, user corrections, new tools, changing data - are captured, turned into new training and evaluation cases, and used to produce better model versions on a regular cadence. PerceptEye runs this as a closed loop: Scout discovers new workflows and failures, Compass turns them into training-ready data and success signals, and Maestro trains and promotes improved models.

A model frozen at ship date decays as data, tools, and workflows change. PerceptEye treats improvement as a continuous loop: production signals feed back into simulation and re-training, every candidate is scored against a stable evaluation before promotion, and prior capabilities are protected by mixing earlier tasks back into training and gating releases on regression checks, so the model adapts without silently losing skills it already had.

Retrain on evidence, not on a calendar. Trigger a new training cycle when evaluation shows a drop in task success, when a new tool or workflow is introduced, or when enough new failure cases have accumulated to move the needle. For most enterprise agents that lands between weekly and monthly. What matters is that every retrained candidate is compared against the current production model on the same held-out evaluation before it is promoted.

Reinforcement learning (RL) trains an agent by letting it attempt tasks, scoring each attempt with a reward, and updating the model to make high-reward behavior more likely. For agents, the reward usually reflects whether the task was completed correctly, efficiently, and safely. RL is what lets an agent learn strategies nobody wrote down - better tool sequences, smarter recovery from errors - which is why it has become the core of modern agent post-training.

SFT teaches a model to imitate labeled examples and is the fastest way to lock in format, tone, and known-correct behavior. RL optimizes against a reward signal instead of fixed labels, so the model can discover better strategies for goals that are easy to score but hard to demonstrate, such as passing a test suite or resolving a ticket. Modern post-training runs SFT first to establish a baseline, then applies RL (RLHF, RLAIF, or verifiable-reward methods like GRPO) to push quality beyond what demonstrations alone can reach.

PPO is the classic RL algorithm used in RLHF; it is powerful but needs a separate value model and careful tuning. DPO skips RL altogether and learns directly from pairs of preferred and rejected responses, which makes it simple and stable for style and preference alignment. GRPO scores a group of sampled answers against each other, removing the value model and pairing naturally with verifiable rewards. For agent tasks with checkable outcomes, GRPO-style RL is usually the best fit; DPO works well for tone and preference polish.

RLVR replaces a learned human-preference model with an automatic, checkable reward: did the code compile, did the tests pass, did the answer match ground truth, did the tool call succeed. Because the reward is deterministic, training is cheaper and far less prone to reward hacking than preference-based RLHF. It fits agent workflows especially well, where success is often objectively measurable, and it is why PerceptEye scores every training iteration against a simulation that provides consistent ground truth.

Use RLVR whenever success can be checked automatically - structured outputs, code, calculations, API workflows, data extraction against a known answer. It scales without human labelers and resists reward hacking because the reward cannot be flattered. RLHF (or RLAIF) remains useful for subjective qualities such as tone, helpfulness, and judgment calls where no ground truth exists. Many production pipelines use RLVR for correctness and a lighter preference stage for style.

Reward hacking happens when a model finds a way to score well on the reward without actually doing the task - for example, special-casing tests instead of fixing code. Prevention comes from rewards that check real outcomes, hidden held-out evaluations the model never trains on, penalties for shortcuts like excessive length or skipped steps, and human review of samples from every training run. Simulation environments with ground truth make these checks systematic rather than ad hoc.

On-policy distillation (OPD) trains a smaller student model on trajectories it generates itself, while a stronger teacher model grades every token of those attempts. Classic distillation copies the teacher's outputs, so the student never learns to recover from its own mistakes. OPD fixes that: the student learns in the situations it actually gets into, and it gets dense token-level feedback instead of the single pass/fail reward used in RL. The result is RL-like robustness at a fraction of the compute, which makes OPD a strong way to compress frontier-level behavior into a small, cheap custom model for an agent.

On-policy self-distillation (OPSD) removes the need for a separate, larger teacher. The same model plays both roles: as the teacher, it sees privileged information such as the reference solution or ground-truth answer; as the student, it sees only the original problem. The student generates its own attempts, and the informed teacher version scores each token, so the model learns to reach answers it could previously only justify with the answer in hand. OPSD is useful when you have verified solutions but no bigger model to distill from, or when a teacher model's license, cost, or data-residency rules put it off limits.

Self-distillation policy optimization (SDPO) is a reinforcement learning method that learns from rich feedback rather than a single scalar reward. After the model attempts a task, the environment returns detailed feedback such as error messages, failing test output, or tool responses. The model, conditioned on that feedback, acts as its own teacher and re-scores its original attempt token by token, and those scores become a dense training signal. SDPO fits agents well because agent environments naturally produce this kind of feedback, and it can learn faster than reward-only methods like GRPO while needing no external teacher model.

All three give the model dense, token-level feedback on its own attempts. The difference is where the teacher's knowledge comes from. OPD uses a stronger external teacher model, which is best when you want to compress a large model into a small one. OPSD uses the model itself conditioned on reference solutions, which is best when you have verified answers but no suitable teacher. SDPO uses the model itself conditioned on environment feedback, which is best for agents whose tools, tests, and APIs return detailed errors. In practice they combine well with SFT and RLVR, and Maestro picks the approach based on the data and signals available for each workflow.

They solve different problems. Retrieval-augmented generation (RAG) gives a model access to fresh or proprietary facts at query time, so it is the right tool when knowledge changes often or must be cited. Fine-tuning changes how the model behaves - its skills, format, reasoning style, and tool use - so it is the right tool when the model needs to perform a task better, faster, or cheaper. Most mature enterprise agents use both: RAG for what the model should know, fine-tuning for how it should act.

Yes, and they work better together. A model fine-tuned on your retrieval format learns to read retrieved documents more carefully, ignore irrelevant passages, and cite sources correctly - a pattern often called retrieval-aware fine-tuning. It also needs shorter prompts, which lowers cost and latency. The retrieval layer keeps knowledge current, while fine-tuning makes the model reliably good at using it.

Prompt engineering is the fastest way to start and should always come first. It stops being enough when prompts grow long and brittle, when quality varies from call to call, when you are paying for thousands of instruction tokens on every request, or when accuracy plateaus below what the business needs. At that point, fine-tuning bakes the instructions and examples into the model itself - producing more consistent results with shorter prompts at lower cost.

Prompting and RAG are the right first step when the base model already has the capability and just needs context or instructions. Fine-tuning earns its place once you have a repetitive, high-volume workflow where you need consistent format and behavior, lower latency and cost, or accuracy the base model cannot reach through context alone. In practice the strongest systems combine all three: RAG supplies fresh facts, prompting sets the task, and fine-tuning bakes in the skill and style so you spend fewer tokens getting there.

A custom AI model is a model your organization owns and controls - typically an open-weight base model fine-tuned on your data and deployed in your own cloud, VPC, or on-premise infrastructure. Your prompts and data never leave your environment, you decide when the model changes, and you are not exposed to a vendor's pricing, rate limits, or deprecation schedule.

Calling frontier APIs directly for repetitive, knowledge-intensive workflows is expensive, slow, leaks data, and produces unpredictable quality. A custom model trained with PerceptEye runs where you choose, keeps your data in your environment, and can match frontier-grade accuracy on your tasks at up to 100x lower inference cost.

Yes. The resulting custom model and its weights belong to your team, and it can be deployed in your cloud, VPC, or on-premise so sensitive data never leaves your environment. PerceptEye is designed to slot into the serving stack you already run, which keeps you free of vendor lock-in and lets security and compliance teams retain full control.

They make compliance far simpler. Because the model runs inside infrastructure you already govern, regulated data such as PHI or personal data never leaves your controlled environment, audit logs stay in your systems, and data residency is guaranteed by where you deploy. PerceptEye adds cyber-aware fine-tuning so compliance and resilience become learned behaviors of the model rather than rules bolted on afterward.

On a specific, well-defined workflow, yes - routinely. Frontier models are generalists optimized for breadth; a 7B to 70B parameter model fine-tuned on your task concentrates all of its capacity on the work you actually do. The result is often equal or better task accuracy at a fraction of the cost and latency. Frontier models still win on open-ended, novel reasoning, which is why many teams route the long tail of unusual requests to a frontier API and handle the high-volume core with a custom model.

Public leaderboards like MMLU, GPQA, and SWE-bench measure broad general capability, so large frontier models usually top them. On a narrow enterprise workflow, a well-tuned smaller model routinely matches or beats a frontier API on task-specific accuracy while running far cheaper and faster. The benchmark that matters is a held-out evaluation built from your own data and success criteria, which is why PerceptEye optimizes against a task-specific eval instead of generic leaderboard scores.

Pick the smallest open-weight model that clears your quality bar after tuning. Shortlist a few candidates across sizes and families, fine-tune each on the same data, and compare them on your held-out evaluation alongside serving cost and latency. Licensing, context length, and tool-calling support matter as much as raw benchmark scores. Maestro automates this comparison so the choice is driven by measured results instead of leaderboard reputation.

Evaluate the outcome, not just the text. A strong agent evaluation runs the agent end-to-end on a representative, held-out set of tasks inside a realistic or simulated environment and checks whether each task was completed correctly. It also tracks the number of steps and tool calls, cost and latency per task, safety and refusal behavior, and regressions against the previous version. Combine automatic checks with periodic human review for anything subjective.

Generic accuracy is rarely enough. The metrics that predict production success are task completion rate on a representative held-out set, cost and latency per resolved task, calibration and refusal behavior on out-of-scope inputs, and regression against your previous model. Tie each of these to a business outcome - tickets resolved, documents processed, dollars saved - and gate every model promotion on the same stable evaluation so improvements are provable rather than anecdotal.

It is useful but should not stand alone. An LLM judge scales well for subjective qualities like helpfulness or tone, yet it can be biased toward longer answers, its own writing style, or confident-sounding mistakes. Use deterministic checks wherever a ground truth exists, calibrate any LLM judge against a sample of human ratings, and keep the judging prompt and model fixed so scores stay comparable across model versions.

Less than most teams expect. A few hundred to a few thousand high-quality, representative examples is often enough for supervised fine-tuning, because coverage of edge cases matters more than raw volume. When labeled data is scarce, simulation and synthetic generation expand coverage and RL against a reward function reduces the need for hand-labeled outputs entirely. Start with the data you already have, measure against a real evaluation, and add targeted examples only where the model is weak.

Quantization stores a model's weights, and sometimes its activations and KV cache, at lower numeric precision - for example 8-bit or 4-bit instead of 16-bit. A 4-bit model needs roughly a quarter of the memory of its 16-bit version, so it fits on smaller or fewer GPUs, serves more concurrent requests, and generates tokens faster. The trade-off is a small loss of accuracy, which is usually negligible at 8-bit and needs to be measured at 4-bit and below.

INT8 and FP8 are 8-bit formats; FP8 is natively accelerated on NVIDIA Hopper and newer GPUs and is close to lossless for most models. INT4 halves memory again and is the common choice when cost or hardware is tight, with 4-bit floating-point formats such as NVFP4 and MXFP4 supported on the latest GPU generations. GPTQ and AWQ are popular post-training methods for producing accurate 4-bit weights, using calibration data to decide how to round each weight. GGUF is a file format used by llama.cpp for running quantized models on CPUs, laptops, and edge devices.

Usually only slightly. 8-bit quantization is close to lossless on most tasks, while 4-bit typically costs a small amount of quality that varies by model and method. Multi-step reasoning, long contexts, and precise tool calling tend to be the most sensitive, so a model that looks fine on a quick chat test can still slip on agent workflows. Measure the quantized model on the same held-out task evaluation as the full-precision version, compare task success, cost, and latency together, and ship the lowest precision that still clears your quality bar.

Post-training quantization (PTQ) compresses an already-trained model in minutes to hours using a small calibration dataset, and it is the default starting point. Quantization-aware training (QAT) simulates low precision during training so the model learns to compensate for rounding errors, which recovers most of the accuracy lost at aggressive precisions like 4-bit. Start with PTQ, and switch to QAT only when evaluation shows an accuracy gap you cannot accept.

Generally after. Fine-tune in higher precision, or use QLoRA, which trains LoRA adapters on top of a 4-bit base model to cut training memory, and then merge and quantize the final model for serving. Always re-run your task evaluation on the quantized artifact you will actually deploy, because the fine-tuned model and its quantized version can behave differently. If the quantized model underperforms, quantization-aware training or a higher-precision format usually closes the gap.

MMLU and MMLU-Pro test broad knowledge across academic subjects. GPQA Diamond tests graduate-level science reasoning. AIME and MATH test competition mathematics. HumanEval and LiveCodeBench test code generation, and SWE-bench Verified tests whether a model can resolve real GitHub issues in real repositories. IFEval measures how well a model follows explicit formatting instructions, while Humanity's Last Exam and ARC-AGI probe the frontier of expert knowledge and novel reasoning. Each measures one slice of general capability, so no single score describes how a model will perform on your work.

Agent benchmarks measure whether a model can complete multi-step tasks, not just answer questions. τ-bench (tau-bench) simulates customer-service conversations with tools and policies. The Berkeley Function Calling Leaderboard (BFCL) measures tool and function-call accuracy. SWE-bench Verified and Terminal-Bench test software engineering in real codebases and shells, WebArena tests web navigation, OSWorld tests operating a desktop computer, and GAIA tests general assistant tasks that combine browsing, reasoning, and tools. They are useful for shortlisting base models, but your own simulated workflows remain the benchmark that decides what ships.

Three reasons. Contamination: benchmark questions leak into training data, so high scores can reflect memorization. Saturation: top models cluster within a few points of each other on popular leaderboards, so the ranking says little about your task. Distribution mismatch: your documents, tools, edge cases, and success criteria look nothing like academic test sets. Treat public benchmarks as a filter for picking candidates, then decide using a private, held-out evaluation built from your own workflows.

pass@k is the probability that at least one of k attempts succeeds, which suits settings where you can generate several candidates and verify them, such as code with tests. pass^k is the probability that all k attempts succeed, and it measures consistency - the question that matters when an agent handles the same kind of request thousands of times a day. A model can score well on pass@k while dropping sharply on pass^k, which reveals an agent that sometimes works rather than one that reliably works.

Use them for tasks with discrete right answers, such as classification, entity extraction, document routing, or flagging risky content. Precision is the share of the model's positive answers that are correct; recall is the share of all true positives the model found; F1 is their harmonic mean. Choose based on the cost of each mistake: favor recall when missing a case is expensive, as in fraud or clinical flags, and favor precision when false alarms are expensive, as in automated actions or customer-facing messages.

Time to first token (TTFT) is how long a user waits before the response starts, driven mainly by prompt length and model size. Time per output token (TPOT), also called inter-token latency, is how fast the rest of the response streams. Throughput is the total tokens or requests a deployment can serve per second, which determines cost per task at scale. Track p50 and p95 values rather than averages, and for agents measure end-to-end time per completed task, because one task may involve many model calls.

In narrow roles. Perplexity measures how well a model predicts text and is handy for catching training problems or quantization damage, but a lower perplexity does not guarantee better task results. BLEU and ROUGE measure word overlap with a reference answer and suit translation or summarization regression checks, yet they penalize correct answers phrased differently. For agents, prefer outcome metrics - task success, exact match on structured fields, and verified tool results - and use overlap metrics only as supporting signals.

Hallucination rate is the share of answers containing claims not supported by the source data or ground truth, measured with grounded test sets where the correct facts are known and, for retrieval systems, by checking that each claim is backed by a retrieved passage. Calibration measures whether the model's confidence matches its accuracy, often summarized as expected calibration error (ECE). Also track abstention: a well-calibrated agent should decline or escalate out-of-scope requests instead of guessing, which matters more in regulated workflows than a small gain in raw accuracy.

Agents are expensive because a single task can trigger many model calls, each carrying long system prompts, tool definitions, and growing context. The biggest levers are running a smaller fine-tuned model instead of a frontier API, shortening prompts by baking instructions into the weights, reducing the number of steps per task through better training, and caching repeated context. Together these routinely cut inference cost by an order of magnitude or more while keeping task accuracy intact.

Compare cost per successfully completed task, not cost per token. Multiply your monthly task volume by the average tokens and calls per task for each option, factor in success rate (a failed task still costs money and often requires human cleanup), and add one-time training plus ongoing retraining costs for the custom model. For high-volume workflows, the break-even point usually arrives within weeks. PerceptEye's ROI calculator models this using your own volumes.

Yes. Fewer parameters mean faster token generation, and shorter prompts mean less time spent processing input. Fine-tuned agents also tend to finish tasks in fewer steps because they have learned the workflow. The combined effect is often a several-fold reduction in end-to-end task time, which matters as much as cost for customer-facing and real-time agents.