Learn
What is agent cost optimization?
Agent cost optimization balances model choice, context size, heartbeat frequency, and tool calls so roles deliver accepted work per dollar — not minimum tokens on a leaderboard.
In depth
Inference spend scales with window size, model tier, number of tool loops, and schedule frequency. Cost optimization targets cost per accepted task — the economic unit that matters to the business.
Levers on Neuro OS:
- Role-scoped budgets — caps per role per day/week (Budgets) - Model routing — smaller model for triage; frontier model for final draft only - Context discipline — RAG top-k, summarize history, skills in git instead of re-prompting procedures - Heartbeat tuning — don't poll every minute if hourly suffices - Receipts — identify roles that loop without shipping output
Cheap runs that humans reject are expensive. A slightly costlier run with 90% acceptance beats a penny run rewritten entirely.
Examples
- Triage with small model; escalation run uses larger model once
- Weekly finance brief instead of daily if anomalies are rare
- Cache retrieved KB chunks for repeated ticket types
- Budget alert when outbound role exceeds $50/day inference
- Kill switch on runaway tool loop after N steps
Related terms
FAQ
Should I always use the cheapest model?
No — optimize accepted output cost, not token price alone.
What is a healthy budget for one role?
Start from pilot volume × observed tokens per accepted task; add 30% buffer.
Run governed agent roles on a company OS — not only definitions in a glossary.