Model choice is not a branding decision. A useful application knows which task needs deep reasoning, which needs a fast classification, how much context it sends, and what it will do when a request is too large or expensive. Those choices should be written down and measured against representative work.
This lesson builds a small routing policy. It counts a request before sending it, chooses a model tier from a documented task profile, and records the budget decision. The goal is not to chase the cheapest answer; it is to make quality, latency, and spend visible trade-offs.
| Signal | Question | Evidence |
|---|---|---|
| Task risk | What happens if this answer is wrong? | A clear review path for high-impact outcomes. |
| Quality | Does the route pass representative cases? | A labelled evaluation set, not a demo. |
| Latency | Can the user wait for this response? | Measured p50 and p95 timing from your application. |
| Cost | Does the input and output fit the product budget? | Token counts before launch and actual usage after it. |
Start with the work, not a model name. A short queue classification, a policy explanation with citations, and a long document analysis have different accuracy, latency, and context requirements. Write the expected output, the cost of a wrong answer, and the evaluation examples before choosing a default.
A route is defensible when a teammate can see the task profile and reproduce why a request was sent to a particular model.
Count tokens before expensive or context-heavy calls. The count helps you reject oversized uploads, warn a user, choose a smaller context window, or split a document before the model request begins. It also makes it easier to separate input growth from model quality problems.
Token counts are planning data. They do not replace usage telemetry from completed requests.
Budgets should cover more than a maximum output length. Put boundaries around input tokens, output tokens, retries, concurrent work, and daily spend. Each boundary needs a useful fallback: shorten a context, queue a batch, request a narrower question, or send the task to a human.
Never silently cut evidence that is needed for a safe decision. Prefer an explicit insufficient-context response.
Escalation is a product rule, not a vague instinct. For example, route a simple classification to the default model, but escalate only when the confidence check, source requirements, or evaluation case justifies it. Measure the benefit of escalation against the extra time and cost.
Keep route reasons in redacted telemetry so a later regression can be investigated.
Model selection is a product decision, not a preference for the most capable name. Start with the quality threshold for the task, then measure latency, cost, output length, and failure behavior on the same evaluation cases. A short ticket classifier and a long document analysis can need different routes.
Use token counting before expensive requests when prompt length, documents, tools, or a new model rollout could change your budget. Keep a record of the model identifier, input estimate, output cap, and observed usage so a cost surprise is something you can explain and prevent.
The count endpoint lets a server decide whether the assembled context fits its policy before it asks a model to respond.
from anthropic import Anthropic
client = Anthropic()
count = client.messages.count_tokens(
model="claude-sonnet-4-20250514",
max_tokens=300,
messages=[{"role": "user", "content": "Summarize this approved policy excerpt."}],
)
print(count.input_tokens)A small decision record makes a budget rule reviewable in tests and telemetry.
def choose_route(input_tokens: int, needs_citations: bool) -> dict:
if input_tokens > 8_000:
return {"status": "needs_narrower_context"}
if needs_citations:
return {"status": "send", "tier": "evidence", "max_output_tokens": 500}
return {"status": "send", "tier": "default", "max_output_tokens": 180}
print(choose_route(640, False)){'status': 'send', 'tier': 'default', 'max_output_tokens': 180}The token-count endpoint accepts the same kind of request structure as a message. Use it to set a budget policy before a long request is created.
import os
from anthropic import Anthropic
client = Anthropic()
count = client.messages.count_tokens(
model=os.environ["CLAUDE_MODEL"],
system="Classify support tickets into a small set of queues.",
messages=[{"role": "user", "content": "The invoice shows an extra charge."}],
)
print(count.input_tokens)A small routing policy makes an oversized request visible before it reaches a more expensive model path.
def choose_route(input_tokens: int, risk: str) -> str:
if risk == "high":
return "reviewed-model-path"
if input_tokens > 8_000:
return "summarize-then-classify"
return "fast-classification-path"
print(choose_route(420, "normal"))
print(choose_route(9_500, "normal"))fast-classification-path
summarize-then-classifyA product economics lab
0 of 2 completed
No. Use the smallest route that meets the evaluated task requirement, then escalate for explicit reasons.
No. It plans input size; completed usage, output length, retries, and current pricing still matter.
Explore 500+ free tutorials across 20+ languages and frameworks.