Streaming improves perceived responsiveness by delivering response events as Claude generates them. It does not make every workflow faster, and it adds UI and cancellation responsibilities. Design the interface to show partial text as provisional until a completed message and stop reason arrive.
Message Batches are suited to large asynchronous workloads. They use a different operational model from interactive requests: submit validated jobs, track status, process results idempotently, and plan for partial failures. Do not use an interactive request loop to process thousands of independent records.
| Need | Use | State you must keep |
|---|---|---|
| A customer is waiting | Streaming | Partial text, completion, cancellation, and a recoverable error. |
| A nightly backlog can wait | Message Batches | Request ID, item ID, processing status, result matching, and retry policy. |
| You are uncertain | A normal request first | A baseline for latency, response quality, and failure behavior. |
Streaming delivers a sequence of events rather than one final object. Buffer text by content block, render it safely, and handle the final event that establishes stop reason and usage. A network disconnect can leave the UI with partial text that must not be silently stored as a complete answer.
Stream over a server endpoint so your backend retains authentication, policy, model selection, and rate limiting.
A streaming UI should show that output is in progress and provide cancellation. Keep partial text in memory or a temporary record. Mark it complete only when the request finishes successfully; otherwise label it interrupted and offer retry or continue behavior.
Avoid triggering actions from partial model text. A stream is presentation, not a permission grant.
Batches are a throughput pattern. Create independent, validated items with stable IDs, submit them, inspect asynchronous results, and write each outcome exactly once. Store item status so a worker restart does not duplicate billing, emails, or database changes.
Batch pricing, retention, and availability are product details that should be verified against current official documentation before launch.
Cancellation is an application concern. Stop forwarding a stream when the user disconnects, abort downstream work where supported, and record final status. For batch consumers, use idempotency keys and state transitions to make retries safe.
Measure queue age, completion rate, failed item rate, and cost per completed artifact.
A streamed response needs a small state machine. The UI can append text while the message is active, but only a completed event should mark the answer ready for storage or reuse.
The same discipline applies to batches. A worker should know whether an item is queued, running, completed, or retryable before it acts on the result.
A simple state object prevents the UI or persistence layer from treating an interrupted stream as final.
state = {"text": "", "complete": False}
def on_text_delta(delta: str) -> None:
state["text"] += delta
def on_message_complete() -> None:
state["complete"] = True
on_text_delta("Hello")
on_text_delta(" world")
on_message_complete()
print(state){'text': 'Hello world', 'complete': True}A completed item is recorded once even if a worker receives the result twice.
processed_ids: set[str] = set()
def consume_result(item_id: str, result: str) -> str:
if item_id in processed_ids:
return "already_processed"
processed_ids.add(item_id)
return f"saved:{result}"
print(consume_result("job-1", "ok"))
print(consume_result("job-1", "ok"))saved:ok
already_processedThe state shown to the user should say that the partial reply did not finish.
def interrupted_state(text: str) -> dict:
return {"text": text, "complete": False, "status": "interrupted"}
print(interrupted_state("Partial answer")["status"])interruptedA small transition guard prevents a retry worker from overwriting an already completed item.
ALLOWED = {"queued": {"running"}, "running": {"succeeded", "retryable"}}
def may_transition(current: str, target: str) -> bool:
return target in ALLOWED.get(current, set())
print(may_transition("running", "succeeded"))TrueA product and operations exercise
0 of 2 completed
No. It changes delivery timing; control output with your prompt and max_tokens.
Only after the same validation, authorization, and idempotency checks used by interactive workflows.
Explore 500+ free tutorials across 20+ languages and frameworks.