Skip to content

Agents Streaming

Agents exchange a continuous flow of tasks, results and events, not just single request/response calls. This page covers the streams an agent reads from and writes to. You can use them to show an agent's work live in a UI, feed it into pipelines, or let agents react to each other while they work.

Stream Direction Section
Tasks in Queue, WebSocket, P2P mesh → agent Input Streams
Results out Agent → exchange, sending agent or Redis Output Routing
Live task stream Agent → UI or consumer, while the task runs: tokens, tool calls, tool results, progress Live Task Streams
LLM token streaming Model → agent Streaming LLM Tokens and Tool Calls
Events Agent ↔ agent over P2P meshes Event Streaming
Conversations User ↔ agent Message Streaming

Input Streams

An agent instance started with main(AgentClass) listens on all of these at once:

Stream Transport Default Used by
Task queue Redis list with acknowledged delivery (see Fault Tolerance) REDIS_URL=redis://localhost:6379/0, key REDIS_STREAM=jobs:inbox The agent executor's load balancer. This is the production path
WebSocket JSON over WebSocket, synchronous replies WS_PORT=8765, optional WS_BEARER_TOKEN Direct clients, UIs, local testing
P2P mesh NATS, meshes from MESH_LIST — Tasks and events from other agents

The WebSocket accepts one task ({"task": {...}}) or a batch ({"tasks": [...]}) per message and answers with {"results": [...]} on the same connection. See Testing Agents.


Output Routing

Results of queued tasks are sent wherever the task's output_ptr.type says:

output_ptr.type Destination Required fields
exchange The task exchange, through the background output reporter (with retries) exchange_id
mesh The sending agent over NATS (reply) internal_task_id, sender_subject_id
redis Any Redis list, with LPUSH host, port, output_queue
(none) Not routed. WebSocket tasks get their results on the connection —

Every result has the same body:

{ "task_id": "...", "job_output": {...}, "job_output_metadata": {...},
  "job_output_trace_data": {...}, "is_error": false, "error_data": {...} }

A Redis-type output_ptr is the simplest way to send an agent's results into your own pipeline:

"output_ptr": { "type": "redis", "host": "redis.my-app", "port": 6379, "output_queue": "agent-results" }

Live Task Streams

While a task runs, an agent can publish what it is doing (tokens, tool calls, tool results, progress and messages) so a UI or another service can follow along. Get a stream for the task from the context:

def on_data(self, task):
    stream = self.context.stream(task)

    stream.progress(10, "searching sources")
    hits = self.search(task.job_data["question"])
    stream.tool_call("search", {"q": task.job_data["question"]}, call_id="c1")
    stream.tool_result("c1", {"hits": len(hits)}, name="search")

    # forward LLM tokens and tool calls as they are generated
    final = stream.relay(self.llm.stream_chat_completions(messages=self.build_messages(hits)))

    stream.close(status="ok")
    return AgentResult(task_id=task.task_id, job_output={"answer": final["content"]})
Method Emits Data
token(text) token {"text"}
tool_call(name, arguments, call_id=None) tool_call {"id", "name", "arguments"}
tool_result(call_id, result, name=None, is_error=False) tool_result {"id", "name", "result", "is_error"}
progress(percent=None, message="", **extra) progress {"percent", "message", ...}
message(text, **extra) message {"text", ...}
emit(type, data) any custom type your data
relay(llm_events, forward_deltas=False) token / tool_call (and tool_call_delta if asked) Returns the final {"content", "tool_calls", "finish_reason"}
close(**summary) end your summary; later calls are ignored

Every event has the same envelope, with seq increasing per task so consumers can order and de-duplicate:

{ "task_id": "t1", "subject_id": "research-agent", "seq": 7, "ts": 1731744000.12,
  "type": "token", "data": { "text": "The " } }

Choosing Where Events Go

The caller chooses the destination by adding a stream entry to the task's output_ptr:

output_ptr.stream Delivery
{"type": "redis", "host": "...", "port": 6379, "channel": "ui-42"} Redis PUBLISH, for live subscribers
{"type": "redis", "host": "...", "port": 6379, "list": "events-42", "ttl": 3600} Redis RPUSH, kept for consumers that read later (optional TTL in seconds)
{"type": "mesh", "subject_id": "ui-agent"} P2P event task_stream to that agent, which receives it in on_event

channel and list can be combined. If the task has no stream target and the instance has STREAM_REDIS_URL set, events are published to the channel <STREAM_CHANNEL_PREFIX>:<task_id> (prefix default agent-stream). Otherwise the stream methods do nothing, so agent code can call them unconditionally. stream.enabled tells you whether a target is set.

Consuming a Stream

import json, redis

r = redis.Redis(host="redis.my-app")
sub = r.pubsub()
sub.subscribe("ui-42")
for msg in sub.listen():
    if msg["type"] != "message":
        continue
    ev = json.loads(msg["data"])
    if ev["type"] == "token":
        print(ev["data"]["text"], end="", flush=True)
    elif ev["type"] == "end":
        break

Streaming LLM Tokens and Tool Calls

The LLM interface (agents_llm) streams chat completions from any registered backend:

from agents_llm import AIGridClient
from agents_llm.custom import OpenAIBlockInferenceSystem

client = AIGridClient()
client.add_custom_block(name="openai", system=OpenAIBlockInferenceSystem(model="gpt-4o-mini"))

for ev in client.stream_chat_completions(
        name="openai",
        messages=[{"role": "user", "content": "What's the weather in Paris?"}],
        data={"tools": [WEATHER_TOOL]}):
    if ev.type == "token":
        print(ev.data["text"], end="")
    elif ev.type == "tool_call":
        print("\ncall", ev.data["name"], ev.data["arguments"])
    elif ev.type == "done":
        final = ev.data            # {"finish_reason", "content", "tool_calls"}
Event Data When
token {"text"} Each piece of generated text
tool_call_delta {"index", "id", "name", "arguments"} (partial JSON) Each fragment of a tool call, as it is generated
tool_call {"index", "id", "name", "arguments"} (complete JSON) Once per tool call, after generation completes
done {"finish_reason", "content", "tool_calls"} Last event

The OpenAI backend (and any OpenAI-compatible endpoint set through base_url) streams natively. Other backends return their complete answer as one token event followed by done, so one loop works with every model. A custom backend can stream natively by implementing stream_chat_completions(messages=..., data=...) and yielding agents_llm.streaming.StreamEvent objects.

To forward a model stream to a live task stream, pass it to stream.relay(...), as shown above. To report tool results as you execute the calls, use stream.tool_result(...).


Event Streaming

Besides tasks, agents can publish and receive events over their P2P meshes, for example progress notices, state changes or presence:

# from agent code, through the P2P manager on the context
self.context.p2p_manager.broadcast_event_sync("progress", {"task_id": tid, "percent": 40})
self.context.p2p_manager.send_event_sync("handoff_ready", {"task_id": tid}, subject_id="reviewer-agent")

# receiving side
class ReviewerAgent:
    def on_event(self, event_name, event_data):
        ...

See Chat and P2P Communication for meshes, events and presence.

Streaming Task Progress from Platform Services

Source Stream
Delegation, voting and verification NATS events. See Task Delegation, Social Choice & Voting
Verification status WebSocket updates. See Verification System
Your own long-running tasks Live task streams

Message Streaming

For conversational, message-by-message streaming between users and agents, use the Chat Server WebSocket protocol.