Testing Agents¶
Test an agent in three layers, from fastest to most realistic:
- Unit-test the agent class: call its hooks directly, with no platform services.
- Run the agent locally: start the real SDK runtime on your machine and send it tasks over WebSocket.
- Smoke-test the deployment: check the deployed executor with health and dry-run calls, then send a real task.
To judge the quality of an agent's output, rather than whether it runs, see Evaluation.
1. Unit-Testing the Agent Class¶
An agent class is a plain Python class (see Writing the Agent Code). Because on_preprocess and on_data take and return plain dataclasses, you can test them without Redis, NATS or a subjects service. Pass a stub in place of context and replace the parts your agent uses:
# test_uppercase_agent.py
from types import SimpleNamespace
from core.types import AgentTask
from agent import UppercaseAgent # your agent module; do not call main() at import time
def make_agent(delegate_reply=None):
context = SimpleNamespace(
subject_id="uppercase-test",
delegator=SimpleNamespace(submit_and_wait=lambda **kw: delegate_reply),
direct=SimpleNamespace(submit=lambda **kw: None),
p2p_manager=SimpleNamespace(send_and_wait_sync=lambda *a, **kw: delegate_reply),
)
return UppercaseAgent(subject=None, context=context)
def test_drops_task_without_text():
agent = make_agent()
assert agent.on_preprocess(AgentTask(task_id="t1", job_data={})) is None
def test_uppercases_text():
agent = make_agent()
[task] = agent.on_preprocess(AgentTask(task_id="t2", job_data={"text": "hi"}))
result = agent.on_data(task)
assert not result.is_error
assert result.job_output == {"text": "HI"}
Tip
Keep main(AgentClass) behind an if __name__ == "__main__": guard, so that importing the module in tests does not start the runtime.
2. Running the Agent Locally¶
main() runs the same runtime that runs in the cluster. It needs:
| Requirement | Why |
|---|---|
| A registered subject | The SDK loads the specification of SUBJECT_ID from SUBJECT_DB_URL at start-up. See Agent Registration |
AGENT_DELEGATE_URL |
Required. The SDK refuses to start without it. Any URL works if the agent does not delegate |
Redis on REDIS_URL (default redis://localhost:6379/0) |
The task queue worker runs in the main thread |
MESH_LIST (optional) |
NATS meshes to join. Without it, P2P is disabled and a warning is logged |
JOB_EXCHANGE_API_URL (optional) |
Needed only if results are reported to a task exchange |
docker run -d --name agent-redis -p 6379:6379 redis:7-alpine
export SUBJECT_ID="uppercase-001"
export SUBJECT_DB_URL="http://<subjects-service-host>:<port>"
export AGENT_DELEGATE_URL="http://<delegate-host>:<port>"
export LOG_LEVEL=DEBUG
python3 agent.py
Sending a Task over WebSocket¶
The runtime serves WebSocket on port 8765 (WS_PORT). Each message is processed synchronously and the results are sent back on the same connection:
import json
from websockets.sync.client import connect
with connect("ws://localhost:8765") as ws:
ws.send(json.dumps({"task": {"task_id": "t1", "job_data": {"text": "hello"}}}))
print(json.loads(ws.recv()))
# {"results": [{"task_id": "t1", "job_output": {"text": "HELLO"}, ..., "is_error": false,
# "origin_task_id": "t1"}]}
To send several tasks at once, use {"tasks": [ {...}, {...} ]}. If WS_BEARER_TOKEN is set, send the header Authorization: Bearer <token>.
Sending a Task through the Queue¶
This is the path the agent executor uses in production. Push a task onto the Redis list and watch the agent's logs:
redis-cli LPUSH jobs:inbox '{"task_id":"t2","job_data":{"text":"queued"},"output_ptr":{}}'
Results of queued tasks are routed by output_ptr (see Output Routing). With an empty output_ptr nothing is sent back; use "output_ptr": {"type": "redis", "host": "localhost", "port": 6379, "output_queue": "results"} and redis-cli BRPOP results 0 to see the result.
3. Smoke-Testing a Deployed Agent¶
Find the executor URL in the agent's runtime subject (runtime_info.executor, see Agent Deployment), then:
EXECUTOR=http://<executor-host>:<node-port>
# 1. Is the executor up?
curl -s $EXECUTOR/healthz # {"ok": true}
# 2. Would a task be accepted? Runs the admission policies without executing the task
curl -s -X POST $EXECUTOR/dryRun -H "Content-Type: application/json" \
-d '{"session_id": "smoke-1", "message_data": {"task_id": "t1", "job_data": {"text": "hi"}}}'
# 3. Submit a real task
curl -s -X POST $EXECUTOR/submitTask -H "Content-Type: application/json" \
-d '{"event_type": "task", "task_id": "t1", "task": {"text": "hi"}, "origin": {}}'
/dryRun returns {"success": true, "message": "ok"} if the task would be accepted, or HTTP 429 with the blocking stage (pre_check or rate_limit) and reason. /submitTask returns HTTP 202 once the task is queued. To get the result back synchronously, call the agent through the Delegate API (context.delegator.submit_and_wait(...) or POST <delegate>/api/submit-and-wait).
Validating Workflows¶
For DSL workflows, use check_execute in the DSL Manager as a dry run before executing.