Skip to content

Behavior Controller

An agent that receives a goal has to answer two questions over and over: what are the steps to reach this goal? and given where I am now, what should I do next? The Behavior Controller is the sub-system that answers both, using LLMs that produce strictly structured (JSON) output so the answers can be executed by machines rather than just read by people.

Why it matters

  • Separates thinking from doing — planning and action selection live in a dedicated service, so the agent's executor stays simple and the reasoning strategy can be changed (prompt, model, backend) without touching agent code.
  • Structured, executable output — plans and selections follow templates, which makes them safe to hand to workflow, tool, or delegation executors.
  • Context-aware — prior results, local context, or FrameDB pointers can be fed in, so decisions take history into account.
  • Auditable — every task and result is persisted, so you can inspect why an agent behaved the way it did and reuse past plans.
  • Model-agnostic — runs against AIOS inference blocks (gRPC) or the OpenAI API.

How it works

The controller works in two phases:

  1. Phase 1 — Behavior planning. Given a goal and context, the LLM produces a structured plan made of tasks, each with an execution type, dependencies, and configuration.
  2. Phase 2 — Behavior selection. Given the plan and a set of candidate behaviors, the LLM selects the most appropriate one to execute next and returns its uid.

Requests are queued and processed asynchronously by background workers. For the full architecture, task lifecycle and schemas see Behavior Controller — Architecture.


At a Glance

The Behavior Controller is a two-phase reasoning and execution module that uses Large Language Models (LLMs) to plan and select structured behaviors for an agent. It provides:

  • REST APIs for Phase 1 (behavior planning) and Phase 2 (behavior selection).
  • Context-aware processing by incorporating prior results or FrameDB pointers.
  • Multiple LLM backends (internal AIOS gRPC or external OpenAI API).
  • Persistent storage for all task data and results to allow auditing and reuse.

A typical client workflow is:

  1. Submit a Phase 1 request via REST API to generate a structured plan.
  2. Optionally reuse prior context for memory-aware planning.
  3. Submit a Phase 2 request with candidate behaviors to select the next action.
  4. Fetch results via REST API or database queries.

Registering Behavior Controller in Subject Specification

The Behavior Controller can be registered as a sub-module in an agent’s Subject Specification by adding an entry of type SubSystemItem. This configuration tells the agent where to reach the Behavior Controller API and what defaults to use for LLM execution.


Data Class Reference

@dataclass
class SubSystemItem:
    sub_system_id: str
    sub_system_type: str = ""
    sub_system_config: Dict[str, Any] = field(default_factory=dict)

Minimal Required Configuration

Below is the minimal JSON block you need to include in the subject specification to enable the Behavior Controller.

{
  "sub_system_id": "behavior-controller-01",
  "sub_system_type": "behavior-controller",
  "sub_system_config": {
    "api_base_url": "http://behavior-controller.default.svc.cluster.local:8888",
    "llm_backend": "aios", 
    "default_phase1_template_id": "phase1-default",
    "default_phase2_template_id": "phase2-default"
  }
}

Field Descriptions

Field Location Type Description
sub_system_id root str Unique identifier for this subsystem entry in the subject spec.
sub_system_type root str Must be "behavior-controller" for the behavior control sub-module.
sub_system_config.api_base_url config str Base REST API URL for the Behavior Controller service. All HTTP calls (Phase 1 and Phase 2 processing) go here.
sub_system_config.llm_backend config str LLM execution backend (aios for internal AIOS gRPC, openai for OpenAI API).
sub_system_config.default_phase1_template_id config str ID of the default Phase 1 prompt template to use when none is specified.
sub_system_config.default_phase2_template_id config str ID of the default Phase 2 prompt template to use when none is specified.

Example in a Subject Specification

{
  "subject_id": "agent-beta",
  "subject_type": "agent",
  "sub_systems": [
    {
      "sub_system_id": "behavior-controller-01",
      "sub_system_type": "behavior-controller",
      "sub_system_config": {
        "api_base_url": "http://behavior-controller.default.svc.cluster.local:8888",
        "llm_backend": "aios",
        "default_phase1_template_id": "phase1-default",
        "default_phase2_template_id": "phase2-default"
      }
    }
  ]
}

This ensures that when the agent needs to plan or select behaviors, it knows exactly where to send API requests, which LLM backend to use, and which default templates to apply.


Import and Setup

REST Client

You can use requests or any HTTP client library.

import requests

BASE_URL = "http://<behavior-controller-api-url>"

Usage Guide

Submitting a Phase 1 Task (Behavior Planning)

payload = {
    "search_id": "task-001",
    "phase_1_data": {
        "input_description": "Plan a 3-step workflow to process satellite imagery."
    }
}

resp = requests.post(f"{BASE_URL}/process-phase-1-task", json=payload)
print(resp.json())

Expected Output:

{
  "success": true,
  "data": {
    "status": "success",
    "processed_data": {
      "plan": [
        {"task_id": "t1", "description": "Download images", "execution_type": "tool_call", "tool_id": "image_fetcher"},
        {"task_id": "t2", "description": "Preprocess images", "execution_type": "dsl", "dsl_id": "preprocess.dsl"},
        {"task_id": "t3", "description": "Run classification", "execution_type": "tool_call", "tool_id": "classifier"}
      ]
    }
  }
}

Submitting a Phase 2 Task (Behavior Selection)

payload = {
    "search_id": "task-001",
    "phase_2_data": {
        "planned_tasks": [
            {"task_id": "t1", "description": "Download images"},
            {"task_id": "t2", "description": "Preprocess images"},
            {"task_id": "t3", "description": "Run classification"}
        ]
    }
}

resp = requests.post(f"{BASE_URL}/process-phase-2-task", json=payload)
print(resp.json())

Expected Output:

{
  "success": true,
  "data": {
    "status": "success",
    "processed_data": {
      "uid": "t2"
    }
  }
}

Fetching Historical Phase 1 Results

resp = requests.get(f"{BASE_URL}/phase1-results/latest?limit=5")
print(resp.json())

Example: Switching LLM Backend in Config

If you want to use OpenAI instead of AIOS:

{
  "sub_system_id": "behavior-controller-01",
  "sub_system_type": "behavior-controller",
  "sub_system_config": {
    "api_base_url": "http://behavior-controller.default.svc.cluster.local:8888",
    "llm_backend": "openai",
    "default_phase1_template_id": "phase1-default",
    "default_phase2_template_id": "phase2-default"
  }
}