Chapter 17: The Loading Pipeline — What Happens Under the Hood¶

← Back to Table of Contents | ← Previous: Name Sanitization

When you call load("config.yaml"), here's exactly what happens:

Step 1: Parse Sources¶

Each config source (file path or raw YAML string) is parsed with yaml.safe_load(). Path objects are always treated as files. Strings are files if the path exists on disk, otherwise parsed as inline YAML — with two refinements worth knowing: a multi-line string is never probed as a path, and a single-line string that looks like a path (contains a separator, or ends .yaml/.yml) but does not exist raises FileNotFoundError rather than being parsed as YAML and failing confusingly later.

Step 2: Strip Anchors and Interpolate Variables¶

For each source independently: 1. Extract and remove the vars block 2. Strip x-* keys (YAML anchor scratch pads) 3. Interpolate ${VAR} references using vars + environment

Step 3: Rewrite Relative Paths¶

All filesystem-based specs (./file.py:func, ./tools/) are rewritten to absolute paths anchored to the config file's directory. This ensures the config works regardless of the working directory.

Step 4: Sanitize Collection Keys¶

Names in all collection sections are sanitized to [a-zA-Z0-9_-]. Cross-references are updated automatically.

Step 5: Merge (If Multi-File)¶

Collection sections are combined, duplicate names detected, singleton fields use last-wins.

Step 6: Normalize, Then Validate Against Schema¶

normalize() runs first and checks version: only "1" exists, and anything else is rejected here rather than producing a confusing field error further down.

The normalized dict is then validated against the Pydantic models. Invalid fields, missing required values, wrong types — all caught here. Only the first Pydantic error is surfaced, reshaped as SchemaValidationError:

Invalid config at 'agents.analyst.model': Input should be a valid string

Step 7: Validate References¶

Cross-references are checked, raising UnresolvedReferenceError — which, unlike the schema error above, lists the names that are available: - Agent model references → must exist in models - Agent mcp references → must exist in mcp_clients - MCP client server references → must exist in mcp_servers - Orchestration agent references → must exist in agents or orchestrations

Step 8: Resolve Infrastructure¶

Models, MCP servers, MCP clients, and session managers are created as Python objects. Nothing is started yet.

resolve_infra() also enforces one rule that is easier to catch here than later: a global session_manager with provider: agentcore is rejected, because that provider needs an actor_id unique to each agent.

Step 9: Start MCP Lifecycle¶

MCP servers are started in background threads. The pipeline waits for all servers to be ready (TCP port check). This happens before agent creation because Agent.__init__ auto-starts MCP clients which need running servers.

Step 10: Create Agents¶

Each agent definition is resolved: model looked up, tools loaded, hooks instantiated, MCP clients attached, session manager wired. Each agent is a fresh strands.Agent instance.

Agents that are nodes of a swarm or graph are checked here and rejected if they carry a session manager, global or per-agent, because strands cannot persist those (see Chapter 7).

Step 11: Wire Orchestrations¶

Orchestrations are topologically sorted and built in dependency order. Inner orchestrations first, outer orchestrations reference the already-built inner ones.

Step 12: Return ResolvedConfig¶

The final ResolvedConfig has: - agents — dict of all agents by name - orchestrators — dict of all built orchestrations by name - entry — the entry point (Agent, Swarm, or Graph) - mcp_lifecycle — for managing shutdown - app_config — the validated AppConfig the rest was built from

Advanced Topic: `load()` vs `load_config()` + `resolve_infra()` + `load_session()`¶

Most users only need:

from kaboo_workflows import load

resolved = load("config.yaml")

That one call runs the whole pipeline:

Parse YAML, then per source: pop vars, strip x-* anchors, interpolate, rewrite relative paths
Sanitize names, per source
Merge sources
Normalize, validate schema, validate references
Apply log_level
Initialise telemetry — process-wide, and the first call wins, so a later load() cannot change it
Resolve infrastructure
Start MCP lifecycle
Create agents and orchestrations

Sanitizing and interpolating happen before the merge, per source, which is why one file's vars never leak into another's interpolation.

But kaboo-workflows also exposes the lower-level split because config parsing and session creation are not always the same thing.

What counts as "config"?¶

load_config() returns a validated AppConfig — just structured data.

At this point, nothing is started and no live strands objects exist yet:

no Agent instances
no orchestration objects
no started MCP servers
no connected MCP clients

This step is useful when you want to parse and validate once at process startup, fail fast on bad YAML, and keep the validated config around.

What counts as "infrastructure"?¶

resolve_infra(app_config) turns the validated config into the shared runtime pieces:

resolved model objects
resolved MCP server objects
resolved MCP client objects
a cold mcp_lifecycle

Note: resolve_infra() does not build session manager instances. Session managers are created per agent and per orchestration at session time (inside load_session()), once a real session ID is known. This avoids creating orphan filesystem folders before a session actually starts.

Important nuance: resolved does not mean started.

After resolve_infra():

MCP servers exist as Python objects, but are not running yet
MCP clients exist as Python objects, but are not connected yet
agents still do not exist
orchestrations still do not exist

You then start the shared MCP runtime explicitly:

from kaboo_workflows.config import load_config, resolve_infra

app_config = load_config("config.yaml")
infra = resolve_infra(app_config)
infra.mcp_lifecycle.start()

What `load_session()` does¶

load_session(app_config, infra, session_id=...) is the final step. It uses the already-started shared infrastructure to create a fresh ResolvedConfig for one session:

fresh agents
fresh orchestrations
fresh entry point
the same shared MCP lifecycle

This is the key distinction:

resolve_infra() gives you shared process-level infrastructure
load_session() gives you session-level agent graph built on top of that infrastructure

Why this split matters for multi-tenant deployments¶

In a multi-tenant server, you usually do not want to re-parse YAML, re-resolve models, or restart MCP servers on every request. Those are process-level concerns.

Instead, you want:

one validated config shared by the process
one resolved infrastructure shared by the process
one started MCP lifecycle shared by the process
one fresh set of agents per tenant/session/request

Typical pattern:

from kaboo_workflows.config import load_config, load_session, resolve_infra

# Once at process startup
app_config = load_config("config.yaml")
infra = resolve_infra(app_config)
infra.mcp_lifecycle.start()

# Per request / websocket / tenant session
resolved = load_session(app_config, infra, session_id="tenant-123")
result = resolved.entry("Hello!")

This avoids paying the startup cost repeatedly while still keeping per-session agent state isolated.

Session manager nuance¶

Session manager instances are not built during resolve_infra(). Instead, load_session() computes a single effective_session_id and threads it down to every agent and orchestration leaf:

If you pass session_id="my-id" to load_session(), that value is used as-is.
If you do not pass a session_id but the config declares a global session_manager:, kaboo-workflows looks for a session_id in session_manager.params. If found, that value is used; otherwise a fresh uuid.uuid4() is generated once and shared by all agents in that call — matching the "one folder per CLI run" behaviour.
If neither a session_id is provided nor a global session_manager: is configured, no session manager instances are created at all. Each individual agent or orchestration with a per-agent session_manager: will generate its own ID as usual.

This design guarantees exactly one folder is created per load_session() call, never an orphan folder from resolve_infra().

When the config itself is per-request¶

The split above assumes one config, many sessions. Push it one step further and the config becomes per-request too: the run submits it, and the process only holds the base it layers over.

That needs one more seam, because load_config() both parses and validates in one call, and here you need the raw parsed base kept around to merge each run's overlay into. So the two halves are separately available.

The base is still validated at boot, and still needs an entry — entry is required and has no default. Give it a fallback agent, as examples/21_runtime_configs does, so the service is valid on its own and a run that submits no workflow still has something to answer with.

from dataclasses import replace

from kaboo_workflows import (
    load_session,
    load_session_config,
    parse_config_sources,
    resolve_infra,
    resolve_run_clients,
    validate_raw_config,
)

# Once at boot: parse the base, keep it raw.
base_raw = parse_config_sources("config.yaml")
infra = resolve_infra(validate_raw_config(base_raw))
infra.mcp_lifecycle.start(pin_clients=False)   # servers only; clients are per-run

# Per run: merge, validate, resolve.
config = load_session_config(base_raw, submitted_yaml, allowed_mcp_hosts=["gateway.internal"])
clients = resolve_run_clients(config, infra)
clients.start(pin_clients=False)               # the run owns these sessions
resolved = load_session(
    config,
    replace(infra, clients=clients.clients, mcp_lifecycle=clients),
    session_id=thread_id,
)

Both additions matter. Without clients.start(...), the run's MCP sessions are never opened; without mcp_lifecycle=clients, load_session keeps the boot-time lifecycle, so shutting the run down closes nothing it opened and the sessions leak for the life of the process.

create_agui_app(config_path, session_config_key="workflow_config") does exactly this for you, reading the overlay from forwardedProps and closing the run's clients when its stream ends. See Chapter 13 for the merge rules, and examples/21_runtime_configs for three runs resolving three different workflows against one base.

What is left at process level shrinks to what is genuinely process-level: the parsed base, the model objects, and any MCP server processes. Everything a conversation touches — agents, orchestrations, entry, client sessions — is built for the run and released with it.

This is only safe because conversation state does not live in those objects. History arrives each turn in state.kaboo_history, and a pending interrupt arrives in state.kaboo_session (Chapter 7). Rebuilding is therefore indistinguishable from retaining, which is what lets you run a second replica, restart mid-approval, or serve a workflow edited thirty seconds ago.

Mental model¶

Use this rule of thumb:

load() = convenience API for scripts and local apps
load_config() = validate and freeze the declarative config
resolve_infra() = build shared runtime dependencies, but do not start them yet
load_session() = build one session's live agents/orchestrations from shared infra
parse_config_sources() + load_session_config() = the config itself varies per run

If you're building a CLI, a notebook, or a one-shot script, use load().

If you're building a long-running web server with many user sessions, use load_config() + resolve_infra() once, then load_session() for each session.

If those sessions are different workflows, keep the base raw and merge an overlay per run.

Next: Chapter 18 — Full Reference →