The missing object in a playground
A canvas can connect a model to several tools while leaving the actual agent undefined. Consider a refund assistant: is it explaining policy, recommending a refund, or issuing one? These responsibilities need different evidence and permissions. Start with a versioned definition containing a stable name, a narrow responsibility, instructions, accepted input and expected output. Then identify the model adapter and tools the runtime may resolve. A visually connected action block is not sufficient authorization.
Keep three contracts separate
The agent contract describes the work. The harness policy describes its execution envelope: deadlines, maximum turns, context limits, allowed resources and approval rules. The evaluation contract defines independent evidence of success. Our assistant proposes a refund; the harness checks ownership and approval; the grader verifies the durable operation record. When one editable prompt controls all three contracts, the component attempting the task can redefine what counts as success.
Resolve references rather than storing secrets
An agent definition should refer to a provider profile identifier, never contain an API key. The server resolves a profile belonging to the authenticated account and checks whether that provider is allowed for the data classification. Resolve tool identifiers through an allowlist. An unknown tool should fail before any model request. Store the accepted definition revision with the run so an edited draft cannot silently change an experiment already in progress.
Bind review to an immutable definition
The exercise computes a deterministic digest and rejects unknown tools. A digest makes approval refer to a particular validated representation; it does not prove that the instructions are safe. Neither does it freeze a mutable provider alias or Ollama model tag. Record available model revisions and adapter versions separately. Normalize configuration consistently before hashing, and never trust a digest supplied by the browser without recomputing it from the server-side record.
Investigate a misleading successful run
Imagine a user reviews definition version three, then changes the tool list before clicking Run. If execution reads the latest draft while approval names version three, the successful result does not prove the reviewed configuration worked. An integration test should save a revision, approve its digest, mutate the draft and verify that execution either uses the approved snapshot or refuses the mismatch. Test this through the API rather than relying on a disabled button.
Connect the definition to the interface
The proposed registry would let a learner inspect exactly which definition a run consumed, which settings an adapter rejected and which evidence justified delivery. Current Agent Canvas and Harness Studio expose separate pieces of this workflow. Importing control order is not equivalent to importing instructions, tools or model settings. The next integration must make that distinction visible instead of displaying a definition that execution ignores.
Run the exercise
Save the code as 05-agent-definition-contract.py and run python 05-agent-definition-contract.py with Python 3.11 or newer. This standard-library exercise uses synthetic data. It does not invoke a model or require an account, network connection or API key.
# Author: Sandeep Belgavi. Synthetic exercise; no network or provider calls.
import hashlib,json
allowed={"policy_search","order_read"}
agent={"name":"refund_reviewer","revision":1,"tools":["policy_search","order_read"]}
assert set(agent["tools"]) <= allowed
encoded=json.dumps(agent,sort_keys=True,separators=(",",":"))
assert len(hashlib.sha256(encoded.encode()).hexdigest())==64
assert not {"shell_execute"} <= allowed
print("definition accepted; unknown tool rejected")
Expected output:
definition accepted; unknown tool rejectedChange one assumption
Add shell_execute to the agent tool list: the first assertion must fail. Next reorder the JSON object keys and confirm the digest remains unchanged. Then change the revision and confirm the digest changes. Explain why array order may still be significant, and why this compact exercise is not a complete schema validator.
Implementation scope and further reading
This is an original AICassindra teaching design. The exercise demonstrates the stated invariant; it is not a complete production implementation. The local Harness Studio extension supports saved read-only agent teams, reviewed count-based routing and bounded sequential execution. Arbitrary imported graphs, effectful tools and automatic self-promotion remain unsupported.
All harness articles · Interactive boundary labs · Our public lab code