Build Your First Agent

Step-by-step guide to creating, validating, and running your first validation agent using the Agent Definition Language.

Overview

This guide walks you through building a validation agent from scratch using the Agent Definition Language (ADL). By the end, you'll have a working agent that scores TypeScript error handling and produces structured findings with failure taxonomy codes.

We'll build a focused agent with four scoring categories. The process is: define → validate → render → install → test → iterate → publish.

Two ways to work with definitions:

Path	Tools	Best For
CLI	UluOps CLI (`ulu`)	Fast local validate/render iteration — no Registry round-trip
Registry MCP	registry MCP server in Claude Code	Full lifecycle — validate, create drafts, render, publish — on your Claude Code subscription

Both paths are shown throughout. Start with the CLI for rapid iteration, then use MCP to publish when you're ready.

Prerequisites

A UluOps account at app.uluops.ai with an API key (set as ULUOPS_API_KEY)
UluOps CLI installed (npm install -g @uluops/cli) — for local validation, rendering, and execution
Claude Code with the registry MCP server configured — for Registry operations and running agents on your subscription

1. Define Your Agent

Create a file called error-handling-validator.agent.yaml. ADL definitions have a specific structure — the key parts are interface, defaults, scoring, and decisions:

yaml

agent:
  interface:
    name: error-handling-validator
    version: 1.0.0
    displayName: "Error Handling Validator"
    description: >
      Validates error handling patterns in TypeScript code. Checks that
      async functions handle errors, public functions validate inputs,
      and error messages are descriptive and actionable.
    agentType: validator
    domain: software
    tools:
      - Read
      - Grep
      - Glob

  defaults:
    model: sonnet
    timeout: 300000

  scoring:
    maxScore: 100

    categories:
      - id: async_safety
        name: "Async Safety"
        weight: 30
        description: "Async error handling and promise safety"
        criteria:
          - id: async_try_catch
            name: "Async functions have try/catch or explicit propagation"
            points: 10
          - id: promise_catch
            name: "Promises have .catch() or are awaited in try/catch"
            points: 10
          - id: no_empty_catch
            name: "No empty catch blocks"
            points: 10

      - id: error_propagation
        name: "Error Propagation"
        weight: 25
        description: "Errors carry context and propagate correctly"
        criteria:
          - id: error_context
            name: "Errors include context about what operation failed"
            points: 10
          - id: no_swallowed_errors
            name: "Errors are not silently swallowed"
            points: 8
          - id: error_types
            name: "Custom error types used for domain errors"
            points: 7

      - id: input_validation
        name: "Input Validation"
        weight: 25
        description: "Public function input checks"
        criteria:
          - id: null_checks
            name: "Null/undefined checks before property access"
            points: 10
          - id: type_guards
            name: "Type guards on external inputs"
            points: 8
          - id: boundary_validation
            name: "Numeric bounds and string length validated"
            points: 7

      - id: error_messages
        name: "Error Messages"
        weight: 20
        description: "Error message quality and actionability"
        criteria:
          - id: descriptive_messages
            name: "Error messages describe what went wrong"
            points: 10
          - id: actionable_messages
            name: "Error messages suggest how to fix the issue"
            points: 10

  decisions:
    vocabulary:
      positive: PASS
      negative: FAIL
    preset: quality_gate

💡

Tip

Scoring constraints: The ADL schema enforces 4-6 categories, each weighted 10-30, with criteria worth 2-10 points each. All weights must sum to 100, and criteria points within each category must equal the category weight. Here: Async Safety has weight 30 with criteria totaling 10 + 10 + 10 = 30. See the ADL Specification for the full schema reference.

What Each Section Does

Section	Purpose
`interface`	Identity, type, and tools the agent can use
`defaults`	Model selection (`haiku`, `sonnet`, `opus`) and execution timeout
`scoring.categories`	What to evaluate — weighted dimensions with specific criteria
`decisions`	How to map scores to PASS/FAIL — effective thresholds come from `decisions.thresholds` (a pass score of 75 applies when none are set); see Scoring Model

For details on the six agent types and when to use each, see Agent Types.

2. Validate the Definition

Check your YAML against the ADL schema to catch structural errors before testing.

CLI — fast, no Registry needed:

bash

ulu definitions validate agent --file error-handling-validator.agent.yaml

MCP — validates via the Registry server:

text

validate_definition(type: "agent",
  file_path: "error-handling-validator.agent.yaml")

Common issues on first attempt:

Category weights not summing to 100
Criteria points not matching their category weight
Missing required fields (agentType, displayName, decisions)

Fix any errors and re-validate until it passes cleanly.

3. Render the Runtime Definition

The YAML is the source of truth, but the agent runs from a rendered markdown prompt. The factory transforms your scoring categories, criteria, and decision vocabulary into a structured prompt with verification checklists, scoring tables, and output format instructions.

CLI — renders locally from the YAML file:

bash

ulu def render agent --file error-handling-validator.agent.yaml

This produces error-handling-validator.md — the file that Claude actually reads when running your agent. It contains:

A role description derived from your interface block
Verification checklists generated from each criterion
A scoring table with category weights and point allocations
Decision thresholds from your decisions block (explicit thresholds, or the default pass score of 75)
Output format instructions for structured findings with failure taxonomy codes

MCP — renders from a definition in the Registry. First create a draft, then render it:

text

# Upload YAML as a draft
create_definition(type: "agent", name: "error-handling-validator",
  file_path: "error-handling-validator.agent.yaml")

# Render the draft to a local file
render_definition(type: "agent", name: "error-handling-validator",
  version: "1.0.0", output_path: "error-handling-validator.md")

💡

Tip

Which to use? During early iteration, ulu definitions validate + ulu def render is faster since it's purely local. Once you're close to publishing, create_definition + render_definition lets you preview exactly what the Registry will serve.

ℹ

Note

You author in YAML, the model receives markdown. This separation means improvements happen at the YAML level and propagate automatically through re-rendering. See Definitions for more on this two-stage pipeline.

4. Install the Agent

Before you can run the agent, Claude Code needs to know where to find it. Copy the rendered markdown to your Claude Code agents directory:

bash

# Install for all projects (global)
cp error-handling-validator.md ~/.claude/agents/

# Or install for the current project only
mkdir -p .claude/agents
cp error-handling-validator.md .claude/agents/

Once installed, the agent appears as a subagent_type option in Claude Code's Agent tool. The filename (minus .md) becomes the agent name.

ℹ

Note

How Claude Code finds agents: Claude Code looks for .md files in .claude/agents/ (project-level, takes priority) and ~/.claude/agents/ (global). Each file defines one agent. The file must have YAML frontmatter with name and tools fields — the factory generates this automatically from your ADL definition.

5. Test Your Agent

Run your agent against a codebase to see how it scores real code.

Claude Code — agents run on your subscription, no separate API credits:

text

Agent(subagent_type: "error-handling-validator",
  prompt: "Validate error handling in ./src")

CLI — uses BYOK (Anthropic API credits), useful for CI/CD and headless environments:

bash

ulu exec agent error-handling-validator -t ./src \
  --project test-run \
  --model haiku \
  --local-definitions .

--local-definitions . loads from the current directory instead of the Registry
--model haiku uses the fastest model for quick iteration (overrides defaults.model)
--project test-run names the run for tracker recording

Reading the Output

The agent produces a score, a decision, and a list of findings. Each finding includes a failure taxonomy code that classifies what kind of issue it is:

text

Score: 72/100 — FAIL (below 85 threshold)

Findings:
  - [SEM-COM/H] Empty catch block swallows authentication error
    File: src/auth/login.ts:45
    Category: error_propagation

  - [PRA-FRA/M] No input validation on user-supplied page parameter
    File: src/api/users.ts:112
    Category: input_validation

The failure code format is {DOMAIN}-{MODE}/{SEVERITY}:

Domains: STR (structural), SEM (semantic), PRA (pragmatic), EPI (epistemic)
Severity: C (critical), H (high), M (medium), L (low), I (info)

See the Failure Taxonomy for the full reference.

Review the output critically:

Check	What to look for
Score	Does the numeric score match your intuition about this codebase?
Decision	Is PASS/FAIL appropriate at the threshold?
Findings	Are the findings real issues or false positives?
Missing issues	Are there obvious problems the agent didn't catch?

6. Iterate on Criteria

If the agent produces false positives, make criteria more specific with verification checks:

yaml

criteria:
  - id: async_try_catch
    name: "Async functions have try/catch or explicit propagation"
    points: 10
    verification:
      method: manual
      checks:
        - "Async functions in request handlers have try/catch"
        - "Async functions that return Promise have .catch() or are awaited in try/catch"
        - "Intentional propagation via typed error returns is acceptable"

If the agent misses real issues, adjust point distribution within a category:

yaml

# Redistribute points within async_safety (must still sum to 30)
criteria:
  - id: async_try_catch
    name: "Async functions have try/catch or explicit propagation"
    points: 8   # reduced from 10
  - id: promise_catch
    name: "Promises have .catch() or are awaited in try/catch"
    points: 10
  - id: no_empty_catch
    name: "No empty catch blocks"
    points: 10
  - id: fire_and_forget
    name: "Fire-and-forget async calls are documented"
    points: 2   # new criterion, takes 2 pts from async_try_catch

After each YAML change: re-validate, re-render, re-install, re-test. The loop is:

bash

ulu definitions validate agent --file error-handling-validator.agent.yaml
ulu def render agent --file error-handling-validator.agent.yaml
cp error-handling-validator.md ~/.claude/agents/
# Then test again in Claude Code

Most agents converge in 2-3 iterations.

7. Publish to Registry

When you're satisfied with the agent's behavior, publish to make it available across projects and to other users.

MCP — create and publish in one step:

text

update_and_publish(type: "agent", name: "error-handling-validator",
  version: "1.0.0", file_path: "error-handling-validator.agent.yaml")

Or if you already created a draft in step 3, just publish it:

text

publish_definition(type: "agent", name: "error-handling-validator",
  version: "1.0.0")

CLI:

bash

# Create draft (skip if already created via MCP)
ulu definitions create agent error-handling-validator \
  -f error-handling-validator.agent.yaml

# Publish
ulu definitions publish agent error-handling-validator 1.0.0

Published agents can be rendered directly from the Registry — no local file needed:

text

render_definition(type: "agent", name: "error-handling-validator",
  output_path: "~/.claude/agents/error-handling-validator.md")

Going Further: A Production Agent

The example above is intentionally minimal. A production agent typically adds several more sections. Here's what a more complete version looks like:

yaml

agent:
  interface:
    name: error-handling-validator
    version: 2.0.0
    # ... same as above ...

  defaults:
    model: sonnet
    timeout: 300000

  mission:
    opener: >
      You are auditing error handling patterns in a TypeScript codebase.
      Your findings directly impact production reliability.
    stakes: >
      Unhandled errors cause silent failures, data corruption, and 3 AM
      pages. Every missed catch block is a potential production incident.
    role_boundaries:
      - "Only evaluate error handling — not code style, naming, or architecture"
      - "Do not suggest rewriting working code that handles errors correctly"

  knowledge_base:
    sections:
      - id: common-mistakes
        title: "Common Error Handling Mistakes"
        content: >
          1. Empty catch blocks that swallow errors silently
          2. Generic catch without re-throwing or logging
          3. Missing try/catch in async Express/Fastify route handlers
          4. Promise chains without terminal .catch()

  auto_fail:
    enabled: true
    conditions:
      - id: unhandled-rejection
        description: "Async route handler with no error boundary"
        trigger: "Top-level async function in a route/controller with no try/catch and no error middleware"

  # ... scoring and decisions same as above ...

The mission block gives the agent identity and scope boundaries. The knowledge_base encodes domain expertise. The auto_fail conditions define issues so severe they fail the agent regardless of score. See the ADL Specification for the full set of available sections.

What's Next

You've built a validation agent — the most common agent type. From here:

Run a Validation — Execute your agent in CI/CD pipelines and track results
Track Issues & Regressions — Monitor findings across runs with fingerprint-based correlation
Improve Your Agents — Use prompt-audit to systematically improve your agent's definition
Agent Types — Explore the five other agent types: executors, analysts, generators, explorers, and forecasters
ADL Specification — Full schema reference for all ADL fields
Failure Taxonomy — Classification system for validation findings