Validation

How validation works in UluOps — agents, scoring, decisions, and runs.

How Validation Works

Validation in UluOps follows a pipeline: define what to check, run agents against code, correlate findings with previous runs, and track results over time.

Each validation run produces:

• Scores — One per agent (0–100)
• Decisions — A status per agent (e.g., PASS/FAIL, DEPLOY/REVISE) based on configured decision presets
• Recommendations — Specific findings with file paths, line numbers, and severity
• Correlation — Fingerprint matching against prior runs (new, recurring, regression)

Agents

A validation agent is defined by an ADL spec. It reads code, evaluates it against criteria, and produces structured output. Validation agents are one of six agent types — each focuses on one quality dimension:

Scoring Model

Each agent produces a numeric score from 0 to 100 and a decision based on configurable thresholds. An agent's thresholds live in its decisions block — set explicitly via decisions.thresholds, or named by a decision preset. Conventional preset configurations:

security and critical complete the five preset names (low_risk, quality_gate, high_stakes, security, critical). The numbers above are conventional examples, not a fixed mapping — an agent's effective thresholds come from its own decisions.thresholds (a pass score of 75 applies when none are set), so check the agent's resolved decisions block rather than assuming a preset's values.

Each agent also defines its own decision vocabulary — the labels used for each threshold tier. Common vocabularies include PASS/FAIL, DEPLOY/REVISE, and SAFE/UNSAFE. Check the decisions block in the agent's YAML to see its specific vocabulary and preset.

Gate Conditions

Workflows define gates between phases. A gate evaluates whether the pipeline should continue:

• all_pass — Every agent in the phase must reach its threshold
• average_above — Average score across agents must exceed threshold
• any_pass — At least one agent must pass

When a gate fails, the pipeline stops and reports all findings collected so far.

Validation Runs

A run is a single execution of a validation workflow. Each run is stored with:

• Project name and workflow type
• Timestamp and run number (auto-incremented per project+workflow)
• Agent scores and statuses
• All recommendations with fingerprints
• Correlation summary (new issues, regressions, resolved)

const result = await client.runs.save({
  project: 'my-project',
  workflowType: 'post-implementation',
  agents: [
    { name: 'code-validator', score: 85, decision: 'PASS' },
  ],
  recommendations: [
    {
      agent: 'code-validator',
      title: 'Missing error handling in API client',
      priority: 'suggested',
      filePath: 'src/api/client.ts',
      lineNumber: 42,
    },
  ],
});

Agent Performance in Practice

The validation tracker records performance metrics across all agents and projects. Here are the most-used agents from the current ecosystem:

Source: UluOps Platform API get_analytics({ metric: "agent_performance" }), 90-day window as of March 2026. Run get_analytics for current data.

Notable patterns:

• test-architect has the lowest pass rate (52.2%) — test quality is the hardest dimension to satisfy, and it produces the most findings per run
• type-safety-validator has the highest pass rate (95.4%) — TypeScript type safety is highly verifiable, so findings tend to be clear-cut
• code-optimizer has a low pass rate (48.6%) because its threshold expects optimization opportunities to be addressed, not just identified

Query your own project's agent performance with get_agent_reliability (MCP) or client.analytics.getAgentReliability() (SDK).

Workflow Types

UluOps supports multiple workflow types for different stages of development:

Each workflow type maintains its own run counter and issue history per project.

Resolution Rates Across Projects

The tracker correlates findings across runs using fingerprint-based matching, enabling resolution tracking per project:

Source: UluOps Platform API get_analytics({ metric: "resolution_rates" }), 90-day window. Showing top projects by volume. Run get_analytics for current data.

The median resolution rate across all 48 tracked projects is approximately 88%. Projects with 100% resolution rates (like registry-uluops-mcp) typically had a focused development sprint followed by shipping. Projects with lower rates have ongoing development where new issues are still being filed.

Across the ecosystem, the run-level regression rate is 11.9% (66 of 554 runs show at least one regression). When regressions occur, they are typically isolated — the issue-level regression rate remains low, indicating that most fixes are durable.