Pipeline Definition Language

A YAML-based specification for defining multi-workflow orchestrations with triggers, approvals, rollback, and external notifications.

Format YAML + JSON SchemaExtension .pipeline.yamlVersion 1.2.0Status DraftUpdated 2026-03-26

Overview

The Pipeline Definition Language (PDL) is a YAML-based specification for defining multi-workflow orchestrations in the UluOps validation framework. Pipelines compose workflows and commands into stages with explicit dependencies, triggers, approvals, rollback strategies, and external notifications.

PDL is the top-level orchestration layer in the UluOps definition hierarchy:

text

ADL (Agent Definition)
       │
       │ wrapped by
       ▼
CDL (Command Definition)
       │
       │ orchestrated by
       ▼
WDL (Workflow Definition)
       │
       │ staged by
       ▼
PDL (Pipeline Definition)  ◄── YOU ARE HERE

What Pipelines Do

Capability	Description
Stage orchestration	Organize workflows/commands into stages with explicit dependencies
Event triggers	Execute on git push, PR events, schedules, webhooks, or manually
Approval gates	Require human authorization before critical stages
Environment management	Variables and secrets scoped to pipeline or stage
Failure recovery	Automatic rollback strategies on deployment failures
External notifications	Slack, email, GitHub status, webhooks
Artifact preservation	Configurable retention policies for build outputs
Cross-run state	Persist state across pipeline runs
Postflight tracker	Persist results to validation tracker with token metrics

Pipeline vs Workflow

Aspect	Workflow (WDL)	Pipeline (PDL)
Contains	Commands in phases	Workflows and/or commands in stages
Triggers	None (invoked by caller)	Git events, webhooks, schedules, manual
Approvals	None	Human gates before stages
Rollback	None	Automatic failure recovery
Notifications	None	External system integration
State	Per-run only	Persisted across runs
Postflight	Decision handlers (on_ship, etc.)	Tracker persistence + conditional actions

Design Principles

💡

Tip

Workflow/Command Orchestration, Not Agent Orchestration — Pipelines compose workflows and commands, not agents directly. Agents must be wrapped in commands, which can then be referenced in workflows or directly in pipeline stages.

💡

Tip

Event-Driven Execution — Pipelines define when they should run via triggers. Unlike workflows which are invoked programmatically, pipelines respond to external events (git push, PR, schedule, webhook, manual).

💡

Tip

Human-in-the-Loop Approvals — Critical stages can require human approval before execution. This enables safe deployment workflows with mandatory review checkpoints.

💡

Tip

Fail-Safe with Rollback — Pipelines can define automatic rollback strategies to recover from failures. This ensures production stability even when deployments fail.

💡

Tip

Declarative Stage Dependencies — Stages declare explicit dependencies. The runtime builds a DAG (Directed Acyclic Graph) and determines execution order, parallelizing where possible.

💡

Tip

Consistent Interface Pattern — Pipelines use the same interface block structure as Agents (ADL), Commands (CDL), and Workflows (WDL) for consistency across the definition language family.

Schema Structure

CI/CD Pipeline

A complete CI/CD pipeline with validation, staging, production deployment, rollback, and postflight tracker integration:

yaml

pipeline:
  interface:
    name: release-pipeline
    version: 1.0.0
    displayName: Release Pipeline
    description: Complete CI/CD pipeline from validation to deployment
    domain: software
    subdomain: ci-cd
    tags: [release, ci-cd, deployment]

  triggers:
    - type: git_push
      branches: [main, master]
      paths:
        include: ["src/**", "package.json"]
        exclude: ["**/*.md", "docs/**"]
    - type: manual
      allowed: true
      parameters:
        - name: environment
          type: string
          options: [staging, production]
          default: staging

  environment:
    variables:
      - name: NODE_ENV
        value: "production"
    secrets:
      - name: DEPLOY_KEY
        required: true
        stages: [deploy-staging, deploy-prod]

  stages:
    - id: preflight
      name: Pre-Flight Checks
      parallel: true
      steps:
        - name: Install dependencies
          command: npm ci
      gate:
        on_failure: abort

    - id: validate
      name: Core Validation
      depends_on: [preflight]
      workflows:
        - ref: ship@>=1.0.0
          args:
            target: "."
      gate:
        threshold: 70
        on_failure: abort

    - id: deploy-staging
      name: Deploy to Staging
      depends_on: [validate]
      steps:
        - name: Deploy
          command: npm run deploy:staging

    - id: deploy-prod
      name: Deploy to Production
      depends_on: [deploy-staging]
      approval:
        required: true
        approvers: ["@release-team"]
        timeout_hours: 24
      steps:
        - name: Deploy
          command: npm run deploy:prod

  rollback:
    enabled: true
    strategy: revert_to_last_success
    triggers:
      - condition: "stages.deploy-prod.failed"
        action: rollback_production

  notifications:
    channels:
      - type: slack
        webhook: "{{ secrets.SLACK_WEBHOOK }}"
        events: [started, success, failure, rollback]
      - type: github_status
        enabled: true

  postflight:
    tracker:
      enabled: true
      project: "{{ pipeline.interface.name }}"
      include_token_metrics: true
    on_success:
      - description: Notify release channel
        command: echo "Release pipeline succeeded"
    on_failure:
      - description: Alert on-call
        command: echo "Release pipeline failed"

Multi-Workflow Orchestration

A simpler pipeline that orchestrates multiple validation workflows:

yaml

pipeline:
  interface:
    name: full-validation
    version: 1.0.0
    displayName: Full Validation Pipeline
    description: Comprehensive validation running multiple specialized workflows
    domain: software

  triggers:
    - type: pull_request
      actions: [opened, synchronize]
    - type: manual
      allowed: true

  stages:
    - id: pre-implementation
      name: Pre-Implementation Review
      workflows:
        - ref: pre-implementation@>=1.0.0
          args:
            target: "."
      gate:
        threshold: 70
        on_failure: warn

    - id: post-implementation
      name: Post-Implementation Validation
      depends_on: [pre-implementation]
      workflows:
        - ref: post-implementation@>=1.0.0
          args:
            target: "."
      gate:
        threshold: 75
        on_failure: abort

    - id: ship-gate
      name: Ship Readiness Gate
      depends_on: [post-implementation]
      workflows:
        - ref: ship@>=1.0.0
          args:
            target: "."
            strict: true
      gate:
        threshold: 80
        on_failure: abort

Hybrid Pipeline (Workflows + Commands)

A pipeline that mixes workflows with direct command invocations:

yaml

pipeline:
  interface:
    name: audit-and-fix
    version: 1.0.0
    displayName: Audit and Fix Pipeline
    description: Run validation audit then attempt automatic fixes
    domain: software

  triggers:
    - type: schedule
      cron: "0 3 * * *"
    - type: manual
      allowed: true
      parameters:
        - name: auto_fix
          type: boolean
          default: true

  stages:
    - id: audit
      name: Full Audit
      workflows:
        - ref: ship@>=1.0.0
          args:
            target: "."
      gate:
        threshold: 0
        on_failure: warn

    - id: auto-fix
      name: Automatic Fixes
      depends_on: [audit]
      condition: "params.auto_fix && stages.audit.score < 90"
      commands:
        - ref: lint-fix@>=1.0.0
          args:
            target: "."
        - ref: type-fix@>=1.0.0
          condition: "context.typescript_detected"
      sequential: true

    - id: verify
      name: Verify Fixes
      depends_on: [auto-fix]
      condition: "stages.auto-fix.completed"
      workflows:
        - ref: ship@>=1.0.0
          args:
            target: "."
      gate:
        threshold: 80
        on_failure: warn

Field Reference

Interface Block

The interface block identifies the pipeline and is consistent with ADL, CDL, and WDL.

Field	Type	Required	Description
`name`	`string`	Yes	Kebab-case identifier. Pattern: ^[a-z][a-z0-9](-[a-z0-9]+)$
`version`	`string`	Yes	Semantic version. Pattern: ^\d+\.\d+\.\d+$
`displayName`	`string`	Yes	Human-readable name (3-50 characters)
`description`	`string`	Yes	Pipeline purpose and scope (20-500 characters)
`domain`	`Domain`	Yes	Primary domain classification
`subdomain`	`string`	No	Domain refinement (kebab-case)
`tags`	`string[]`	No	Searchable discovery tags

Triggers Block

Triggers define when the pipeline executes. Six trigger types are supported.

Field	Type	Required	Description
`type`	`enum`	Yes	Trigger type: git_push, pull_request, schedule, webhook, manual, pipeline
`branches`	`string[]`	No	Branch patterns (glob supported). For git_push and pull_request triggers
`tags`	`string[]`	No	Tag patterns. For git_push trigger
`paths`	`object`	No	File path filters with include/exclude arrays (glob patterns)
`actions`	`string[]`	No	PR actions: opened, synchronize, reopened, closed, labeled, etc.
`cron`	`string`	No	Cron expression for schedule trigger
`timezone`	`string`	No	Timezone for schedule (default: UTC)
`event`	`string`	No	Event name for webhook trigger
`secret`	`string`	No	HMAC verification secret for webhook
`allowed`	`boolean`	No	Enable manual triggering
`parameters`	`Parameter[]`	No	Input parameters for manual trigger (name, type, options, default)

Trigger Types:

Type	Description	Key Fields
`git_push`	Triggered on push to branch	`branches`, `tags`, `paths`
`pull_request`	Triggered on PR events	`actions`, `branches`, `paths`
`schedule`	Cron-based scheduling	`cron`, `timezone`
`webhook`	External HTTP webhook	`event`, `secret`, `payload_filter`
`manual`	Manual invocation	`allowed`, `parameters`
`pipeline`	Triggered by another pipeline	`pipeline`, `condition`

Environment Block

Environment variables and secrets for the pipeline.

Field	Type	Required	Description
`variables`	`Variable[]`	No	Environment variables with name, value, and optional stage scoping
`secrets`	`Secret[]`	No	Secret references with name, required flag, stage scoping, and provider config

Stages Block

Stages are the primary execution units in a pipeline.

Field	Type	Required	Description
`id`	`string`	Yes	Unique stage identifier (kebab-case)
`name`	`string`	Yes	Human-readable stage name
`depends_on`	`string[]`	No	Stage dependencies (empty = root stage)
`condition`	`string`	No	Enable condition expression
`timeout`	`number`	No	Timeout in milliseconds
`parallel`	`boolean`	No	Run items in parallel (default: false)
`workflows`	`WorkflowRef[]`	No	Workflow references with ref, args, condition, timeout
`commands`	`CommandRef[]`	No	Command references with ref, args, condition, timeout
`agents`	`AgentRef[]`	No	Direct agent references by subagent_type identifier (ref, model, args, condition, timeout)
`steps`	`Step[]`	No	Inline shell steps with name, command, env, retries
`sequential`	`boolean`	No	Force sequential execution of items
`gate`	`Gate`	No	Pass/fail gate with threshold, aggregate, on_failure
`artifacts`	`Artifact[]`	No	Stage-specific artifact preservation
`approval`	`Approval`	No	Human approval requirement for this stage
`env`	`Record`	No	Stage-specific environment overrides
`inputs`	`object`	No	Input parameters from dependency stage outputs
`outputs`	`Output[]`	No	Stage outputs (name, value or command expression)
`artifacts_from`	`string[]`	No	Import artifacts from other stages by stage ID

Approvals Block

Human approval requirements for stages.

Field	Type	Required	Description
`required`	`boolean`	No	Whether approval is required (default: false)
`approvers`	`string[]`	No	List of approvers (team mentions, emails, platform-specific)
`min_approvals`	`number`	No	Minimum approvals needed (default: 1)
`timeout_hours`	`number`	No	Hours before auto-reject (default: 72)
`auto_reject_on_change`	`boolean`	No	Reject if source code changes (default: true)
`allow_self_approval`	`boolean`	No	Allow triggerer to approve (default: false)
`notification`	`Notification`	No	Approval request notification configuration

Approval Timeout Behavior:

Behavior	Description
`reject`	Auto-reject after timeout (default)
`approve`	Auto-approve after timeout
`extend`	Notify and extend timeout
`pause`	Pause pipeline indefinitely

Rollback Block

Automatic failure recovery configuration.

Field	Type	Required	Description
`enabled`	`boolean`	No	Enable automatic rollback (default: false)
`strategy`	`enum`	No	Rollback strategy: revert_to_last_success, revert_to_previous, custom, none
`triggers`	`RollbackTrigger[]`	No	Conditions that trigger rollback (condition + action pairs)
`actions`	`Record<string, Action>`	No	Named rollback actions with steps
`preserve_logs`	`boolean`	No	Preserve logs on rollback (default: true)
`notify_on_rollback`	`boolean`	No	Send notification on rollback
`max_rollback_attempts`	`number`	No	Maximum rollback retry count

Rollback Strategies:

Strategy	Description
`revert_to_last_success`	Restore last successful deployment
`revert_to_previous`	Restore immediate previous state
`custom`	Execute custom rollback actions
`none`	No automatic rollback

Notifications Block

External system notification configuration.

Field	Type	Required	Description
`channels`	`Channel[]`	No	Notification channels: slack, email, github_status, webhook, pagerduty, teams
`templates`	`Record<string, string>`	No	Message templates for events (started, success, failure, rollback)
`rate_limit`	`object`	No	Rate limiting with max_per_hour and deduplicate options

Channel Types:

Type	Description
`slack`	Slack webhook notifications
`email`	Email notifications
`github_status`	GitHub commit status checks
`webhook`	Generic HTTP webhook
`pagerduty`	PagerDuty incident creation
`teams`	Microsoft Teams notifications

Notification Events: started, stage_started, stage_complete, approval_required, approved, rejected, success, failure, rollback, timeout

Artifacts Block

File preservation configuration.

Field	Type	Required	Description
`name`	`string`	Yes	Artifact identifier
`path`	`string`	No	File glob pattern
`source`	`string`	No	Stage output reference
`format`	`string`	No	Output format: json, yaml, raw
`retention_days`	`number`	No	Days to retain (default: 30)
`compression`	`string`	No	Compression type: gzip, none
`required`	`boolean`	No	Fail if artifact missing
`condition`	`string`	No	Conditional preservation expression

State Block

Cross-run state persistence.

Field	Type	Required	Description
`persistence`	`boolean`	No	Enable state persistence
`ttl`	`string`	No	Time to live (e.g., "30d", "168h")
`tracked`	`string[]`	No	Automatically tracked values (timestamps, counters)
`custom`	`CustomState[]`	No	User-defined state values with name, value, and trigger condition
`conditions`	`Conditions`	No	State-based behavior (pause_on_consecutive_failures, require_manual_after_rollback)

Postflight Block

Actions executed after all stages complete. Handles tracker persistence, token metric collection, and conditional handlers.

Field	Type	Required	Description
`tracker`	`PostflightTracker`	No	Validation tracker integration for persisting pipeline results (project, workflow_type, token metrics, field mappings)
`on_success`	`PostflightAction[]`	No	Actions to run when all gates pass (description, command, condition)
`on_failure`	`PostflightAction[]`	No	Actions to run when any gate fails (description, command, condition)
`always`	`PostflightAction[]`	No	Actions to run regardless of outcome (description, command, condition)

Postflight Tracker

Configuration for saving pipeline results to the UluOps validation tracker.

Field	Type	Required	Description
`enabled`	`boolean`	No	Enable tracker persistence (default: true)
`project`	`string`	No	Tracker project name. Supports template expressions (e.g., {{ pipeline.interface.name }})
`workflow_type`	`string`	No	Workflow type identifier for the tracker run. Defaults to pipeline.interface.name
`include_token_metrics`	`boolean`	No	Collect token usage metrics from agent-metrics buffer (default: true)
`token_metrics_command`	`string`	No	Command to extract token metrics. Must output tracker-compatible format
`field_mappings`	`FieldMappings`	No	Maps pipeline output fields to tracker fields (score_source, status_source, model_source, recommendations_source)
`verify_save`	`boolean`	No	After saving, verify that total_issues matches the saved count (default: true)

Stage Dependency Graph

Pipelines build a DAG (Directed Acyclic Graph) from stage dependencies. Independent stages run in parallel automatically.

yaml

stages:
  - id: lint          # Root
  - id: test          # Root
  - id: typecheck     # Root
  # All three run in parallel

  - id: build
    depends_on: [lint, test, typecheck]
    # Waits for all three

text

  lint    test    typecheck
    \      |      /
     \     |     /
        build

Conditions create runtime-determined paths:

yaml

stages:
  - id: validate
  - id: deploy-staging
    depends_on: [validate]
    condition: "trigger.branch != 'main'"
  - id: deploy-prod
    depends_on: [validate]
    condition: "trigger.branch == 'main'"

⚠

Warning

Cycle Detection — The runtime detects circular dependencies and fails validation. For example, A → C → B → A would be rejected.

Composition Rules

ℹ

Note

Pipelines can reference Workflows (WDL), Commands (CDL), and Agents (ADL) — Pipeline stages can contain workflow references, command references, and direct agent references (by subagent_type), composing them into multi-stage orchestrations.

ℹ

Note

Pipelines can contain inline shell steps — Simple operations can be expressed as inline steps without requiring a formal CDL command definition.

ℹ

Note

Stages form a DAG via depends_on — Stage dependencies create a Directed Acyclic Graph. Independent stages run in parallel; cycles are detected and rejected at validation time.

ℹ

Note

Triggers are exclusive to pipelines — Only PDL definitions have triggers. Workflows and commands are invoked programmatically or by a pipeline.

ℹ

Note

Approvals and rollback are pipeline-level concerns — Human approval gates and automatic rollback strategies exist only in PDL, not in WDL or CDL.

ℹ

Note

Postflight handles tracker persistence — The postflight block integrates with the UluOps validation tracker to persist pipeline results, collect token metrics, and run conditional cleanup actions.

Revision History

Version	Date	Changes
1.2.0	2026-03-26	Added postflight block with tracker integration (project, workflow_type, token metrics, field mappings) Added postflight conditional handlers: on_success, on_failure, always Added direct agent references in stages (agents field with subagent_type identifiers) Added stage inputs, outputs, and artifacts_from for cross-stage data flow
1.0.0	2026-01-14	Initial PDL specification Trigger types: git_push, pull_request, schedule, webhook, manual, pipeline Stage orchestration with DAG-based dependency resolution Human approval workflows with multi-level support Rollback strategies: revert_to_last_success, revert_to_previous, custom Notification channels: slack, email, github_status, webhook, pagerduty, teams Artifact preservation with retention policies Cross-run state persistence