QAP Service-Level Objectives¶

Targets the Quality Assurance Pipeline meets in steady state. Each SLO maps to one Grafana panel and (after PR 7) one alert rule. Numbers are starting points based on the staged rollout — revisit after a quarter of production data.

Phase-duration SLOs¶

Phase	p95 budget	Metric
qa_provisioning_build (bootstrap chain)	6 min	qap_phase_duration_seconds{phase="qa_provisioning_build"}
qa_testing_prototype	90 min	qap_phase_duration_seconds{phase="qa_testing_prototype"}
qa_validating_master	180 min	qap_phase_duration_seconds{phase="qa_validating_master"}

The bootstrap chain budget is dominated by the Agno /workflows/build call (~3 min p95). The two polling phases' budgets reflect Copilot's review-loop latency and master-branch CI cost.

Success-rate SLO¶

≥ 95% of QAP sessions reach qa_completed without operator intervention, excluding sessions that fail with max_heal_attempts (which is a code-quality signal, not a pipeline failure).

Metric:

(
  sum(rate(qap_failures_total{reason!="max_heal_attempts"}[1d]))
  /
  sum(rate(qap_active_sessions{phase="qa_completed"}[1d]))
) < 0.05

Agno latency SLO¶

Path	p95 budget
/workflows/build	180 s
/workflows/test	60 s
/workflows/triage-failures	30 s

Metric: agno_request_duration_seconds_bucket.

Rate-limit SLO¶

< 1 agno_rate_limit_hits_total per hour during business hours.

Sustained rate-limit hits mean either client-side ramp pushed past the documented 100/min ceiling or the limiter AGNO_RATE_LIMIT_PER_MIN configuration drifted out of sync. Investigate before the next ramp stage.

Tick-budget SLO¶

< 1 qap_tick_budget_exhausted_total per 100 tracker invocations.

Anything higher means trackers are spending most of a tick on one slot and falling behind on overall throughput. Mitigations: lower the per-tick slot work, or raise AGNO_TICK_BUDGET_SECONDS.

Heal-cycle outcome distribution¶

≥ 80% of qap_heal_attempts_total{outcome="issue_opened"} should ultimately produce a green PR (downstream signal: qa_completed follows the same session).

This is harder to query as a single Prometheus expression; the dashboard shows it as the ratio of qap_heal_attempts_total{outcome="issue_opened"} to qap_failures_total{reason=~"heal_.*|max_heal_attempts"} over the same 24h window.

What's not an SLO yet¶

• End-to-end wall-clock duration (sum of phase budgets). Track per-phase first; total time is too sensitive to Copilot review latency.
• Per-tenant cost. Wait until billing is wired before alerting on this.