docs(agents): add hardened AGENTS instructions for root and submodules

This commit is contained in:
BrunoGaoSZ
2026-02-26 07:44:14 +01:00
parent ae78a9bbf2
commit 34e3924c08
4 changed files with 598 additions and 58 deletions
+168 -28
View File
@@ -1,34 +1,174 @@
# agent-platform/AGENTS.md - 项目根主指令(硬化版)
# Agent Platform — Project Root Instructions
## 目标
企业智能体平台项目以可靠性、可审计性、可观测性和低风险变更为首要目标。
## Project Context
Enterprise AI agent platform powering three production scenarios:
1. Presales — proposal documents (Word/Excel) in ~30 min from days
2. Security — incident reports from hours to minutes
3. Knowledge QA — citation-backed answers, ≥90 % reference coverage
## 全局约束
- Python 版本必须为 3.11+。
- 所有新增函数必须有完整类型注解;禁止裸 `Any`。
- Pydantic 的 `list`/`dict` 字段必须使用 `Field(default_factory=...)`。
- 涉及货币、成本、比率的字段统一使用 `Decimal`,禁止 `float`。
- 配置加载统一通过 `get_settings()` 惰性单例,禁止模块级实例化 `Settings()`。
- 依赖安装与执行统一使用 `uv`,并保持 `uv.lock` 与代码同步。
## Tech Stack (all versions locked in uv.lock)
- Orchestration : langgraph ≥ 0.3.0
StateGraph + interrupt/resume for human-in-the-loop
- LLM cloud : anthropic ≥ 0.50.0 → claude-sonnet-4-6-20260217
- LLM local : Qwen2.5-14B / 7B via OpenAI-compatible HTTP endpoint
- Vector DB : pymilvus ≥ 2.4.0 + BGE-M3 embeddings + BGE-reranker
- API : fastapi ≥ 0.115.0 + uvicorn[standard]
- ORM : sqlalchemy[asyncio] ≥ 2.0.30 + asyncpg ≥ 0.29.0
- Migrations : alembic ≥ 1.13.0
- Observability : opentelemetry-sdk ≥ 1.27.0
+ prometheus exporter + grafana dashboards
- Auth : python-jose[cryptography] + passlib[bcrypt]
- Encryption : cryptography ≥ 42.0.0 (AES-256-GCM, audit payloads)
- Docs : python-docx ≥ 1.1.2
## 代码组织
- `core/` 放置领域逻辑,不依赖 `api/`。
- `api/` 仅做协议层、鉴权、中间件与路由编排。
- `db/` 负责仓储和迁移,不承载业务流程。
- `profiles/` 仅存 SOP/prompt 资产,不写运行时代码。
- `tests/` 按 `unit/integration/contracts` 分层。
## Repository Layout
```
agent-platform/
├── core/
│ ├── engine/ # LangGraph: state, 5 nodes, graph builder
│ ├── profile/ # SOP YAML schema (Pydantic), loader, registry
│ ├── tools/ # BaseTool, ToolResult, ToolExecutor, registry
│ ├── llm/ # ClaudeClient, PromptBuilder
│ ├── verify/ # EvidenceScorer
│ ├── audit/ # AuditLogger (3-tier), Sanitizer, Encryptor
│ ├── observability/ # OTel setup, decorators, CostTracker, dashboards/
│ └── router/ # ModelRouter
├── api/
│ ├── routes/ # health, chat, approve, trace
│ ├── contracts/ # Pydantic req/resp models, permissions matrix
│ └── middleware/ # tracing, auth (JWT), rbac, rate_limit
├── profiles/
│ ├── presales/prompts/
│ ├── security_analysis/prompts/
│ └── knowledge_qa/prompts/
├── db/
│ ├── base.py
│ ├── models.py
│ ├── repositories/
│ └── migrations/versions/
├── tests/
│ ├── unit/
│ ├── integration/
│ ├── contracts/ # real Claude API; @pytest.mark.claude_api
│ └── fixtures/
├── scripts/
│ ├── setup_env.sh
│ ├── health_check.sh
│ ├── seed_data.py
│ └── gen_audit_key.py
├── docker/
│ ├── Dockerfile.api
│ ├── otel-collector-config.yml
│ └── prometheus.yml
├── .env.example
├── tests/.env.test # committed; no real secrets
├── pyproject.toml
├── uv.lock # MUST be committed; MUST pass uv sync --frozen
├── alembic.ini
├── Makefile
└── docker-compose.yml
```
## 质量门禁
- 提交前必须通过:`ruff check`、`mypy`、`pytest --collect-only`。
- 任何 schema、鉴权、审计、加密相关变更必须补充测试。
- 禁止绕过失败门禁直接合并。
## Critical Architecture Rules (NEVER Violate)
## 安全与审计
- 严禁提交真实密钥、令牌、证书。
- 审计 payload 必须加密存储,且支持 keyring 轮换。
- 生产环境日志禁止输出敏感字段原文。
### R1 — Tool Whitelist Enforcement
`ToolExecutor.execute_tool()` checks `profile.allowed_tools` BEFORE
calling any tool.
Violation → `raise ToolNotAllowedError(tool_name, profile_name)`
+ write AuditEvent(event_type="whitelist_violation").
Tool implementations NEVER perform whitelist checks themselves.
## 变更原则
- 小步提交,保持可回滚。
- 不在无关文件中引入格式化噪音。
- 新增依赖必须说明必要性与替代方案不可行性。
### R2 — Exception Boundary
| Layer | Raises? | On failure |
|------------------------|---------------|------------------------|
| ToolExecutor / Act node| YES — policy | ToolNotAllowedError |
| BaseTool.execute() | NEVER | ToolResult(success=False) |
| LangGraph nodes | NEVER escape | set state.error → END |
### R3 — State Serialization
`AgentState` is a `BaseModel` (fully JSON-serializable at all times).
Large objects (documents, embeddings, raw HTML) → stored in DB /
object-store; only UUID / string references live inside state.
Validate with `AgentState.validate_serializable(state)` after every
node that mutates state.
### R4 — Approval Gates
Any tool whose `ToolDeclaration.requires_approval` is `True`
MUST call `interrupt(ApprovalPayload(...))` before execution.
Resume via `Command(resume=ApprovalDecision(...))`.
There are NO exceptions to this rule.
### R5 — Audit Completeness
Every tool execution — success, failure, AND skipped — MUST produce
an `AuditEvent` written by `AuditLogger`.
A missing audit entry is a P0 production bug.
`audit_event` rows are append-only; enforced by DB trigger
`trg_audit_event_no_mutation`.
### R6 — Sensitive Data
`Sanitizer.mask()` MUST be applied to all user inputs before they
appear in logs, audit payloads, or API responses.
Patterns:
- Phone `1[3-9]\d{9}` → `138****8888`
- ID card 18-digit → `****19700101****`
- password / key fields → `[REDACTED]`
### R7 — Async Discipline
No synchronous blocking inside any `async def`.
CPU-bound operations → `asyncio.to_thread()`.
Every external I/O call carries an explicit `timeout=` parameter.
No `asyncio.create_task()` fire-and-forget without error collection.
### R8 — Decimal for Money
All cost / fee / score calculations use `Decimal` with explicit
precision and `ROUND_HALF_UP`.
DB columns: `Numeric(12,6)` for USD, `Numeric(12,4)` for CNY,
`Numeric(5,4)` for evidence scores.
NEVER store or compute these values as `float`.
## Settings Access Pattern
```python
# CORRECT — lazy singleton, test-safe
from core.config import get_settings
s = get_settings()
# WRONG — instantiated at import time, breaks test monkeypatching
from core.config import settings # this symbol does not exist
```
## Gate Definitions
Every task MUST pass its gate before being declared complete.
### Static Gate (no Docker required)
```bash
uv sync --frozen
uv run ruff check . --select E,F,I --quiet
uv run mypy core/ api/ --ignore-missing-imports --no-error-summary
uv run pytest tests/unit/ -x -q --tb=short
```
### Enhanced Gate (Docker required)
```bash
docker compose up -d --wait
uv run alembic upgrade head
uv run pytest tests/integration/ -x -q --tb=short
bash scripts/health_check.sh
```
## Testing Standards
- Unit tests : zero external I/O; mock DB, HTTP, Claude, Milvus, Redis
- Integration : real services via testcontainers or running Docker stack
- Contract : real Claude API; `@pytest.mark.claude_api`;
skipped automatically if `ANTHROPIC_API_KEY` is unset
- Coverage gate: `--cov-fail-under=80` across `core/` and `api/`
- Secret injection: via `tests/conftest.py` `settings_override` fixture;
NEVER `os.environ` assignment in test files directly
## Commit Convention
```
feat(engine): implement interrupt/resume approval in act_node
fix(verify): correct EWMA weight formula in EvidenceScorer
test(api): add integration test for /approve expired-token path
chore(deps): add alembic, cryptography to pyproject.toml
refactor(tools): extract _resolve_params to standalone module
```
+160 -10
View File
@@ -1,13 +1,163 @@
# api/AGENTS.md
# API Layer — Submodule Instructions
## 范围
`api/` 负责 FastAPI 路由、请求响应契约、中间件和权限控制。
## FastAPI Patterns
- All route handlers: `async def`
- Return types: always a declared Pydantic response model — NEVER
raw `dict`, `None`, or untyped `Any`
- Error format (ALL error responses, without exception):
```python
class ErrorResponse(BaseModel):
code: str # machine-readable, e.g. "TOOL_NOT_ALLOWED"
message: str # human-readable Chinese message
trace_id: str
details: dict = Field(default_factory=dict)
```
- HTTP status codes:
| Situation | Code |
|--------------------------------------|------|
| Success | 200 |
| Async task accepted | 202 |
| Client validation error | 400 |
| Missing / invalid JWT | 401 |
| Insufficient permission | 403 |
| Resource not found | 404 |
| State conflict (e.g. already decided)| 409 |
| Pydantic validation failure | 422 |
| Dependency unhealthy | 503 |
| Unhandled server error | 500 |
## 约束
- 路由层不承载业务规则,仅做参数校验与编排。
- 权限、限流、鉴权失败返回明确 HTTP 状态码与错误码。
- 合同模型字段禁止可变默认值,统一 `Field(default_factory=...)`。
## Settings Access
```python
# CORRECT
from core.config import get_settings
s = get_settings()
## 工程要求
- 新增路由必须包含至少一个契约测试。
- 中间件顺序变更需说明兼容性影响。
# WRONG — do not import a module-level `settings` variable
from core.config import settings # symbol does not exist; will ImportError
```
## AgentEngine Lifecycle
- Initialised ONCE in `api/main.py` `lifespan` startup
- Stored at `app.state.engine`
- NEVER instantiate `AgentEngine` inside a route handler or
per-request dependency
## Middleware Stack (outermost → innermost, registration order matters)
1. `TracingMiddleware`
- Read `X-Trace-ID` header; generate `uuid4()` if absent
- Store in `request.state.trace_id`
- Add `X-Trace-ID` to response headers
- Create OTel HTTP span (method, path, status_code, duration_ms)
- Bind `trace_id` to `structlog` context for the request lifetime
2. `LoggingMiddleware`
- Log request: method, path, content-length, trace_id
- Log response: status_code, duration_ms, trace_id
- Sanitize `Authorization` header (log only first 12 chars + `…`)
- NEVER log request body (may contain secrets)
3. `RateLimitMiddleware`
- Redis token-bucket keyed on `tenant_id` (extracted from JWT claim
without full JWT validation — just base64-decode the payload)
- Limit: `settings.REDIS_RATE_LIMIT_PER_MINUTE` per tenant per minute
- Exceeded → 429 `ErrorResponse(code="RATE_LIMIT_EXCEEDED", …)`
4. CORS
- `ENVIRONMENT=development` → `allow_origins=["*"]`
- `ENVIRONMENT=production` → `allow_origins=settings.ALLOWED_ORIGINS`
Authentication is handled by `require_auth` FastAPI `Depends`, NOT a
global middleware, so that `/health`, `/docs`, `/redoc`, `/openapi.json`
remain public.
## RBAC Permission Matrix
```python
ROLE_PERMISSIONS: dict[str, set[str]] = {
"admin": {"chat:create","chat:read","approve:write",
"trace:read","user:manage"},
"manager": {"chat:create","chat:read","approve:write","trace:read"},
"analyst": {"chat:create","chat:read","trace:read"},
"viewer": {"chat:read","trace:read"},
}
```
Usage pattern:
```python
@router.post("/approve/{trace_id}")
async def approve(
trace_id: str,
body: ApprovalRequest,
user: UserContext = Depends(require_auth),
_: None = Depends(require_permission("approve:write")),
db: AsyncSession = Depends(get_db),
): ...
```
## SSE Event Contract (/api/v1/chat)
All server-sent events MUST conform to this schema:
```
event: node_complete
data: {"node":"ask","status":"success","duration_ms":234,"trace_id":"…"}
event: approval_required
data: {
"trace_id":"…","step_id":"…",
"action_description":"…","risk_level":"high",
"approver_role":"manager","params_preview":{…}
}
event: final_result
data: {
"output_type":"document|text|insufficient_data|clarification_needed",
"content":"…",
"citations":[…],
"evidence_score":"0.8700", ← Decimal serialised as string
"evidence_level":"sufficient",
"trace_id":"…"
}
event: error
data: {"code":"TOOL_FAILED","message":"…","trace_id":"…"}
data: [DONE]
```
Notes:
- `evidence_score` MUST be serialised as a string to preserve
`Decimal` precision in JSON (avoids float rounding)
- The `[DONE]` terminator MUST always be sent, even after an error
- Do NOT send any events after `[DONE]`
## Approval Endpoint Rules (/api/v1/approve/{trace_id})
Exact sequence — do not reorder:
1. Verify caller has `approve:write` permission
2. Load `PendingApproval` where `trace_id = :trace_id AND status = 'pending'`
→ 404 if not found
3. Check `expired_at > now()`
→ 409 `{"code":"APPROVAL_EXPIRED"}` if expired
4. Atomic DB transaction:
a. `PendingApproval.status` ← decision
b. `PendingApproval.decider_id` ← `user.user_id`
c. `PendingApproval.decided_at` ← `now()`
d. `TraceSession.status` ← `"running"` (if approved) or `"failed"`
5. Fire-and-forget resume (do NOT await completion in the request):
`asyncio.create_task(engine.resume(session_id, Command(resume=…)))`
with structured error logging on task failure
6. Return 202: `{"status":"resuming","trace_id":"…"}`
## Health Endpoint Contract (GET /health)
```python
class HealthResponse(BaseModel):
status: Literal["ok", "degraded"]
version: str
environment: str
services: dict[str, bool] # {"db": True, "redis": True, "milvus": False, …}
checked_at: datetime
# Any service False → HTTP 503 (body still JSON, allows LB to see detail)
# All services True → HTTP 200
```
Real connectivity checks (not just config presence):
- `db` : `SELECT 1` via async engine
- `redis` : `PING` via aioredis
- `milvus`: `list_collections()` via pymilvus
+147 -11
View File
@@ -1,14 +1,150 @@
# core/AGENTS.md
# Core Engine — Submodule Instructions
## 范围
`core/` 负责智能体引擎、LLM 路由、工具执行、校验、审计与可观测基础能力。
## Module Responsibilities
```
core/engine/state.py AgentState (BaseModel, fully serializable)
core/engine/nodes.py 5 node functions: ask, plan, act, verify, report
core/engine/graph.py StateGraph wiring + conditional edges
core/profile/schema.py Pydantic Profile model (frozen, validated)
core/profile/loader.py YAML → Profile with full contract validation
core/profile/registry.py ProfileRegistry singleton with hot-reload
core/tools/base.py BaseTool, ToolResult, ToolDeclaration
core/tools/executor.py ToolExecutor — whitelist + permission + audit
core/tools/registry.py ToolRegistry singleton
core/llm/claude_client.py Async Claude API wrapper
core/llm/prompt_builder.py All prompt text centralised here
core/router/model_router.py Routing logic (cloud vs. local)
core/verify/scorer.py EvidenceScorer (formula + threshold)
core/audit/logger.py 3-tier AuditLogger (L1 PG / L2 OTel / L3 enc)
core/audit/sanitizer.py PII masking
core/audit/encryptor.py AES-256-GCM for L3 payloads
```
## 约束
- 保持纯业务内核,禁止引入 API 框架对象。
- 所有状态对象、输入输出契约需类型稳定、可序列化。
- 成本、预算、阈值统一 `Decimal`。
- 配置访问必须走 `get_settings()`。
## LangGraph Strict Patterns
## 工程要求
- 复杂分支逻辑应有最小单元测试覆盖。
- 审计与加密路径出现异常时必须返回可诊断错误,不可静默吞错。
### State
- `AgentState(BaseModel)` — JSON-serializable at all times
- `frozen=False` (nodes mutate state in-place and return it)
- Large objects stored externally; state holds only references (UUIDs / paths)
- After every node that mutates state, call
`AgentState.validate_serializable(state)` in debug/test mode
### Checkpointer
- Production : `AsyncPostgresSaver` (from langgraph-checkpoint-postgres)
- Tests only : `MemorySaver`
- NEVER use `MemorySaver` in production code paths
### Conditional Edges
- Use string literals for node names in `add_conditional_edges`
- No lambda closures that capture mutable external state
- Every node MUST have a defined exit route to `END` on error
### Approval Flow
```
act_node detects requires_approval=True
→ build ApprovalPayload
→ interrupt(payload) # LangGraph pauses here
→ [ external /approve call ]
→ Command(resume=ApprovalDecision(...))
→ act_node resumes from saved checkpoint
```
## Exception Boundary
### Node Error Boundary
Each node wraps its body in `try/except Exception`:
- Unrecoverable error → `state.error = {"code":…, "message":…, "node":…}`
return state (graph routes to END via conditional edge)
- NEVER let an unhandled exception escape a node function
## Claude API Contract
- Use the `anthropic` SDK version locked in `uv.lock`
- Follow SDK type hints exactly; do NOT hardcode beta-header string
literals; import SDK constants where available
- Structured output: use `tool_choice={"type": "any"}` to force
tool calling in Ask / Plan / Verify nodes
- Parse tool results: `content_block.input` (dict) directly —
NEVER regex-parse the text representation of tool calls
- Prompt Caching: add `cache_control={"type": "ephemeral"}` to
system message blocks; if the SDK version does not support this
field, omit it gracefully (do NOT crash)
- Token tracking: record `usage.input_tokens`, `usage.output_tokens`,
`usage.cache_read_input_tokens` after EVERY API call
- Timeout: pass `httpx.Timeout(connect=10, read=60, write=10, pool=5)`
to `AsyncAnthropic` constructor — NOT per-call
## Tool Contract
Every class in `core/tools/` that implements a tool MUST:
1. Subclass `BaseTool`
2. Declare a `@property declaration(self) -> ToolDeclaration`
3. Declare a `@property input_schema(self) -> type[BaseModel]`
4. Implement `async _execute_impl(params, user_id, tenant_id,
data_scope) -> ToolResult`
5. Rely on the base `execute()` for: param validation → retry
(tenacity, max=2, wait_fixed=1 s) → timeout
(asyncio.wait_for, limit = declaration.max_duration_ms / 1000 s)
→ catch-all → ToolResult(success=False, error=sanitized)
6. NEVER check the tool whitelist inside a tool
7. Honour Mock Policy: check `get_settings().ENVIRONMENT` before
falling back to mock data; always set `meta["mock"] = True`
## Evidence Scoring Formula
DO NOT modify this formula without also updating
`tests/unit/test_scorer.py` and the Profile schema defaults.
```python
# All operands and result are Decimal, rounded to 4 decimal places
score = (
Decimal("0.4") * retrieval_score
+ Decimal("0.3") * completeness_score
+ Decimal("0.3") * historical_accuracy
)
# Default thresholds (overridable per Profile):
# sufficient : score >= Decimal("0.85") → normal output
# limited : score >= Decimal("0.60") → output with ⚠️ warning
# insufficient : score < Decimal("0.60") → reject, explain gaps
```
## Audit Layer Contracts
### L1 — PostgreSQL (AuditEvent)
- Append-only: DB trigger `trg_audit_event_no_mutation` enforces
no UPDATE or DELETE at the database level
- ORM `AuditEventRepository` exposes NO `update`, `delete`,
`remove`, `modify`, or `patch` methods
- `mock_flag` column MUST be set to `True` when `ToolResult.meta`
contains `"mock": True`
### L2 — OpenTelemetry spans
- Auto-collected by `@tool_span`, `@llm_span`, `@agent_span` decorators
- Span attributes follow OpenTelemetry GenAI semantic conventions where
applicable
### L3 — Encrypted payloads (AuditPayload)
- Cipher: AES-256-GCM
- Key source: `get_settings().AUDIT_KEYRING_JSON` parsed as
`dict[key_id, base64_key]`; active key from `AUDIT_ACTIVE_KEY_ID`
- IV: `os.urandom(12)` — fresh per encryption, prepended to ciphertext
- Stored layout: `IV (12 B) | tag (16 B) | ciphertext`
- Key rotation: use a new `key_id`; old payloads remain decryptable
as long as old key is still in keyring
## _resolve_params Template Syntax
`act_node` resolves parameter templates before every tool call.
Supported patterns (all `{{…}}` delimited):
| Template | Resolves to |
|---------------------------------------|------------------------------------------------|
| `{{user_input}}` | `state.user_input` (raw string) |
| `{{user_input.field_name}}` | `state.extracted_params.get("field_name")` |
| `{{steps.step_id.summary}}` | `state.step_outputs["step_id"]["summary"]` |
| `{{steps.step_id.data_ref}}` | `state.step_outputs["step_id"]["data_ref"]` |
| `{{profile.output_template}}` | `active_profile.output_template` |
| `{{profile.terminology}}` | `active_profile.terminology` |
Rules:
- Recurse into dict values and list items
- Unresolvable key → keep original `{{…}}` string; log a WARNING;
record in `meta["unresolved_templates"]` of the step result
- NEVER raise an exception for a missing template key
+123 -9
View File
@@ -1,11 +1,125 @@
# tests/AGENTS.md
# Tests — Submodule Instructions
## 测试分层
- `unit/`: 纯函数与小范围组件测试,禁止外部依赖。
- `integration/`: 数据库、缓存、向量库、第三方 API 集成测试。
- `contracts/`: API 请求响应与权限矩阵契约测试。
## Test Isolation Contract
| Layer | External I/O allowed? | Services needed |
|-------------|----------------------|--------------------------|
| unit/ | NONE | none |
| integration/| YES (mocked or real) | testcontainers or Docker |
| contracts/ | Real Claude API only | ANTHROPIC_API_KEY set |
## 约束
- 测试中密钥统一由 fixture 注入,不从真实环境读取。
- 测试数据必须可重复、可清理。
- 对审计、鉴权、预算阈值相关逻辑优先补全边界测试。
## Secret Injection (MANDATORY pattern)
```python
# tests/conftest.py — root fixture, autouse=True
@pytest.fixture(autouse=True)
def settings_override(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-ant-test-not-real")
monkeypatch.setenv("DATABASE_URL",
"postgresql+asyncpg://test:test@localhost/test_db")
monkeypatch.setenv("JWT_SECRET_KEY",
"test_jwt_secret_key_32chars_padded_x")
monkeypatch.setenv("AUDIT_KEYRING_JSON",
'{"test_key_1": "<base64-of-32-A-bytes>"}')
monkeypatch.setenv("AUDIT_ACTIVE_KEY_ID", "test_key_1")
get_settings.cache_clear()
yield
get_settings.cache_clear()
```
NEVER assign `os.environ[…]` directly in test files.
NEVER import a module-level `settings` object in test files.
## Fixture Catalogue (tests/conftest.py)
| Fixture name | Scope | Description |
|------------------------|----------|------------------------------------------|
| `settings_override` | function | autouse; injects all required secrets |
| `async_engine` | session | in-memory SQLite or testcontainers PG |
| `db_session` | function | isolated session; rolls back after test |
| `async_client` | function | `httpx.AsyncClient` against test app |
| `mock_claude` | function | respx router for api.anthropic.com |
| `sample_presales_profile` | session | minimal Profile object (no file I/O) |
| `tool_registry` | function | registry with mock `_execute_impl` |
| `agent_graph_memory` | function | graph with MemorySaver (tests only) |
## Test Naming Convention
```
test_<unit_under_test>_<scenario>_<expected_outcome>
Examples:
test_ask_node_missing_required_fields_returns_clarification
test_act_node_high_risk_tool_triggers_interrupt
test_tool_executor_unlisted_tool_raises_and_audits
test_evidence_scorer_below_reject_threshold_returns_insufficient
test_cost_calculation_batch_mode_halves_cost
test_audit_event_repo_has_no_update_method
```
## Coverage Requirements (enforced by gate)
```
core/engine/ ≥ 90 %
core/verify/ ≥ 95 % ← formula is critical path
core/audit/ ≥ 85 %
core/tools/ ≥ 80 %
api/routes/ ≥ 85 %
api/middleware/ ≥ 80 %
─────────────────────────────
core/ + api/ overall ≥ 80 % ← enforced by --cov-fail-under=80
```
## Mock Claude API Pattern
```python
# tests/fixtures/mock_claude.py
import respx, httpx, json
def make_tool_use_response(tool_name: str, input_dict: dict,
input_tokens: int = 800,
output_tokens: int = 150,
cache_read_tokens: int = 600) -> dict:
return {
"id": "msg_mock_001",
"type": "message",
"role": "assistant",
"content": [{
"type": "tool_use",
"id": "toolu_mock_001",
"name": tool_name,
"input": input_dict,
}],
"model": "claude-sonnet-4-6-20260217",
"stop_reason": "tool_use",
"usage": {
"input_tokens": input_tokens,
"output_tokens": output_tokens,
"cache_read_input_tokens": cache_read_tokens,
"cache_creation_input_tokens": 0,
},
}
def make_text_response(text: str, input_tokens: int = 500,
output_tokens: int = 800) -> dict:
return {
"id": "msg_mock_002",
"type": "message",
"role": "assistant",
"content": [{"type": "text", "text": text}],
"model": "claude-sonnet-4-6-20260217",
"stop_reason": "end_turn",
"usage": {
"input_tokens": input_tokens,
"output_tokens": output_tokens,
"cache_read_input_tokens": 0,
"cache_creation_input_tokens": 0,
},
}
```
## Acceptance Thresholds (contract tests with real Claude API)
These thresholds are checked by `tests/contracts/` and must pass
before any production deployment:
| Metric | Gate |
|--------------------------------|------------|
| Task success rate (presales) | ≥ 90 % |
| Citation coverage | ≥ 90 % |
| Hallucination rate (sampled) | ≤ 5 % |
| RAG Recall@5 | ≥ 95 % |
| P95 end-to-end latency | ≤ 15 s |
| Evidence score (sufficient) | ≥ 0.85 |