Compare commits
3 commits
d61f0f2993
...
2053a9d26c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2053a9d26c | ||
|
|
ea6a0a399a | ||
|
|
fd8f9c7816 |
4 changed files with 325 additions and 4 deletions
97
agents/prompts/cto_advisor.md
Normal file
97
agents/prompts/cto_advisor.md
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
You are a CTO Advisor for the Kin multi-agent orchestrator.
|
||||||
|
|
||||||
|
Your job: evaluate an architectural plan from a strategic CTO perspective — business risks, scalability, platform vs product complexity — and issue a recommendation. No code changes, analysis only.
|
||||||
|
|
||||||
|
## Input
|
||||||
|
|
||||||
|
You receive:
|
||||||
|
- PROJECT: id, name, path, tech stack
|
||||||
|
- TASK: id, title, brief describing the feature or change under review
|
||||||
|
- DECISIONS: known conventions and gotchas for this project
|
||||||
|
- PREVIOUS STEP OUTPUT: architect plan (required) or department head output with context_packet
|
||||||
|
|
||||||
|
## Working Mode
|
||||||
|
|
||||||
|
0. If PREVIOUS STEP OUTPUT contains a `context_packet` field — read it FIRST before opening any files. It contains essential handoff context from the prior agent.
|
||||||
|
1. Read `DESIGN.md` and `agents/specialists.yaml` to understand the current architecture and project constraints.
|
||||||
|
2. Read the architectural plan from PREVIOUS STEP OUTPUT — focus on fields: `implementation_steps`, `schema_changes`, `affected_modules`. If no architectural plan is present in PREVIOUS STEP OUTPUT, return blocked immediately.
|
||||||
|
3. For each architectural decision, evaluate across three axes:
|
||||||
|
- **Business risks** — what could go wrong from a product/business standpoint
|
||||||
|
- **Scalability** — horizontal scaling ability, bottlenecks, single points of failure
|
||||||
|
- **Platform vs Product complexity** — does this solution introduce platform-level abstraction for a product-level problem?
|
||||||
|
4. For each identified risk, assign severity using these definitions:
|
||||||
|
- `critical` — blocks launch; the system cannot go live without resolving this
|
||||||
|
- `high` — blocks deploy; must be resolved before production release
|
||||||
|
- `medium` — flagged with conditions; acceptable if mitigation is applied
|
||||||
|
- `low` — note only; acceptable as-is, worth monitoring
|
||||||
|
5. Issue a strategic verdict:
|
||||||
|
- `approved` — no critical or high risks found; safe to proceed
|
||||||
|
- `concerns` — medium risks present or tradeoffs worth escalating; proceed with awareness
|
||||||
|
- `critical_concerns` — one or more critical or high risks found; do not proceed without resolution
|
||||||
|
|
||||||
|
## Focus On
|
||||||
|
|
||||||
|
- `platform_vs_product`: always evaluate and state explicitly — even if the answer is "product" (correct level), the field is REQUIRED. This distinguishes over-engineered solutions from appropriately scoped ones.
|
||||||
|
- Scalability score 1–5: 1 = single node, no growth path; 5 = horizontally scalable, stateless, distributed-ready.
|
||||||
|
- Severity calibration: reserve `critical` for launch-blocking issues (data loss, security hole, core failure mode). Do not inflate severity — it degrades signal quality.
|
||||||
|
- Business risk specificity: avoid generic formulations like "might break". Name the concrete failure scenario and its business impact.
|
||||||
|
- `strategic_verdict = approved` requires: zero critical risks AND zero high risks. Any high risk → at minimum `concerns`.
|
||||||
|
|
||||||
|
## Quality Checks
|
||||||
|
|
||||||
|
- `scalability_assessment` is present with all 4 sub-fields: `score`, `notes`, `platform_vs_product`, `complexity_appropriateness`
|
||||||
|
- `strategic_risks` is an array; each element has `risk`, `severity`, and `mitigation`
|
||||||
|
- `strategic_verdict` is exactly one of: `approved`, `concerns`, `critical_concerns`
|
||||||
|
- `recommendation` is a concrete actionable string, not a summary of the risks already listed
|
||||||
|
- `platform_vs_product` is explicitly set — never omitted even when the answer is `product`
|
||||||
|
- No code is written, no files are modified — output is analysis only
|
||||||
|
|
||||||
|
## Return Format
|
||||||
|
|
||||||
|
Return ONLY valid JSON (no markdown, no explanation):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "done",
|
||||||
|
"scalability_assessment": {
|
||||||
|
"score": 3,
|
||||||
|
"notes": "Single SQLite instance creates a write bottleneck beyond ~100 concurrent users",
|
||||||
|
"platform_vs_product": "product",
|
||||||
|
"complexity_appropriateness": "appropriate"
|
||||||
|
},
|
||||||
|
"strategic_risks": [
|
||||||
|
{
|
||||||
|
"risk": "No rollback plan for DB schema migration",
|
||||||
|
"severity": "high",
|
||||||
|
"mitigation": "Add pg_dump backup step + blue-green deployment before ALTER TABLE runs"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"strategic_verdict": "concerns",
|
||||||
|
"recommendation": "Proceed after adding a DB backup and rollback procedure to the deployment runbook. No architectural changes required.",
|
||||||
|
"notes": "If user growth exceeds 500 DAU within 6 months, revisit SQLite → PostgreSQL migration plan."
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Valid values for `status`: `"done"`, `"partial"`, `"blocked"`.
|
||||||
|
|
||||||
|
- `"partial"` — analysis completed with limited data; include `"partial_reason": "..."`.
|
||||||
|
- `"blocked"` — unable to proceed; include `"reason": "..."` and `"blocked_at": "<ISO-8601 datetime>"`.
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
- Do NOT write or modify any files — produce analysis and recommendations only
|
||||||
|
- Do NOT implement code — strategic assessment output only
|
||||||
|
- Do NOT evaluate without reading the architectural plan first
|
||||||
|
- Do NOT return `strategic_verdict: approved` if any critical or high severity risk is present
|
||||||
|
- Do NOT omit `platform_vs_product` — it is required even when the answer is `product`
|
||||||
|
- Do NOT add new Python dependencies, modify DB schema, or touch frontend files
|
||||||
|
|
||||||
|
## Blocked Protocol
|
||||||
|
|
||||||
|
If you cannot perform the task (no architectural plan in PREVIOUS STEP OUTPUT, no file access, ambiguous requirements, task outside your scope), return this JSON **instead of** the normal output:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"status": "blocked", "reason": "<clear explanation>", "blocked_at": "<ISO-8601 datetime>"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Use current datetime for `blocked_at`. Do NOT guess or partially complete — return blocked immediately.
|
||||||
|
|
@ -272,6 +272,23 @@ specialists:
|
||||||
model_recommendation: "{ recommended_model: string, rationale: string, alternatives: array of { model, tradeoffs } }"
|
model_recommendation: "{ recommended_model: string, rationale: string, alternatives: array of { model, tradeoffs } }"
|
||||||
notes: string
|
notes: string
|
||||||
|
|
||||||
|
cto_advisor:
|
||||||
|
name: "CTO Advisor"
|
||||||
|
model: opus
|
||||||
|
tools: [Read, Grep, Glob]
|
||||||
|
description: "Strategic technical reviewer: evaluates architectural plans for business risks, scalability, and platform vs product complexity. Analysis-only — no code changes. See also: constitutional_validator (alignment gate), architect (plan author)."
|
||||||
|
permissions: read_only
|
||||||
|
context_rules:
|
||||||
|
decisions: all
|
||||||
|
modules: all
|
||||||
|
output_schema:
|
||||||
|
status: "done | partial | blocked"
|
||||||
|
scalability_assessment: "{ score: 1-5, notes: string, platform_vs_product: platform|product|hybrid, complexity_appropriateness: appropriate|over-engineered|under-engineered }"
|
||||||
|
strategic_risks: "array of { risk: string, severity: critical|high|medium|low, mitigation: string }"
|
||||||
|
strategic_verdict: "approved | concerns | critical_concerns"
|
||||||
|
recommendation: "string — final strategic recommendation"
|
||||||
|
notes: string
|
||||||
|
|
||||||
knowledge_synthesizer:
|
knowledge_synthesizer:
|
||||||
name: "Knowledge Synthesizer"
|
name: "Knowledge Synthesizer"
|
||||||
model: sonnet
|
model: sonnet
|
||||||
|
|
|
||||||
|
|
@ -115,11 +115,11 @@ class TestAllPromptsContainStandardStructure:
|
||||||
class TestPromptCount:
|
class TestPromptCount:
|
||||||
"""Проверяет, что число промптов не изменилось неожиданно."""
|
"""Проверяет, что число промптов не изменилось неожиданно."""
|
||||||
|
|
||||||
def test_prompt_count_is_28(self):
|
def test_prompt_count_is_29(self):
|
||||||
"""В agents/prompts/ ровно 28 файлов .md."""
|
"""В agents/prompts/ ровно 29 файлов .md."""
|
||||||
count = len(_prompt_files())
|
count = len(_prompt_files())
|
||||||
assert count == 28, ( # 28 промптов — актуально на 2026-03-19, +repo_researcher (KIN-DOCS-006, см. git log agents/prompts/)
|
assert count == 29, ( # 29 промптов — актуально на 2026-03-19, +cto_advisor (KIN-DOCS-007, см. git log agents/prompts/)
|
||||||
f"Ожидалось 28 промптов, найдено {count}. "
|
f"Ожидалось 29 промптов, найдено {count}. "
|
||||||
"Если добавлен новый промпт — обнови этот тест."
|
"Если добавлен новый промпт — обнови этот тест."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
||||||
207
tests/test_kin_docs_007_regression.py
Normal file
207
tests/test_kin_docs_007_regression.py
Normal file
|
|
@ -0,0 +1,207 @@
|
||||||
|
"""Regression tests for KIN-DOCS-007 — cto_advisor: strategic technical reviewer.
|
||||||
|
|
||||||
|
Acceptance criteria:
|
||||||
|
1. cto_advisor зарегистрирован в specialists.yaml с model=opus
|
||||||
|
2. cto_advisor имеет permissions=read_only
|
||||||
|
3. output_schema содержит поля: status, scalability_assessment, strategic_risks,
|
||||||
|
strategic_verdict, recommendation, notes
|
||||||
|
4. agents/prompts/cto_advisor.md существует и содержит все 5 стандартных секций
|
||||||
|
5. Промпт содержит поля выходной схемы: scalability_assessment, strategic_verdict
|
||||||
|
6. Промпт содержит Blocked Protocol (blocked_at)
|
||||||
|
7. cto_advisor НЕ входит ни в один department workers (опциональный специалист)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
|
||||||
|
SPECIALISTS_YAML = Path(__file__).parent.parent / "agents" / "specialists.yaml"
|
||||||
|
PROMPTS_DIR = Path(__file__).parent.parent / "agents" / "prompts"
|
||||||
|
CTO_ADVISOR_PROMPT = PROMPTS_DIR / "cto_advisor.md"
|
||||||
|
|
||||||
|
REQUIRED_SECTIONS = [
|
||||||
|
"## Working Mode",
|
||||||
|
"## Focus On",
|
||||||
|
"## Quality Checks",
|
||||||
|
"## Return Format",
|
||||||
|
"## Constraints",
|
||||||
|
]
|
||||||
|
|
||||||
|
CTO_ADVISOR_REQUIRED_SCHEMA_FIELDS = {
|
||||||
|
"status",
|
||||||
|
"scalability_assessment",
|
||||||
|
"strategic_risks",
|
||||||
|
"strategic_verdict",
|
||||||
|
"recommendation",
|
||||||
|
"notes",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_yaml():
|
||||||
|
return yaml.safe_load(SPECIALISTS_YAML.read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
# ===========================================================================
|
||||||
|
# 1. cto_advisor — регистрация в specialists.yaml
|
||||||
|
# ===========================================================================
|
||||||
|
|
||||||
|
class TestCtoAdvisorSpecialistsEntry:
|
||||||
|
"""cto_advisor зарегистрирован в specialists.yaml с корректной базовой структурой."""
|
||||||
|
|
||||||
|
def test_cto_advisor_exists_in_specialists(self):
|
||||||
|
"""cto_advisor присутствует в секции specialists."""
|
||||||
|
data = _load_yaml()
|
||||||
|
assert "cto_advisor" in data.get("specialists", {}), (
|
||||||
|
"cto_advisor отсутствует в specialists.yaml"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_cto_advisor_model_is_opus(self):
|
||||||
|
"""cto_advisor использует модель opus (стратегический ревьюер требует Opus)."""
|
||||||
|
data = _load_yaml()
|
||||||
|
role = data["specialists"]["cto_advisor"]
|
||||||
|
assert role.get("model") == "opus", (
|
||||||
|
f"Ожидался model=opus, получили: {role.get('model')}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_cto_advisor_permissions_is_read_only(self):
|
||||||
|
"""cto_advisor имеет permissions=read_only (анализ без изменений)."""
|
||||||
|
data = _load_yaml()
|
||||||
|
role = data["specialists"]["cto_advisor"]
|
||||||
|
assert role.get("permissions") == "read_only", (
|
||||||
|
f"Ожидался permissions=read_only, получили: {role.get('permissions')}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_cto_advisor_tools_include_read_grep_glob(self):
|
||||||
|
"""cto_advisor имеет инструменты Read, Grep, Glob."""
|
||||||
|
data = _load_yaml()
|
||||||
|
tools = data["specialists"]["cto_advisor"].get("tools", [])
|
||||||
|
for tool in ("Read", "Grep", "Glob"):
|
||||||
|
assert tool in tools, f"cto_advisor должен иметь инструмент {tool!r}"
|
||||||
|
|
||||||
|
def test_cto_advisor_has_output_schema(self):
|
||||||
|
"""cto_advisor имеет поле output_schema."""
|
||||||
|
data = _load_yaml()
|
||||||
|
role = data["specialists"]["cto_advisor"]
|
||||||
|
assert "output_schema" in role, "cto_advisor должен иметь output_schema"
|
||||||
|
|
||||||
|
|
||||||
|
# ===========================================================================
|
||||||
|
# 2. cto_advisor — output_schema поля
|
||||||
|
# ===========================================================================
|
||||||
|
|
||||||
|
class TestCtoAdvisorOutputSchemaFields:
|
||||||
|
"""output_schema cto_advisor содержит все обязательные поля."""
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("required_field", sorted(CTO_ADVISOR_REQUIRED_SCHEMA_FIELDS))
|
||||||
|
def test_output_schema_contains_required_field(self, required_field):
|
||||||
|
"""output_schema cto_advisor содержит обязательное поле."""
|
||||||
|
data = _load_yaml()
|
||||||
|
schema = data["specialists"]["cto_advisor"]["output_schema"]
|
||||||
|
assert required_field in schema, (
|
||||||
|
f"output_schema cto_advisor обязана содержать поле {required_field!r}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ===========================================================================
|
||||||
|
# 3. cto_advisor — промпт существует и имеет стандартную структуру
|
||||||
|
# ===========================================================================
|
||||||
|
|
||||||
|
class TestCtoAdvisorPromptStructure:
|
||||||
|
"""agents/prompts/cto_advisor.md существует и содержит все 5 стандартных секций."""
|
||||||
|
|
||||||
|
def test_prompt_file_exists(self):
|
||||||
|
"""Файл agents/prompts/cto_advisor.md существует."""
|
||||||
|
assert CTO_ADVISOR_PROMPT.exists(), (
|
||||||
|
f"Промпт cto_advisor не найден: {CTO_ADVISOR_PROMPT}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_prompt_file_is_not_empty(self):
|
||||||
|
"""Файл cto_advisor.md не пустой."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert len(content.strip()) > 100
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("section", REQUIRED_SECTIONS)
|
||||||
|
def test_prompt_has_required_section(self, section):
|
||||||
|
"""Промпт cto_advisor.md содержит каждую из 5 стандартных секций."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert section in content, (
|
||||||
|
f"cto_advisor.md не содержит обязательную секцию {section!r}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_prompt_sections_in_correct_order(self):
|
||||||
|
"""5 обязательных секций расположены в правильном порядке."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
positions = [content.find(sec) for sec in REQUIRED_SECTIONS]
|
||||||
|
assert all(p != -1 for p in positions), "Не все 5 секций найдены в cto_advisor.md"
|
||||||
|
assert positions == sorted(positions), (
|
||||||
|
f"Секции в cto_advisor.md расположены не по порядку. "
|
||||||
|
f"Позиции: {dict(zip(REQUIRED_SECTIONS, positions))}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_prompt_has_input_section(self):
|
||||||
|
"""Промпт cto_advisor.md содержит секцию ## Input."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert "## Input" in content, "cto_advisor.md не содержит секцию '## Input'"
|
||||||
|
|
||||||
|
def test_prompt_contains_blocked_protocol(self):
|
||||||
|
"""Промпт cto_advisor.md содержит Blocked Protocol."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert "blocked_at" in content, (
|
||||||
|
"cto_advisor.md не содержит 'blocked_at' — Blocked Protocol обязателен"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ===========================================================================
|
||||||
|
# 4. cto_advisor — специфические поля выходной схемы в промпте
|
||||||
|
# ===========================================================================
|
||||||
|
|
||||||
|
class TestCtoAdvisorPromptOutputFields:
|
||||||
|
"""Промпт cto_advisor.md определяет ключевые поля стратегической оценки."""
|
||||||
|
|
||||||
|
def test_prompt_defines_scalability_assessment(self):
|
||||||
|
"""Промпт определяет поле 'scalability_assessment'."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert "scalability_assessment" in content, (
|
||||||
|
"cto_advisor.md должен определять поле 'scalability_assessment'"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_prompt_defines_strategic_verdict(self):
|
||||||
|
"""Промпт определяет поле 'strategic_verdict'."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert "strategic_verdict" in content, (
|
||||||
|
"cto_advisor.md должен определять поле 'strategic_verdict'"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_prompt_defines_strategic_risks(self):
|
||||||
|
"""Промпт определяет поле 'strategic_risks'."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert "strategic_risks" in content, (
|
||||||
|
"cto_advisor.md должен определять поле 'strategic_risks'"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_prompt_defines_platform_vs_product(self):
|
||||||
|
"""Промпт явно упоминает platform_vs_product."""
|
||||||
|
content = CTO_ADVISOR_PROMPT.read_text(encoding="utf-8")
|
||||||
|
assert "platform_vs_product" in content, (
|
||||||
|
"cto_advisor.md должен упоминать поле 'platform_vs_product'"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ===========================================================================
|
||||||
|
# 5. cto_advisor НЕ входит в department workers
|
||||||
|
# ===========================================================================
|
||||||
|
|
||||||
|
class TestCtoAdvisorNotInDepartments:
|
||||||
|
"""cto_advisor является опциональным специалистом — не входит в departments."""
|
||||||
|
|
||||||
|
def test_cto_advisor_not_in_any_department_workers(self):
|
||||||
|
"""cto_advisor не является обязательным членом ни одного департамента."""
|
||||||
|
data = _load_yaml()
|
||||||
|
for dept_name, dept in data.get("departments", {}).items():
|
||||||
|
workers = dept.get("workers", [])
|
||||||
|
assert "cto_advisor" not in workers, (
|
||||||
|
f"cto_advisor не должен быть в workers департамента '{dept_name}'. "
|
||||||
|
"cto_advisor — опциональный стратегический ревьюер, не постоянный участник."
|
||||||
|
)
|
||||||
Loading…
Add table
Add a link
Reference in a new issue