homelab/ansible/archive/documentation/reports/prompt-analysis-2026-01-09.md

title, date, author, type

title	date	author	type
Prompt Repository Analysis Report	2026-01-09	FrankGPT v4	Analysis

Prompt Repository Analysis Report

Executive Summary

Analyzed 26 prompt files across the .github/prompts/ directory. The repository contains a mix of production-ready, draft, and deprecated prompts with varying levels of sophistication.

Key Findings:

• Overlap Issues: 7 prompts have significant overlap and can be converged
• Deprecated Content: 3 "OLD.*" prompts should be archived or removed
• Draft Quality: 4 draft prompts lack implementation detail
• Top 5 Adjustments Needed: See Section 4 for detailed recommendations

---

1. Overlap Analysis: Convergence Opportunities

1.1 Service Management Workflows (High Overlap)

Affected Prompts:

• service-new.prompt.md
• service-review.prompt.md
• service-standardize.prompt.md
• service-troubleshoot.prompt.md
• service-decommission.prompt.md
• service-migration.prompt.md

Analysis: All six prompts share a common structure:

• Gated, step-by-step workflow
• Service-focused (Docker/Compose)
• Inventory integration (.github/knowledge/inventory.md)
• Explicit confirmation phrases
• Upstream documentation validation

Current Duplication:

• Pre-flight checks: SSH validation, service discovery logic repeated 6 times
• Inventory lookups: Same RAG pattern in service-new, service-review, service-standardize
• Gate structure: Nearly identical gate format across all service prompts
• Output format: All produce Markdown reports with similar sections

Convergence Recommendation:

Option A: Meta-Prompt Architecture (Recommended)

Create a single service-workflow.meta.prompt.md that defines:

# service-workflow.meta.prompt.md
workflows:
  - name: new
    gates: [0, 1, 2, 3, 4, 5]
    phases: [validate_sources, plan, analyze, patch, verify]
  - name: review
    gates: [0, 1, 2, 3, 4]
    phases: [discover, compare, report, patch, verify]
  - name: standardize
    gates: [0, 1, 2, 3, 4]
    phases: [locate, assess_risk, propose, apply, bounce]

Then reduce individual prompts to:

# service-new.prompt.md
---
extends: service-workflow.meta
workflow: new
---
[Workflow-specific customizations only]

Option B: Consolidate to Single File with Modes

Create service-management.prompt.md with mode flags:

# Usage
/service-management mode=new app=traefik
/service-management mode=review app=immich

Impact:

• Reduction: 6 files → 1 meta-prompt + 6 lightweight configs (or 1 unified file)
• Maintenance: Single source of truth for gates, inventory logic, security checks
• Risk: Low if phased migration

---

1.2 Session Management (Medium Overlap)

Affected Prompts:

• session-start.prompt.md
• session-end.prompt.md
• session-status.prompt.md
• OLD.session-start.prompt.md
• OLD.session-end.prompt.md
• OLD.session-status.prompt.md

Analysis:

• OLD. versions:* Clearly deprecated (no frontmatter, less structured)
• Current versions: All reference SESSION_SNAPSHOT*.md and perform RAG searches
• Overlap: All three prompts perform git status checks and snapshot retrieval

Convergence Recommendation:

Create: session-lifecycle.prompt.md

# session-lifecycle.prompt.md
modes:
  - start: Load snapshot, check drift, present menu
  - status: Quick realignment without full context
  - end: Generate snapshot, git operations

Impact:

• Reduction: 6 files → 1 unified prompt
• Archive: Move OLD.* to .github/prompts/archive/
• Risk: Very low, well-defined workflows

---

1.3 Markdown Conversion (Low Overlap but Redundant)

Affected Prompts:

• md2htmlDARK.prompt.md
• md2htmlLIGHT.prompt.md

Analysis: Both prompts are 90% identical, differing only in CSS color schemes.

Convergence Recommendation:

Single Prompt with Parameter:

# md2html.prompt.md
theme: ${input:theme}  # Options: dark, light

Impact:

• Reduction: 2 files → 1 file
• Risk: None

---

1.4 Draft Prompts (Should Be Eliminated or Completed)

Affected Prompts:

• service-decommission.prompt.md (draft)
• service-migration.prompt.md (draft)
• security-hardening.prompt.md (draft)
• performance-tuning.prompt.md (draft)

Analysis: All four are labeled "Draft" with generic checklists. They lack:

• Gate structure used in other prompts
• RAG integration
• Specific commands or validation steps
• Safety guardrails

Recommendation: Either:

1. Complete them using the pattern from service-new.prompt.md (gated workflow)
2. Archive them to .github/prompts/drafts/ until needed
3. Eliminate them if not actively used

Impact:

• Reduces "prompt noise" in main directory
• Sets quality bar for production prompts

---

2. Summary of Convergence Opportunities

Prompt Group	Current Count	Proposed Count	Reduction
Service Management	6	1 (+ 6 configs)	83% code duplication
Session Lifecycle	6	1	83%
Markdown HTML	2	1	50%
Drafts	4	0 (archived)	100%
Total Prompts	26	15–17	35–42% reduction

---

3. Quality Tiers

Tier 1: Production-Ready (8 prompts)

These prompts have complete implementation, gate structure, and clear success criteria:

1. ✅ service-new.prompt.md - Best-in-class structure
2. ✅ service-review.prompt.md - Comprehensive validation
3. ✅ service-standardize.prompt.md - Clear versioning logic
4. ✅ service-troubleshoot.prompt.md - OODA loop methodology
5. ✅ sso-onboarding.prompt.md - Authentik integration
6. ✅ create-commit.msg.prompt.md - RAG + Conventional Commits
7. ✅ clean-git.prompt.md - ReAct protocol, security checks
8. ✅ generateVulnerabilitiesReport.prompt.md - Structured output

Tier 2: Functional but Needs Polish (5 prompts)

9. 🟡 session-start.prompt.md - Missing detailed menu structure
10. 🟡 session-end.prompt.md - Template fallback not defined
11. 🟡 session-status.prompt.md - Drift detection logic vague
12. 🟡 reviewDockerCompose.prompt.md - Good but lacks gates
13. 🟡 ansible-tutor.prompt.md - Too brief, needs examples

Tier 3: Draft/Incomplete (9 prompts)

14. 🔴 service-decommission.prompt.md - Generic checklist only
15. 🔴 service-migration.prompt.md - Generic checklist only
16. 🔴 security-hardening.prompt.md - Generic checklist only
17. 🔴 performance-tuning.prompt.md - Generic checklist only
18. 🔴 create-readme.prompt.md - Incomplete template
19. 🔴 doc-lint.prompt.md - Phase 3 cut off mid-section
20. 🔴 md2htmlDARK.prompt.md - Functional but unmaintained
21. 🔴 md2htmlLIGHT.prompt.md - Duplicate
22. 🔴 README.md - Outdated references

Tier 4: Deprecated (3 prompts)

23. ⚫ OLD.session-start.prompt.md - Archive
24. ⚫ OLD.session-end.prompt.md - Archive
25. ⚫ OLD.create-commit-msg.prompt.md - Archive

---

4. Top 5 Prompts Needing Adjustments

🥇 Rank 1: reviewDockerCompose.prompt.md

Current State: Functional mentor-led review prompt but lacks the safety gates present in newer prompts.

Issues:

• No explicit confirmation gates (user can't stop workflow)
• No RAG integration with inventory or upstream docs
• Security audit logic not DRY (duplicates generateVulnerabilitiesReport.prompt.md)
• Missing rollback/recovery procedures

Impact Score: 9/10 (Used for critical security audits)

Recommended Improvements:

1. Add Gate Structure:

   ## Gate 0 — confirm target file
   User must reply exactly: `REVIEW: <compose-file>`
   
   ## Gate 1 — confirm findings
   User must reply exactly: `CONFIRM FINDINGS: <file>`
   
   ## Gate 2 — apply patches (if requested)
   User must reply exactly: `APPLY PATCHES: <file>`
   

2. Integrate with Vulnerability Report:

   ## Step 1 — Run Security Scan First
   Before manual review, execute:
   `/generateVulnerabilityReport` on the target file.
   Reference its output to avoid duplicating security checks.
   

3. Add Inventory Cross-Check:

   ## Step 2 — Validate Against Inventory
   Search `.github/knowledge/inventory.md` for the service.
   Compare declared image version vs. upstream latest.
   

4. Define Rollback:

   ## Recovery Procedure
   If changes break the service:
   1. `git checkout HEAD -- docker-compose.yml`
   2. `docker compose up -d`
   

---

🥈 Rank 2: ansible-tutor.prompt.md

Current State: Minimal prompt with good intent but lacks examples and structure.

Issues:

• Only ~15 lines (vs. 150+ in mature prompts)
• No gate structure for safety
• No examples of "good" vs. "bad" Ansible patterns
• Missing integration with existing playbooks in the repo
• No validation steps

Impact Score: 8/10 (Critical for teaching correct Ansible patterns)

Recommended Improvements:

1. Add Real-World Examples:

   ## Anti-Pattern Detection
   
   ### ❌ Bad: Shell Command Overuse
   ```yaml
   - name: Install Docker
     shell: curl -fsSL get.docker.com | bash
   

   ✅ Good: Idempotent Module Use

   - name: Install Docker
     apt:
       name: docker-ce
       state: present
   

2. Integrate with Existing Repo:

   ## Step 1 — Scan Existing Playbooks
   Before generating new code:
   1. Search workspace for `playbooks/*.yml`
   2. Extract patterns from `roles/*/tasks/main.yml`
   3. Align new code with existing style
   

3. Add Safety Gates:

   ## Gate 1 — Destructive Action Check
   If the proposed task includes any of these modules:
   - `shell` with `rm`, `dd`, `mkfs`
   - `file` with `state: absent` on system paths
   
   STOP and require explicit confirmation:
   User must reply: `I UNDERSTAND THE RISK: <task-name>`
   

4. Add Validation Workflow:

   ## Step 4 — Validation (Required)
   1. Run `ansible-playbook --syntax-check playbook.yml`
   2. Run `ansible-playbook --check playbook.yml` (dry-run)
   3. Provide copy/paste commands for user verification
   

---

🥉 Rank 3: session-status.prompt.md

Current State: Cognitive realignment prompt with vague drift detection logic.

Issues:

• "Drift Check" criteria poorly defined
• No quantifiable metrics (how far off-track is "drift"?)
• Missing actionable output (no clear commands)
• Phase 3 output format not standardized

Impact Score: 7/10 (Used frequently but output inconsistent)

Recommended Improvements:

1. Define Drift Quantitatively:

   ## Phase 2: Drift Calculation
   
   Compute drift score:
   - Active file NOT in snapshot "Files Changed": +2 drift
   - Terminal command NOT in snapshot "Next Steps": +1 drift
   - Open files > 5 and none in snapshot: +3 drift
   
   Drift Levels:
   - 0-1: ✅ On track
   - 2-3: ⚠️ Minor drift
   - 4+: 🚨 Major drift (pruning required)
   

2. Standardize HUD Output:

   ## Phase 3: Heads-Up Display (HUD)
   
   ### Status Report
   | Metric | Status | Action |
   |:---|:---|:---|
   | Drift Score | 4 🚨 | Pruning recommended |
   | Last Snapshot | 2h ago | Recent |
   | Active Task | Fix traefik labels | ⚠️ Not in snapshot |
   | Blockers | None | - |
   
   ### Recommended Command
   To realign, run:
   ```bash
   git checkout main
   cd _thelab/core/web/traefik
   

3. Add Memory Compression:

   ## Phase 4: Context Compression (If Drift > 5)
   Summarize current conversation in 3 bullets:
   - What we tried
   - What failed
   - What's next
   
   Then clear terminal history to reduce cognitive load.
   

---

🏅 Rank 4: Service Draft Prompts (Group)

Affected: service-decommission, service-migration, security-hardening, performance-tuning

Current State: All are generic checklists with no implementation logic.

Issues:

• No gate structure
• No integration with existing tooling
• No validation steps
• No examples or commands

Impact Score: 6/10 (Blocking future workflows)

Recommended Improvements:

Template to Follow: Use service-new.prompt.md as the gold standard.

Example: Complete service-decommission.prompt.md

---
description: "Guided, gated workflow for safely decommissioning a service."
---

# [ROLE]
You are a **DevOps SRE** acting as a **decomm specialist**.

# [GOAL]
Safely retire a service by:
- Backing up all data and configs
- Validating no dependencies
- Removing from production
- Updating documentation

# [INPUTS]
- Target service name: `${input:serviceName}`
- Backup destination: `${input:backupPath}`
- Inventory file path: `${input:inventoryFile}`

# [WORKFLOW]

## Gate 0 — select service for decommission
User must reply exactly: `DECOMMISSION: <service-name>`

## Step 1 — dependency scan
Search all `docker-compose.yml` files for:
- Services with `depends_on: <service-name>`
- Networks shared with this service
- Volumes referenced by other services

If dependencies found, STOP and list them.

## Gate 1 — confirm no dependencies
User must reply exactly: `CONFIRM NO DEPS: <service-name>`

## Step 2 — backup execution
1. Export service data: `docker compose cp <service>:/data ./backup/`
2. Export configs: `docker compose config > backup/compose.yml`
3. Verify backup integrity

## Gate 2 — confirm backup complete
User must reply exactly: `BACKUP VERIFIED: <service-name>`

## Step 3 — removal
1. Stop service: `docker compose stop <service>`
2. Remove container: `docker compose rm <service>`
3. Remove from compose file
4. Remove from inventory

## Step 4 — validation
1. `docker compose config` (syntax check)
2. `docker compose ps` (ensure service gone)
3. Check logs for errors in dependent services

## Gate 3 — confirm clean removal
User must reply exactly: `REMOVAL CONFIRMED: <service-name>`

## Step 5 — documentation update
Update:
- `.github/knowledge/inventory.md` (mark as decommissioned)
- `documentation/architecture/` (remove service from diagrams)
- `README.md` (if listed)

---

🏅 Rank 5: doc-lint.prompt.md

Current State: Incomplete - Phase 3 report section is cut off.

Issues:

• Output section truncated at line 50 (file continues to 61)
• Missing "Recommended Fixes" and "Low Priority" sections
• No auto-fix capability
• No integration with style.markdown.md validation

Impact Score: 5/10 (Useful but incomplete)

Recommended Improvements:

1. Complete the Report Structure:

   ### Phase 3: The Report
   
   #### 🔴 Critical Errors (Must Fix)
   - [Line 42] Missing language tag in code block
   - [Line 105] Broken internal link: `./missing-file.md`
   
   #### 🟡 Recommended Improvements
   - [Line 12] Use Sentence Case for heading
   - [Line 67] Replace "e.g." with "for example"
   
   #### 🔵 Low Priority / Style
   - [Line 89] Consider adding more whitespace between sections
   

2. Add Auto-Fix Mode:

   ## Phase 4: Auto-Fix (Optional)
   
   If user replies exactly: `AUTO-FIX: <filename>`
   
   Then apply these corrections:
   - Add language tags to code blocks
   - Convert headers to Sentence Case
   - Remove trailing whitespace
   - Fix relative links
   

3. Add Validation:

   ## Phase 5: Validation
   
   After fixes:
   1. Re-run lint
   2. Confirm 0 Critical Errors
   3. Generate pass/fail badge for README
   

---

5. Implementation Roadmap

Phase 1: Immediate Cleanup (Week 1)

• Archive OLD.* prompts to .github/prompts/archive/
• Move draft prompts to .github/prompts/drafts/
• Converge md2html into single parameterized prompt
• Update README.md with accurate inventory

Phase 2: High-Impact Improvements (Weeks 2-3)

• Enhance reviewDockerCompose.prompt.md (Rank 1)
• Expand ansible-tutor.prompt.md (Rank 2)
• Fix session-status.prompt.md drift logic (Rank 3)
• Complete doc-lint.prompt.md (Rank 5)

Phase 3: Service Prompt Convergence (Week 4)

• Create service-workflow.meta.prompt.md
• Refactor 6 service prompts to use meta-prompt
• Test all workflows with real use cases

Phase 4: Draft Completion (Weeks 5-6)

• Complete service-decommission.prompt.md
• Complete service-migration.prompt.md
• Complete security-hardening.prompt.md
• Complete performance-tuning.prompt.md

---

6. Metrics & Success Criteria

Baseline (Current State)

• Total Prompts: 26
• Production-Ready: 8 (31%)
• Code Duplication: ~60% across service prompts
• Deprecated Content: 3 prompts

Target State (Post-Implementation)

• Total Prompts: 15-17 (-35%)
• Production-Ready: 15 (88%)
• Code Duplication: <20%
• Deprecated Content: 0 (archived)

Quality Gates

• ✅ All production prompts have gate structure
• ✅ All prompts have YAML frontmatter
• ✅ All prompts reference methodology (ReAct, CoT, etc.)
• ✅ All prompts include validation steps
• ✅ All prompts have rollback procedures

---

7. Recommendations Summary

Critical Actions

1. Converge service prompts → Single meta-prompt pattern (saves ~800 lines of duplicate code)
2. Fix reviewDockerCompose.prompt.md → Add gates and integrate with vulnerability scanning
3. Expand ansible-tutor.prompt.md → Add examples, safety checks, and validation

High Priority

4. Archive deprecated prompts → Clean up OLD.* files
5. Complete doc-lint.prompt.md → Finish truncated output section
6. Standardize session-status.prompt.md → Quantify drift detection

Medium Priority

7. Converge md2html prompts → Single parameterized version
8. Complete draft prompts → Follow service-new.prompt.md pattern

Low Priority

9. Update README.md → Reflect actual prompt inventory
10. Add testing framework → Validate prompts before deployment

---

8. Conclusion

The prompt repository has strong foundational patterns (gated workflows, RAG integration, safety guardrails) but suffers from:

• Duplication: 60% code overlap in service management prompts
• Inconsistency: 3 quality tiers with 9 incomplete drafts
• Maintenance Burden: 26 prompts to update when patterns evolve

Recommended Strategy: Phased convergence using meta-prompt architecture, starting with service management workflows (highest ROI). This reduces maintenance burden while preserving flexibility for specialized workflows.

Estimated Effort:

• Phase 1 (Cleanup): 2-4 hours
• Phase 2 (High-Impact): 8-12 hours
• Phase 3 (Convergence): 16-20 hours
• Phase 4 (Draft Completion): 12-16 hours
• Total: 38-52 hours over 6 weeks

---

Report Generated: 2026-01-09
Methodology: Static analysis + pattern detection + quality scoring
Scope: 26 prompt files in .github/prompts/
Next Review: 2026-02-09 (post-Phase 2 completion)