homelab/documentation/plans/plan-promptDistribution.md

Centralized Prompt Repository with Automated Distribution

Goal

Create a centralized "llm-prompts" repository that automatically distributes prompt files, instructions, and knowledge bases to multiple consumer repositories whenever changes are pushed.

Estimated Time to Complete (Option 1): 1.5-2.5 hours (initial setup) | 30-45 minutes (experienced operator)

---

Time Breakdown by Implementation Option

Option	Approach	Initial Setup Time	Ongoing Maintenance
Option 1	Gitea Webhook + Sync Script	1.5-2.5 hours	5-10 min/change
Option 2	Git Submodules	30-45 minutes	2-5 min/update
Option 3	NFS Shared Directory	15-20 minutes	0 min (instant)

Recommended: Option 1 (detailed time breakdown below)

Option 1: Detailed Phase Breakdown

Phase	Description	Time Estimate
Setup	Create Central Repository	15-20 minutes
Automation	Deploy Sync Script & Webhook	40-60 minutes
Testing	Validate Single Target	20-30 minutes
Scaling	Add Multiple Repositories	15-25 minutes
Documentation	Security & Troubleshooting	10-15 minutes
Total	End-to-End Implementation	1.5-2.5 hours

Architecture Overview

graph LR
    A[llm-prompts repo<br/>Central Source] -->|Push| B[Gitea Webhook]
    B -->|Trigger| C[Sync Script]
    C -->|Git Pull & Copy| D[homelab repo]
    C -->|Git Pull & Copy| E[repo-2]
    C -->|Git Pull & Copy| F[repo-N]
    
    style A fill:#ff6b6b,color:#fff
    style C fill:#326ce5,color:#fff

---

Option 1: Gitea Webhook + Sync Script (Recommended)

Total Time: 1.5-2.5 hours (initial setup)

Best for: Your current stack (Gitea, automation-friendly)

Step 1: Create Central Repository

Time: 15-20 minutes

# On Gitea: Create new repo "llm-prompts"
# URL: https://git.castaldifamily.com/nathan/llm-prompts

# Initialize locally
mkdir ~/llm-prompts
cd ~/llm-prompts

# Move your existing prompts
cp -r ~/homelab/.github/prompts ./
cp -r ~/homelab/.github/instructions ./
cp -r ~/homelab/.github/knowledge ./

git init
git add .
git commit -m "feat: initialize centralized prompt library"
git remote add origin https://git.castaldifamily.com/nathan/llm-prompts.git
git push -u origin main

Step 2: Create Sync Script

Time: 20-25 minutes

Create /mnt/appdata/scripts/sync-prompts.sh on Heimdall:

#!/bin/bash
# Sync Script: Distribute prompts from central repo to consumers
set -euo pipefail

CENTRAL_REPO="/mnt/appdata/git-repos/llm-prompts"
TARGET_REPOS=(
  "/mnt/appdata/git-repos/homelab"
  "/mnt/appdata/git-repos/another-project"
  # Add more repos here
)

# Pull latest from central repo
cd "$CENTRAL_REPO"
git pull origin main

# Sync to each target repo
for REPO in "${TARGET_REPOS[@]}"; do
  echo "Syncing to $REPO..."
  
  # Create .github directory if it doesn't exist
  mkdir -p "$REPO/.github"
  
  # Copy directories (overwrite)
  rsync -av --delete "$CENTRAL_REPO/prompts/" "$REPO/.github/prompts/"
  rsync -av --delete "$CENTRAL_REPO/instructions/" "$REPO/.github/instructions/"
  rsync -av --delete "$CENTRAL_REPO/knowledge/" "$REPO/.github/knowledge/"
  
  # Auto-commit in target repo
  cd "$REPO"
  if [[ -n $(git status --porcelain) ]]; then
    git add .github/
    git commit -m "chore: sync prompts from llm-prompts@$(cd $CENTRAL_REPO && git rev-parse --short HEAD)"
    git push origin main
  fi
done

echo "✅ Sync complete"

Step 3: Configure Gitea Webhook

Time: 5-8 minutes

1. In Gitea UI → llm-prompts repo → Settings → Webhooks
2. Add Webhook:
   • Target URL: http://script-runner:8080/sync-prompts (see Option A or B below)
   • Content Type: application/json
   • Trigger: Push events only
   • Active: ✅

Step 4: Choose Webhook Receiver

Time: 15-25 minutes (Option A) | 20-30 minutes (Option B)

Option A: Simple HTTP Server (Quick)

# On Heimdall, install webhook listener
mkdir -p /mnt/appdata/webhook-listener
cat > /mnt/appdata/webhook-listener/docker-compose.yaml <<EOF
services:
  webhook:
    image: adnanh/webhook:latest
    container_name: webhook-listener
    restart: unless-stopped
    ports:
      - "9000:9000"
    volumes:
      - ./hooks.json:/etc/webhook/hooks.json:ro
      - /mnt/appdata/scripts:/scripts:ro
      - /mnt/appdata/git-repos:/repos:rw
    command: -verbose -hooks=/etc/webhook/hooks.json
EOF

# Create hooks configuration
cat > /mnt/appdata/webhook-listener/hooks.json <<EOF
[
  {
    "id": "sync-prompts",
    "execute-command": "/scripts/sync-prompts.sh",
    "command-working-directory": "/repos",
    "response-message": "Syncing prompts...",
    "trigger-rule": {
      "match": {
        "type": "payload-hash-sha256",
        "secret": "YOUR_WEBHOOK_SECRET_HERE",
        "parameter": {
          "source": "header",
          "name": "X-Gitea-Signature"
        }
      }
    }
  }
]
EOF

docker compose up -d

Option B: Komodo Stack (Integrated)

Add to your Komodo deployment if you prefer centralized management.

---

Option 2: Git Submodules (Traditional)

Total Time: 30-45 minutes (initial setup) | 2-5 minutes per update

Best for: Simple, standard Git workflow

# In each target repo
cd ~/homelab
git submodule add https://git.castaldifamily.com/nathan/llm-prompts.git .github/prompts-lib

# Create symlinks
ln -s .github/prompts-lib/prompts .github/prompts
ln -s .github/prompts-lib/instructions .github/instructions

# Update submodules in all repos
git submodule update --remote

Pros:

• ✅ Native Git feature
• ✅ Version pinning per repo
• ✅ No automation needed

Cons:

• ⚠️ Manual update required (git submodule update)
• ⚠️ Complexity with symlinks

---

Option 3: NFS Shared Directory (Simplest)

Total Time: 15-20 minutes (one-time setup) | 0 minutes (instant sync)

Best for: Instant sync across all repos

# Move prompts to NFS share
mkdir /mnt/appdata/shared-prompts
mv ~/homelab/.github/prompts/* /mnt/appdata/shared-prompts/

# In each repo, create symlink
cd ~/homelab
rm -rf .github/prompts
ln -s /mnt/appdata/shared-prompts .github/prompts

Pros:

• ✅ Zero latency (instant sync)
• ✅ No automation needed
• ✅ Single source of truth

Cons:

• ⚠️ Not versioned per-repo
• ⚠️ All repos must access same NFS

---

Recommended Approach for Your Setup

Use Option 1 (Webhook + Script) because:

1. ✅ You already have Gitea webhooks working
2. ✅ Fits your GitOps workflow (everything versioned)
3. ✅ Gives you audit trail (commit per sync)
4. ✅ Can add validation/testing before sync
5. ✅ Scales to non-.github files (configs, etc.)

---

Enhanced Version: With Validation

Improve the sync script to validate prompts before pushing:

#!/bin/bash
# sync-prompts.sh (Enhanced with validation)
set -euo pipefail

CENTRAL_REPO="/mnt/appdata/git-repos/llm-prompts"

# Pull latest
cd "$CENTRAL_REPO"
git pull origin main

# Validate structure
echo "🔍 Validating prompt structure..."
if [[ ! -d "prompts" ]] || [[ ! -d "instructions" ]]; then
  echo "❌ ERROR: Missing required directories"
  exit 1
fi

# Check for required files
REQUIRED_FILES=("instructions/core.instructions.md")
for FILE in "${REQUIRED_FILES[@]}"; do
  if [[ ! -f "$FILE" ]]; then
    echo "❌ ERROR: Missing $FILE"
    exit 1
  fi
done

echo "✅ Validation passed"

# Sync to targets (same as before)
# ... rest of script ...

---

Migration Strategy

Total Timeline: 4 weeks (staged rollout) | 1 day (aggressive deployment)

Week 1: Setup Central Repository

Time: 2-3 hours

• Create llm-prompts repo on Gitea
• Copy existing prompts/instructions/knowledge from homelab
• Test structure and validate files
• Document repository layout

Week 2: Test Automation (Single Target)

Time: 1-2 hours + monitoring

• Deploy webhook listener container
• Create sync script in /mnt/appdata/scripts/
• Configure Gitea webhook
• Test with homelab repo only
• Monitor logs and verify commits

Week 3: Scale to Multiple Repos

Time: 30-60 minutes + validation

• Add additional target repositories to sync script
• Test with 2-3 repos
• Verify no conflicts or issues
• Document any edge cases

Week 4: Production Rollout

Time: 1-2 hours (documentation + monitoring setup)

• Archive old .github directories in target repos
• Add monitoring/alerting for sync failures
• Create troubleshooting guide
• Update documentation with new workflow

---

Security Considerations

Webhook Authentication

• Generate strong webhook secret: openssl rand -hex 32
• Store in Gitea webhook configuration
• Match in hooks.json file
• Never commit secrets to Git

Script Permissions

• Sync script should run as dedicated user (not root)
• Git repos should have appropriate ownership
• Use SSH keys for Git authentication (not passwords)
• Validate all inputs before executing commands

Access Control

• Central repo: Restricted write access (you only)
• Target repos: Can be read-only or read-write depending on workflow
• Webhook listener: Internal network only (not exposed to internet)

---

Monitoring & Troubleshooting

Log Files

# Webhook listener logs
docker logs webhook-listener

# Sync script execution
# Add to script: exec > >(tee -a /var/log/prompt-sync.log)

# Check last sync status
cd /mnt/appdata/git-repos/homelab
git log --oneline | grep "sync prompts"

Common Issues

Issue: Sync script fails with "permission denied"

• Check file ownership: ls -la /mnt/appdata/scripts/sync-prompts.sh
• Verify execute permission: chmod +x /mnt/appdata/scripts/sync-prompts.sh
• Check Git authentication: ssh -T git@git.castaldifamily.com

Issue: Webhook not triggering

• Verify webhook URL is accessible from Gitea
• Check Gitea webhook delivery logs
• Test webhook manually: curl -X POST http://webhook-listener:9000/hooks/sync-prompts

Issue: Merge conflicts in target repos

• Never manually edit .github/prompts/ in target repos
• If conflict occurs, revert local changes: git checkout HEAD -- .github/prompts/
• Re-run sync manually

---

Alternative: GitHub Actions Style Workflow

If you want more advanced features (testing, validation, notifications):

# In llm-prompts repo: .gitea/workflows/sync.yaml
name: Sync Prompts to Consumer Repos

on:
  push:
    branches:
      - main

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Validate prompt structure
        run: |
          # Add validation logic
          
  sync:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - name: Sync to homelab
        run: |
          # Clone target repo
          # Copy files
          # Push changes

Note: Gitea Actions support varies by version. Verify your Gitea version supports Actions before implementing.

---

Next Steps

1. Decision Point: Choose Option 1, 2, or 3 based on your priorities
2. Create Infrastructure: Set up webhook listener or NFS share
3. Test in Isolation: Start with single target repo
4. Document Process: Add to SOPs when stable
5. Scale Gradually: Add repos incrementally

---

Related Documentation

• Gitea Webhooks: https://docs.gitea.io/en-us/webhooks/
• Webhook Tool: https://github.com/adnanh/webhook
• Git Submodules: https://git-scm.com/book/en/v2/Git-Tools-Submodules

---

Future Enhancements

• Selective Sync: Only sync specific directories based on target repo config
• Version Pinning: Allow target repos to specify prompt version
• Rollback Mechanism: Automatic rollback if validation fails in target
• Notification System: Slack/Discord notifications on sync completion
• Dry-Run Mode: Preview changes before applying
• Conflict Resolution: Automatic strategies for handling local modifications