homelab/documentation/plans/plan-promptDistribution.md

# Centralized Prompt Repository with Automated Distribution

## Goal

Create a centralized "llm-prompts" repository that automatically distributes prompt files, instructions, and knowledge bases to multiple consumer repositories whenever changes are pushed.

## Architecture Overview

```mermaid
graph LR
    A[llm-prompts repo<br/>Central Source] -->|Push| B[Gitea Webhook]
    B -->|Trigger| C[Sync Script]
    C -->|Git Pull & Copy| D[homelab repo]
    C -->|Git Pull & Copy| E[repo-2]
    C -->|Git Pull & Copy| F[repo-N]
    
    style A fill:#ff6b6b,color:#fff
    style C fill:#326ce5,color:#fff
```

---

## Option 1: Gitea Webhook + Sync Script (Recommended)

**Best for:** Your current stack (Gitea, automation-friendly)

### Step 1: Create Central Repository

```bash
# On Gitea: Create new repo "llm-prompts"
# URL: https://git.castaldifamily.com/nathan/llm-prompts

# Initialize locally
mkdir ~/llm-prompts
cd ~/llm-prompts

# Move your existing prompts
cp -r ~/homelab/.github/prompts ./
cp -r ~/homelab/.github/instructions ./
cp -r ~/homelab/.github/knowledge ./

git init
git add .
git commit -m "feat: initialize centralized prompt library"
git remote add origin https://git.castaldifamily.com/nathan/llm-prompts.git
git push -u origin main
```

### Step 2: Create Sync Script

Create `/mnt/appdata/scripts/sync-prompts.sh` on Heimdall:

```bash
#!/bin/bash
# Sync Script: Distribute prompts from central repo to consumers
set -euo pipefail

CENTRAL_REPO="/mnt/appdata/git-repos/llm-prompts"
TARGET_REPOS=(
  "/mnt/appdata/git-repos/homelab"
  "/mnt/appdata/git-repos/another-project"
  # Add more repos here
)

# Pull latest from central repo
cd "$CENTRAL_REPO"
git pull origin main

# Sync to each target repo
for REPO in "${TARGET_REPOS[@]}"; do
  echo "Syncing to $REPO..."
  
  # Create .github directory if it doesn't exist
  mkdir -p "$REPO/.github"
  
  # Copy directories (overwrite)
  rsync -av --delete "$CENTRAL_REPO/prompts/" "$REPO/.github/prompts/"
  rsync -av --delete "$CENTRAL_REPO/instructions/" "$REPO/.github/instructions/"
  rsync -av --delete "$CENTRAL_REPO/knowledge/" "$REPO/.github/knowledge/"
  
  # Auto-commit in target repo
  cd "$REPO"
  if [[ -n $(git status --porcelain) ]]; then
    git add .github/
    git commit -m "chore: sync prompts from llm-prompts@$(cd $CENTRAL_REPO && git rev-parse --short HEAD)"
    git push origin main
  fi
done

echo "✅ Sync complete"
```

### Step 3: Configure Gitea Webhook

1. **In Gitea UI** → llm-prompts repo → Settings → Webhooks
2. **Add Webhook:**
   - Target URL: `http://script-runner:8080/sync-prompts` (see Option A or B below)
   - Content Type: `application/json`
   - Trigger: `Push events` only
   - Active: ✅

### Step 4: Choose Webhook Receiver

**Option A: Simple HTTP Server (Quick)**

```bash
# On Heimdall, install webhook listener
mkdir -p /mnt/appdata/webhook-listener
cat > /mnt/appdata/webhook-listener/docker-compose.yaml <<EOF
services:
  webhook:
    image: adnanh/webhook:latest
    container_name: webhook-listener
    restart: unless-stopped
    ports:
      - "9000:9000"
    volumes:
      - ./hooks.json:/etc/webhook/hooks.json:ro
      - /mnt/appdata/scripts:/scripts:ro
      - /mnt/appdata/git-repos:/repos:rw
    command: -verbose -hooks=/etc/webhook/hooks.json
EOF

# Create hooks configuration
cat > /mnt/appdata/webhook-listener/hooks.json <<EOF
[
  {
    "id": "sync-prompts",
    "execute-command": "/scripts/sync-prompts.sh",
    "command-working-directory": "/repos",
    "response-message": "Syncing prompts...",
    "trigger-rule": {
      "match": {
        "type": "payload-hash-sha256",
        "secret": "YOUR_WEBHOOK_SECRET_HERE",
        "parameter": {
          "source": "header",
          "name": "X-Gitea-Signature"
        }
      }
    }
  }
]
EOF

docker compose up -d
```

**Option B: Komodo Stack (Integrated)**

Add to your Komodo deployment if you prefer centralized management.

---

## Option 2: Git Submodules (Traditional)

**Best for:** Simple, standard Git workflow

```bash
# In each target repo
cd ~/homelab
git submodule add https://git.castaldifamily.com/nathan/llm-prompts.git .github/prompts-lib

# Create symlinks
ln -s .github/prompts-lib/prompts .github/prompts
ln -s .github/prompts-lib/instructions .github/instructions

# Update submodules in all repos
git submodule update --remote
```

**Pros:**
- ✅ Native Git feature
- ✅ Version pinning per repo
- ✅ No automation needed

**Cons:**
- ⚠️ Manual update required (`git submodule update`)
- ⚠️ Complexity with symlinks

---

## Option 3: NFS Shared Directory (Simplest)

**Best for:** Instant sync across all repos

```bash
# Move prompts to NFS share
mkdir /mnt/appdata/shared-prompts
mv ~/homelab/.github/prompts/* /mnt/appdata/shared-prompts/

# In each repo, create symlink
cd ~/homelab
rm -rf .github/prompts
ln -s /mnt/appdata/shared-prompts .github/prompts
```

**Pros:**
- ✅ Zero latency (instant sync)
- ✅ No automation needed
- ✅ Single source of truth

**Cons:**
- ⚠️ Not versioned per-repo
- ⚠️ All repos must access same NFS

---

## Recommended Approach for Your Setup

**Use Option 1 (Webhook + Script)** because:

1. ✅ You already have Gitea webhooks working
2. ✅ Fits your GitOps workflow (everything versioned)
3. ✅ Gives you audit trail (commit per sync)
4. ✅ Can add validation/testing before sync
5. ✅ Scales to non-.github files (configs, etc.)

---

## Enhanced Version: With Validation

Improve the sync script to validate prompts before pushing:

```bash
#!/bin/bash
# sync-prompts.sh (Enhanced with validation)
set -euo pipefail

CENTRAL_REPO="/mnt/appdata/git-repos/llm-prompts"

# Pull latest
cd "$CENTRAL_REPO"
git pull origin main

# Validate structure
echo "🔍 Validating prompt structure..."
if [[ ! -d "prompts" ]] || [[ ! -d "instructions" ]]; then
  echo "❌ ERROR: Missing required directories"
  exit 1
fi

# Check for required files
REQUIRED_FILES=("instructions/core.instructions.md")
for FILE in "${REQUIRED_FILES[@]}"; do
  if [[ ! -f "$FILE" ]]; then
    echo "❌ ERROR: Missing $FILE"
    exit 1
  fi
done

echo "✅ Validation passed"

# Sync to targets (same as before)
# ... rest of script ...
```

---

## Migration Strategy

### Week 1: Setup Central Repository
- Create `llm-prompts` repo on Gitea
- Copy existing prompts/instructions/knowledge from homelab
- Test structure and validate files
- Document repository layout

### Week 2: Test Automation (Single Target)
- Deploy webhook listener container
- Create sync script in `/mnt/appdata/scripts/`
- Configure Gitea webhook
- Test with homelab repo only
- Monitor logs and verify commits

### Week 3: Scale to Multiple Repos
- Add additional target repositories to sync script
- Test with 2-3 repos
- Verify no conflicts or issues
- Document any edge cases

### Week 4: Production Rollout
- Archive old `.github` directories in target repos
- Add monitoring/alerting for sync failures
- Create troubleshooting guide
- Update documentation with new workflow

---

## Security Considerations

### Webhook Authentication
- Generate strong webhook secret: `openssl rand -hex 32`
- Store in Gitea webhook configuration
- Match in `hooks.json` file
- Never commit secrets to Git

### Script Permissions
- Sync script should run as dedicated user (not root)
- Git repos should have appropriate ownership
- Use SSH keys for Git authentication (not passwords)
- Validate all inputs before executing commands

### Access Control
- Central repo: Restricted write access (you only)
- Target repos: Can be read-only or read-write depending on workflow
- Webhook listener: Internal network only (not exposed to internet)

---

## Monitoring & Troubleshooting

### Log Files
```bash
# Webhook listener logs
docker logs webhook-listener

# Sync script execution
# Add to script: exec > >(tee -a /var/log/prompt-sync.log)

# Check last sync status
cd /mnt/appdata/git-repos/homelab
git log --oneline | grep "sync prompts"
```

### Common Issues

**Issue: Sync script fails with "permission denied"**
- Check file ownership: `ls -la /mnt/appdata/scripts/sync-prompts.sh`
- Verify execute permission: `chmod +x /mnt/appdata/scripts/sync-prompts.sh`
- Check Git authentication: `ssh -T git@git.castaldifamily.com`

**Issue: Webhook not triggering**
- Verify webhook URL is accessible from Gitea
- Check Gitea webhook delivery logs
- Test webhook manually: `curl -X POST http://webhook-listener:9000/hooks/sync-prompts`

**Issue: Merge conflicts in target repos**
- Never manually edit `.github/prompts/` in target repos
- If conflict occurs, revert local changes: `git checkout HEAD -- .github/prompts/`
- Re-run sync manually

---

## Alternative: GitHub Actions Style Workflow

If you want more advanced features (testing, validation, notifications):

```yaml
# In llm-prompts repo: .gitea/workflows/sync.yaml
name: Sync Prompts to Consumer Repos

on:
  push:
    branches:
      - main

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Validate prompt structure
        run: |
          # Add validation logic
          
  sync:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - name: Sync to homelab
        run: |
          # Clone target repo
          # Copy files
          # Push changes
```

**Note:** Gitea Actions support varies by version. Verify your Gitea version supports Actions before implementing.

---

## Next Steps

1. **Decision Point:** Choose Option 1, 2, or 3 based on your priorities
2. **Create Infrastructure:** Set up webhook listener or NFS share
3. **Test in Isolation:** Start with single target repo
4. **Document Process:** Add to SOPs when stable
5. **Scale Gradually:** Add repos incrementally

---

## Related Documentation

- Gitea Webhooks: https://docs.gitea.io/en-us/webhooks/
- Webhook Tool: https://github.com/adnanh/webhook
- Git Submodules: https://git-scm.com/book/en/v2/Git-Tools-Submodules

---

## Future Enhancements

- **Selective Sync:** Only sync specific directories based on target repo config
- **Version Pinning:** Allow target repos to specify prompt version
- **Rollback Mechanism:** Automatic rollback if validation fails in target
- **Notification System:** Slack/Discord notifications on sync completion
- **Dry-Run Mode:** Preview changes before applying
- **Conflict Resolution:** Automatic strategies for handling local modifications