nathan eded49e711 feat(documentation): add centralized prompt repository plan with automated distribution

2026-04-12 00:42:51 -04:00

10 KiB

Raw Blame History

Centralized Prompt Repository with Automated Distribution

Goal

Create a centralized "llm-prompts" repository that automatically distributes prompt files, instructions, and knowledge bases to multiple consumer repositories whenever changes are pushed.

Architecture Overview

graph LR
    A[llm-prompts repo<br/>Central Source] -->|Push| B[Gitea Webhook]
    B -->|Trigger| C[Sync Script]
    C -->|Git Pull & Copy| D[homelab repo]
    C -->|Git Pull & Copy| E[repo-2]
    C -->|Git Pull & Copy| F[repo-N]
    
    style A fill:#ff6b6b,color:#fff
    style C fill:#326ce5,color:#fff

Option 1: Gitea Webhook + Sync Script (Recommended)

Best for: Your current stack (Gitea, automation-friendly)

Step 1: Create Central Repository

# On Gitea: Create new repo "llm-prompts"
# URL: https://git.castaldifamily.com/nathan/llm-prompts

# Initialize locally
mkdir ~/llm-prompts
cd ~/llm-prompts

# Move your existing prompts
cp -r ~/homelab/.github/prompts ./
cp -r ~/homelab/.github/instructions ./
cp -r ~/homelab/.github/knowledge ./

git init
git add .
git commit -m "feat: initialize centralized prompt library"
git remote add origin https://git.castaldifamily.com/nathan/llm-prompts.git
git push -u origin main

Step 2: Create Sync Script

Create /mnt/appdata/scripts/sync-prompts.sh on Heimdall:

#!/bin/bash
# Sync Script: Distribute prompts from central repo to consumers
set -euo pipefail

CENTRAL_REPO="/mnt/appdata/git-repos/llm-prompts"
TARGET_REPOS=(
  "/mnt/appdata/git-repos/homelab"
  "/mnt/appdata/git-repos/another-project"
  # Add more repos here
)

# Pull latest from central repo
cd "$CENTRAL_REPO"
git pull origin main

# Sync to each target repo
for REPO in "${TARGET_REPOS[@]}"; do
  echo "Syncing to $REPO..."
  
  # Create .github directory if it doesn't exist
  mkdir -p "$REPO/.github"
  
  # Copy directories (overwrite)
  rsync -av --delete "$CENTRAL_REPO/prompts/" "$REPO/.github/prompts/"
  rsync -av --delete "$CENTRAL_REPO/instructions/" "$REPO/.github/instructions/"
  rsync -av --delete "$CENTRAL_REPO/knowledge/" "$REPO/.github/knowledge/"
  
  # Auto-commit in target repo
  cd "$REPO"
  if [[ -n $(git status --porcelain) ]]; then
    git add .github/
    git commit -m "chore: sync prompts from llm-prompts@$(cd $CENTRAL_REPO && git rev-parse --short HEAD)"
    git push origin main
  fi
done

echo "✅ Sync complete"

Step 3: Configure Gitea Webhook

In Gitea UI → llm-prompts repo → Settings → Webhooks
Add Webhook:
- Target URL: http://script-runner:8080/sync-prompts (see Option A or B below)
- Content Type: application/json
- Trigger: Push events only
- Active: ✅

Step 4: Choose Webhook Receiver

Option A: Simple HTTP Server (Quick)

# On Heimdall, install webhook listener
mkdir -p /mnt/appdata/webhook-listener
cat > /mnt/appdata/webhook-listener/docker-compose.yaml <<EOF
services:
  webhook:
    image: adnanh/webhook:latest
    container_name: webhook-listener
    restart: unless-stopped
    ports:
      - "9000:9000"
    volumes:
      - ./hooks.json:/etc/webhook/hooks.json:ro
      - /mnt/appdata/scripts:/scripts:ro
      - /mnt/appdata/git-repos:/repos:rw
    command: -verbose -hooks=/etc/webhook/hooks.json
EOF

# Create hooks configuration
cat > /mnt/appdata/webhook-listener/hooks.json <<EOF
[
  {
    "id": "sync-prompts",
    "execute-command": "/scripts/sync-prompts.sh",
    "command-working-directory": "/repos",
    "response-message": "Syncing prompts...",
    "trigger-rule": {
      "match": {
        "type": "payload-hash-sha256",
        "secret": "YOUR_WEBHOOK_SECRET_HERE",
        "parameter": {
          "source": "header",
          "name": "X-Gitea-Signature"
        }
      }
    }
  }
]
EOF

docker compose up -d

Option B: Komodo Stack (Integrated)

Add to your Komodo deployment if you prefer centralized management.

Option 2: Git Submodules (Traditional)

Best for: Simple, standard Git workflow

# In each target repo
cd ~/homelab
git submodule add https://git.castaldifamily.com/nathan/llm-prompts.git .github/prompts-lib

# Create symlinks
ln -s .github/prompts-lib/prompts .github/prompts
ln -s .github/prompts-lib/instructions .github/instructions

# Update submodules in all repos
git submodule update --remote

Pros:

✅ Native Git feature
✅ Version pinning per repo
✅ No automation needed

Cons:

⚠️ Manual update required (git submodule update)
⚠️ Complexity with symlinks

Option 3: NFS Shared Directory (Simplest)

Best for: Instant sync across all repos

# Move prompts to NFS share
mkdir /mnt/appdata/shared-prompts
mv ~/homelab/.github/prompts/* /mnt/appdata/shared-prompts/

# In each repo, create symlink
cd ~/homelab
rm -rf .github/prompts
ln -s /mnt/appdata/shared-prompts .github/prompts

Pros:

✅ Zero latency (instant sync)
✅ No automation needed
✅ Single source of truth

Cons:

⚠️ Not versioned per-repo
⚠️ All repos must access same NFS

Recommended Approach for Your Setup

Use Option 1 (Webhook + Script) because:

✅ You already have Gitea webhooks working
✅ Fits your GitOps workflow (everything versioned)
✅ Gives you audit trail (commit per sync)
✅ Can add validation/testing before sync
✅ Scales to non-.github files (configs, etc.)

Enhanced Version: With Validation

Improve the sync script to validate prompts before pushing:

#!/bin/bash
# sync-prompts.sh (Enhanced with validation)
set -euo pipefail

CENTRAL_REPO="/mnt/appdata/git-repos/llm-prompts"

# Pull latest
cd "$CENTRAL_REPO"
git pull origin main

# Validate structure
echo "🔍 Validating prompt structure..."
if [[ ! -d "prompts" ]] || [[ ! -d "instructions" ]]; then
  echo "❌ ERROR: Missing required directories"
  exit 1
fi

# Check for required files
REQUIRED_FILES=("instructions/core.instructions.md")
for FILE in "${REQUIRED_FILES[@]}"; do
  if [[ ! -f "$FILE" ]]; then
    echo "❌ ERROR: Missing $FILE"
    exit 1
  fi
done

echo "✅ Validation passed"

# Sync to targets (same as before)
# ... rest of script ...

Migration Strategy

Week 1: Setup Central Repository

Create llm-prompts repo on Gitea
Copy existing prompts/instructions/knowledge from homelab
Test structure and validate files
Document repository layout

Week 2: Test Automation (Single Target)

Deploy webhook listener container
Create sync script in /mnt/appdata/scripts/
Configure Gitea webhook
Test with homelab repo only
Monitor logs and verify commits

Week 3: Scale to Multiple Repos

Add additional target repositories to sync script
Test with 2-3 repos
Verify no conflicts or issues
Document any edge cases

Week 4: Production Rollout

Archive old .github directories in target repos
Add monitoring/alerting for sync failures
Create troubleshooting guide
Update documentation with new workflow

Security Considerations

Webhook Authentication

Generate strong webhook secret: openssl rand -hex 32
Store in Gitea webhook configuration
Match in hooks.json file
Never commit secrets to Git

Script Permissions

Sync script should run as dedicated user (not root)
Git repos should have appropriate ownership
Use SSH keys for Git authentication (not passwords)
Validate all inputs before executing commands

Access Control

Central repo: Restricted write access (you only)
Target repos: Can be read-only or read-write depending on workflow
Webhook listener: Internal network only (not exposed to internet)

Monitoring & Troubleshooting

Log Files

# Webhook listener logs
docker logs webhook-listener

# Sync script execution
# Add to script: exec > >(tee -a /var/log/prompt-sync.log)

# Check last sync status
cd /mnt/appdata/git-repos/homelab
git log --oneline | grep "sync prompts"

Common Issues

Issue: Sync script fails with "permission denied"

Check file ownership: ls -la /mnt/appdata/scripts/sync-prompts.sh
Verify execute permission: chmod +x /mnt/appdata/scripts/sync-prompts.sh
Check Git authentication: ssh -T git@git.castaldifamily.com

Issue: Webhook not triggering

Verify webhook URL is accessible from Gitea
Check Gitea webhook delivery logs
Test webhook manually: curl -X POST http://webhook-listener:9000/hooks/sync-prompts

Issue: Merge conflicts in target repos

Never manually edit .github/prompts/ in target repos
If conflict occurs, revert local changes: git checkout HEAD -- .github/prompts/
Re-run sync manually

Alternative: GitHub Actions Style Workflow

If you want more advanced features (testing, validation, notifications):

# In llm-prompts repo: .gitea/workflows/sync.yaml
name: Sync Prompts to Consumer Repos

on:
  push:
    branches:
      - main

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Validate prompt structure
        run: |
          # Add validation logic
          
  sync:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - name: Sync to homelab
        run: |
          # Clone target repo
          # Copy files
          # Push changes

Note: Gitea Actions support varies by version. Verify your Gitea version supports Actions before implementing.

Next Steps

Decision Point: Choose Option 1, 2, or 3 based on your priorities
Create Infrastructure: Set up webhook listener or NFS share
Test in Isolation: Start with single target repo
Document Process: Add to SOPs when stable
Scale Gradually: Add repos incrementally

Gitea Webhooks: https://docs.gitea.io/en-us/webhooks/
Webhook Tool: https://github.com/adnanh/webhook
Git Submodules: https://git-scm.com/book/en/v2/Git-Tools-Submodules

Future Enhancements

Selective Sync: Only sync specific directories based on target repo config
Version Pinning: Allow target repos to specify prompt version
Rollback Mechanism: Automatic rollback if validation fails in target
Notification System: Slack/Discord notifications on sync completion
Dry-Run Mode: Preview changes before applying
Conflict Resolution: Automatic strategies for handling local modifications

10 KiB Raw Blame History