feat(readme): update infrastructure description and enhance automation details
This commit is contained in:
parent
481d206749
commit
e087670ca5
190
README.md
190
README.md
@ -1,8 +1,9 @@
|
||||
# Castaldi Family Homelab
|
||||
|
||||
> **A GitOps-managed, self-hosted infrastructure running media services, container orchestration, and automation across distributed ARM and x86 nodes.**
|
||||
> **A GitOps-managed, Ansible-automated infrastructure running media services, container orchestration, and hypervisor management across distributed ARM and x86 nodes.**
|
||||
|
||||
[](https://komo.do)
|
||||
[](#automation)
|
||||
[](#architecture)
|
||||
[](documentation/)
|
||||
|
||||
@ -11,11 +12,13 @@
|
||||
## 🚀 Why This Homelab?
|
||||
|
||||
- **Zero-Touch Deployments:** Push to Git → Auto-deploy via webhooks → Containers update automatically
|
||||
- **Infrastructure as Code:** All services defined in version-controlled `compose.yaml` files
|
||||
- **GPU Transcoding:** Hardware-accelerated media streaming with NVIDIA GTX 1060
|
||||
- **Distributed Architecture:** Services intelligently distributed across VM, physical server, and Raspberry Pi
|
||||
- **Ansible Automation:** All nodes managed by Ansible from watchtower control plane
|
||||
- **Infrastructure as Code:** Services defined in `compose.yaml` + infrastructure managed with Ansible playbooks
|
||||
- **GPU Transcoding:** Hardware-accelerated media streaming with NVIDIA GTX 1060 Mobile
|
||||
- **Distributed Architecture:** Services across Proxmox hypervisor, VMs, physical servers, and Raspberry Pi
|
||||
- **Self-Hosted Git:** No external dependencies—Gitea runs on-premise with automated backups
|
||||
- **Production-Grade Networking:** Traefik reverse proxy with automatic SSL (Cloudflare DNS challenge)
|
||||
- **Hypervisor Management:** Proxmox VE for VM orchestration with automated post-install configuration
|
||||
|
||||
---
|
||||
|
||||
@ -27,25 +30,32 @@ graph TB
|
||||
CF[Cloudflare DNS]
|
||||
end
|
||||
|
||||
subgraph "Heimdall (Proxmox VM - 10.0.0.151)"
|
||||
subgraph "PVE01 - Proxmox VE Hypervisor (10.0.0.201)"
|
||||
subgraph "Heimdall VM (10.0.0.151)"
|
||||
Traefik[Traefik Reverse Proxy<br/>:80, :443]
|
||||
Komodo[Komodo Core<br/>Container Orchestrator]
|
||||
Gitea[Gitea<br/>Self-Hosted Git]
|
||||
Redis[Redis Cache]
|
||||
Trek[Trek]
|
||||
Vault[Vaultwarden]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph "Waldorf (Physical Server - 10.0.0.251)"
|
||||
subgraph "Waldorf - Physical Server (10.0.0.251)"
|
||||
Plex[Plex Media Server<br/>GPU Transcoding]
|
||||
Tunarr[Tunarr<br/>IPTV Channels]
|
||||
GPU[NVIDIA GTX 1060]
|
||||
GPU[NVIDIA GTX 1060 Mobile<br/>6GB VRAM]
|
||||
KomodoW[Komodo Periphery]
|
||||
end
|
||||
|
||||
subgraph "Watchtower (Raspberry Pi 5 - 10.0.0.200)"
|
||||
Periphery[Komodo Periphery<br/>Remote Agent]
|
||||
subgraph "Watchtower - Raspberry Pi 5 (10.0.0.200)"
|
||||
Ansible[Ansible Control Node<br/>Infrastructure Automation]
|
||||
KomodoP[Komodo Periphery]
|
||||
VSCode[VS Code Server]
|
||||
end
|
||||
|
||||
subgraph "TerraMaster NAS (10.0.0.250)"
|
||||
NFS[NFS Storage<br/>/Volume1/appdata]
|
||||
NFS[NFS Storage<br/>Volume1: /appdata<br/>Volume2: /media]
|
||||
end
|
||||
|
||||
CF -->|HTTPS| Traefik
|
||||
@ -54,32 +64,36 @@ graph TB
|
||||
Traefik --> Plex
|
||||
Traefik --> Tunarr
|
||||
|
||||
Komodo <-->|WebSocket| Periphery
|
||||
Komodo <-->|WebSocket| KomodoW
|
||||
Komodo <-->|WebSocket| KomodoP
|
||||
Gitea -->|Webhook| Komodo
|
||||
|
||||
Ansible -.->|SSH| PVE01
|
||||
Ansible -.->|SSH| Heimdall
|
||||
Ansible -.->|SSH| Waldorf
|
||||
|
||||
Plex --> GPU
|
||||
Tunarr --> GPU
|
||||
|
||||
Heimdall -.->|NFSv3| NFS
|
||||
Waldorf -.->|NFSv3| NFS
|
||||
Watchtower -.->|NFSv3| NFS
|
||||
Heimdall -.->|NFS v4| NFS
|
||||
Waldorf -.->|NFS v4| NFS
|
||||
Watchtower -.->|NFS v3| NFS
|
||||
|
||||
style Traefik fill:#326ce5,color:#fff
|
||||
style Komodo fill:#ff6b6b,color:#fff
|
||||
style GPU fill:#76b900,color:#fff
|
||||
style NFS fill:#f9a825,color:#000
|
||||
style PVE01 fill:#e57000,color:#fff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Infrastructure Inventory
|
||||
|
||||
| Node | IP | Hardware | Role | Services |
|
||||
|------|------|----------|------|----------|
|
||||
| **Heimdall** | `10.0.0.151` | Proxmox VM<br/>Intel N100, 16GB RAM | Core Services | Komodo, Gitea, Traefik, Redis |
|
||||
| **Waldorf** | `10.0.0.251` | Physical Server<br/>i7-7820HQ, GTX 1060, 16GB | Media Processing | Plex, Tunarr (GPU transcoding) |
|
||||
| **Watchtower** | `10.0.0.200` | Raspberry Pi 5<br/>ARM Cortex-A76, 16GB | Periphery Node | Komodo Agent |
|
||||
| **TerraMaster** | `10.0.0.250` | NAS | Shared Storage | NFSv3 (`/Volume1/appdata`) |
|
||||
| Node | IP | Hardware | Platform/OS | Role | Services |
|
||||
|------|------|----------|----------|------|----------|
|
||||
| **PVE01** | `10.0.0.201` | Physical Server<br/>Intel i5-13500T (14c), 15GB RAM | Proxmox VE 9.1.7 | Hypervisor | Hosts Heimdall VM |
|
||||
| **Heimdall** | `10.0.0.151` | Proxmox VM on PVE01<br/>Intel N100 (4c), 15GB RAM | Ubuntu 24.04 | Core Services | Komodo Core, Gitea, Traefik, Redis, Trek, Vaultwarden |
|
||||
| **Waldorf** | `10.0.0.251` | Physical Server<br/>i7-7820HQ (8c), GTX 1060, 16GB | Ubuntu 24.04 | Media Processing | Plex, Tunarr (GPU transcoding), Komodo Periphery |
|
||||
| **Watchtower** | `10.0.0.200` | Raspberry Pi 5<br/>ARM Cortex-A76 (4c), 16GB | Debian Trixie | Control Plane | Ansible, Komodo Periphery, VS Code Server |
|
||||
| **TerraMaster** | `10.0.0.250` | NAS | TOS | Shared Storage | NFS (Volume1: `/appdata`, Volume2: `/media`) |
|
||||
|
||||
---
|
||||
|
||||
@ -109,10 +123,49 @@ docker compose up -d
|
||||
|
||||
### Automated GitOps Workflow
|
||||
|
||||
1. **Edit** `nodes/{node}/{service}/compose.yaml`
|
||||
2. **Commit** and push to `main` branch
|
||||
3. **Webhook** triggers Komodo pull
|
||||
4. **Auto-deploy** updates running containers
|
||||
1. **Edit** `nodes/{node}/{service}/compose.yaml` locally
|
||||
2. **Commit** and push to Gitea: `git add . && git commit -m "feat: update service" && git push`
|
||||
3. **Webhook** triggers Komodo Core (heimdall)
|
||||
4. **Auto-deploy** pulls latest code and restarts containers
|
||||
5. **Monitor** via Komodo UI at `http://10.0.0.151:9000`
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Automation
|
||||
|
||||
### Ansible Control Plane
|
||||
|
||||
**Watchtower** (10.0.0.200) manages all infrastructure via Ansible:
|
||||
|
||||
```bash
|
||||
# SSH into control node
|
||||
ssh chester@10.0.0.200
|
||||
cd ~/homelab/ansible
|
||||
|
||||
# Test connectivity to all nodes
|
||||
ansible all -m ping
|
||||
|
||||
# Gather live system facts
|
||||
ansible-playbook playbooks/gather-node-facts.yml
|
||||
|
||||
# Deploy Proxmox post-install config
|
||||
ansible-playbook playbooks/onboard-proxmox.yml --limit pve01
|
||||
|
||||
# Run commands across node groups
|
||||
ansible docker_nodes -m command -a "docker ps"
|
||||
ansible proxmox_cluster -m command -a "pveversion"
|
||||
```
|
||||
|
||||
### Managed Node Groups
|
||||
|
||||
```yaml
|
||||
control_plane: watchtower
|
||||
docker_nodes: heimdall, waldorf
|
||||
proxmox_cluster: pve01
|
||||
nfs_clients: heimdall, waldorf
|
||||
core_services: heimdall
|
||||
media_services: waldorf
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@ -122,13 +175,13 @@ docker compose up -d
|
||||
|
||||
| Status | Mission | Details |
|
||||
|--------|---------|---------|
|
||||
| 🟢 | **GitOps Migration** | All production stacks migrated to Git-based deployment |
|
||||
| 🟢 | **Webhook Automation** | Gitea webhooks trigger auto-deploy on push |
|
||||
| 🟢 | **GPU Passthrough** | NVIDIA GTX 1060 accessible in Plex/Tunarr containers |
|
||||
| 🟢 | **Komodo GitOps** | All stacks migrated to Git sources with webhook automation |
|
||||
| 🟢 | **GPU Transcoding** | GTX 1060 Mobile accessible in Plex/Tunarr containers |
|
||||
| 🟢 | **Documentation Structure** | KBAs and SOPs organized in `documentation/` |
|
||||
| 🟢 | **Ansible Automation** | All 4 nodes onboarded and managed by Ansible from Watchtower |
|
||||
| 🟢 | **Proxmox Post-Install** | PVE01 configured: subscription nag removed, repos optimized |
|
||||
| 🟡 | **Hardware Transcoding Validation** | Monitor Plex for `(hw)` indicator during active streams |
|
||||
| 🟢 | **NFS Mount Stability** | NFSv3 forced on Raspberry Pi to prevent ID-domain errors |
|
||||
| 🟢 | **Credential Security** | Secrets managed via Komodo Environment Variables (not Git) |
|
||||
| 🟢 | **NFS Mount Stability** | NFSv3 on Pi, NFSv4 on x86 nodes |
|
||||
|
||||
---
|
||||
|
||||
@ -136,20 +189,36 @@ docker compose up -d
|
||||
|
||||
```
|
||||
homelab/
|
||||
├── ansible/ # Ansible automation (active)
|
||||
│ ├── inventory/ # Managed hosts and groups
|
||||
│ │ ├── hosts.ini # 4-node inventory
|
||||
│ │ └── host_vars/ # Per-node configuration
|
||||
│ ├── playbooks/ # Automation workflows
|
||||
│ │ ├── onboard-nodes.yml # Node SSH key deployment
|
||||
│ │ ├── onboard-proxmox.yml # Proxmox post-install
|
||||
│ │ └── gather-node-facts.yml # System discovery
|
||||
│ ├── roles/ # Reusable automation
|
||||
│ │ └── proxmox_post_install/ # Nag removal, repo config
|
||||
│ └── group_vars/ # Global variables
|
||||
├── nodes/ # Service definitions per node
|
||||
│ ├── heimdall/ # Core infrastructure (VM)
|
||||
│ ├── heimdall/ # Core infrastructure (VM on PVE01)
|
||||
│ │ ├── core/ # Komodo, Traefik, Redis
|
||||
│ │ └── gitea/ # Self-hosted Git
|
||||
│ │ ├── trek/ # Trek service
|
||||
│ │ ├── vaultwarden/ # Password manager
|
||||
│ │ └── (gitea via Komodo) # Self-hosted Git
|
||||
│ ├── waldorf/ # Media services (Physical)
|
||||
│ │ ├── plex/ # Media server + GPU
|
||||
│ │ └── tunarr/ # IPTV channels + GPU
|
||||
│ └── watchtower/ # Periphery agent (Pi 5)
|
||||
│ └── watchtower/ # Control plane (Pi 5)
|
||||
│ └── vscode/ # Remote development
|
||||
├── documentation/ # Technical knowledge base
|
||||
│ ├── KBAs/ # Troubleshooting guides
|
||||
│ ├── SOPs/ # Operational procedures
|
||||
│ ├── plans/ # Implementation roadmaps
|
||||
│ └── TECHNICAL_RUNBOOK.md # Emergency reference
|
||||
├── ansible/ # (Future) Automated provisioning
|
||||
└── scripts/ # Utility scripts
|
||||
├── bootstrap.sh # Day-0 node initialization
|
||||
└── lib/ # Shared function libraries
|
||||
```
|
||||
|
||||
---
|
||||
@ -231,10 +300,11 @@ Example:
|
||||
```yaml
|
||||
# In Git (compose.yaml)
|
||||
environment:
|
||||
- PLEX_CLAIM=${PLEX_CLAIM} # Placeholder
|
||||
- PUID=1000
|
||||
- PGID=1000
|
||||
- API_KEY=${PLEX_API_KEY} # Injected by Komodo
|
||||
|
||||
# In Komodo UI → Stack → Environment Variables
|
||||
PLEX_CLAIM=claim-xxxxxxxxx
|
||||
# In Komodo UI: Set PLEX_API_KEY in Environment Variables
|
||||
```
|
||||
|
||||
### NFS Mount Configuration
|
||||
@ -242,10 +312,34 @@ PLEX_CLAIM=claim-xxxxxxxxx
|
||||
**Critical:** Raspberry Pi requires NFSv3 (not v4) due to ID-domain mismatches:
|
||||
|
||||
```bash
|
||||
# /etc/fstab on Watchtower
|
||||
10.0.0.250:/Volume1/appdata /mnt/appdata nfs rw,nfsvers=3,hard,intr,x-systemd.automount,nolock 0 0
|
||||
# /etc/fstab on Watchtower (Pi 5)
|
||||
10.0.0.250:/Volume1/appdata /mnt/appdata nfs nfsvers=3,rw,sync 0 0
|
||||
|
||||
# /etc/fstab on Heimdall/Waldorf (x86 Ubuntu)
|
||||
10.0.0.250:/Volume1/appdata /mnt/appdata nfs4 rw,sync 0 0
|
||||
```
|
||||
|
||||
### Backup Strategy
|
||||
|
||||
- **Git Repository:** Daily backups via Gitea's built-in backup feature
|
||||
- **Docker Volumes:** Weekly snapshots to `/mnt/appdata/backups/`
|
||||
- **Proxmox VMs:** Daily snapshots with 7-day retention
|
||||
- **Configuration Files:** Tracked in Git under `nodes/{hostname}/`
|
||||
|
||||
---
|
||||
|
||||
## 📊 Stats
|
||||
|
||||
- **Total Nodes:** 5 (1 hypervisor + 3 compute + 1 storage)
|
||||
- **Automation:** Ansible managing 4 active nodes from Watchtower
|
||||
- **Container Orchestration:** Komodo v2.1.2
|
||||
- **Active Services:** 12+ (Traefik, Plex, Tunarr, Gitea, Trek, Vaultwarden, etc.)
|
||||
- **Total RAM:** 62GB (15GB PVE01 + 15GB Heimdall + 16GB Waldorf + 16GB Watchtower)
|
||||
- **Total CPU Cores:** 30 physical (14c i5-13500T + 8c i7-7820HQ + 4c N100 + 4c ARM)
|
||||
- **Virtualization:** Proxmox VE 9.1.7 hosting 1 VM (expandable)
|
||||
- **GPU Acceleration:** NVIDIA GTX 1060 Mobile (6GB VRAM)
|
||||
- **Storage:** TerraMaster NAS (NFSv3/v4)
|
||||
|
||||
---
|
||||
|
||||
## 🔥 Emergency Procedures
|
||||
@ -297,17 +391,6 @@ This is a personal homelab, but documentation improvements and issue reports are
|
||||
|
||||
---
|
||||
|
||||
## 📊 Stats
|
||||
|
||||
- **Total Nodes:** 4 (3 compute + 1 storage)
|
||||
- **Container Orchestration:** Komodo v2.1.2
|
||||
- **Active Services:** 8+ (Traefik, Plex, Tunarr, Gitea, etc.)
|
||||
- **Total RAM:** 48GB (across compute nodes)
|
||||
- **GPU Acceleration:** NVIDIA GTX 1060 Mobile (6GB)
|
||||
- **Storage:** TerraMaster NAS + local node storage
|
||||
|
||||
---
|
||||
|
||||
## 📜 License
|
||||
|
||||
Personal infrastructure configuration. Documentation licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/).
|
||||
@ -315,5 +398,6 @@ Personal infrastructure configuration. Documentation licensed under [CC BY-SA 4.
|
||||
---
|
||||
|
||||
**Maintained by:** Nathan Castaldi
|
||||
**Last Updated:** April 12, 2026
|
||||
**Last Updated:** April 13, 2026
|
||||
**Status:** 🟢 Operational
|
||||
**Automation Status:** 🟢 Ansible Fully Deployed
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user