nathan/homelab
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Releases Wiki Activity

feat(readme): update infrastructure description and enhance automation details

Browse Source
This commit is contained in:
Nathan 2026-04-13 21:01:57 -04:00
parent 481d206749
commit e087670ca5
1 changed files with 142 additions and 58 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

198
README.md
View File

@ -1,8 +1,9 @@
# Castaldi Family Homelab # Castaldi Family Homelab
> **A GitOps-managed, self-hosted infrastructure running media services, container orchestration, and automation across distributed ARM and x86 nodes.** > **A GitOps-managed, Ansible-automated infrastructure running media services, container orchestration, and hypervisor management across distributed ARM and x86 nodes.**
[![GitOps](https://img.shields.io/badge/GitOps-Komodo-blue?style=flat-square)](https://komo.do) [![GitOps](https://img.shields.io/badge/GitOps-Komodo-blue?style=flat-square)](https://komo.do)
[![Automation](https://img.shields.io/badge/Automation-Ansible-red?style=flat-square)](#automation)
[![Infrastructure](https://img.shields.io/badge/Infrastructure-Multi--Node-green?style=flat-square)](#architecture) [![Infrastructure](https://img.shields.io/badge/Infrastructure-Multi--Node-green?style=flat-square)](#architecture)
[![Documentation](https://img.shields.io/badge/Docs-KBAs%20%2B%20SOPs-orange?style=flat-square)](documentation/) [![Documentation](https://img.shields.io/badge/Docs-KBAs%20%2B%20SOPs-orange?style=flat-square)](documentation/)
@ -11,11 +12,13 @@
## 🚀 Why This Homelab? ## 🚀 Why This Homelab?
- **Zero-Touch Deployments:** Push to Git → Auto-deploy via webhooks → Containers update automatically - **Zero-Touch Deployments:** Push to Git → Auto-deploy via webhooks → Containers update automatically
- **Infrastructure as Code:** All services defined in version-controlled `compose.yaml` files - **Ansible Automation:** All nodes managed by Ansible from watchtower control plane
- **GPU Transcoding:** Hardware-accelerated media streaming with NVIDIA GTX 1060 - **Infrastructure as Code:** Services defined in `compose.yaml` + infrastructure managed with Ansible playbooks
- **Distributed Architecture:** Services intelligently distributed across VM, physical server, and Raspberry Pi - **GPU Transcoding:** Hardware-accelerated media streaming with NVIDIA GTX 1060 Mobile
- **Distributed Architecture:** Services across Proxmox hypervisor, VMs, physical servers, and Raspberry Pi
- **Self-Hosted Git:** No external dependencies—Gitea runs on-premise with automated backups - **Self-Hosted Git:** No external dependencies—Gitea runs on-premise with automated backups
- **Production-Grade Networking:** Traefik reverse proxy with automatic SSL (Cloudflare DNS challenge) - **Production-Grade Networking:** Traefik reverse proxy with automatic SSL (Cloudflare DNS challenge)
- **Hypervisor Management:** Proxmox VE for VM orchestration with automated post-install configuration
--- ---
@ -27,25 +30,32 @@ graph TB
CF[Cloudflare DNS] CF[Cloudflare DNS]
end end
subgraph "Heimdall (Proxmox VM - 10.0.0.151)" subgraph "PVE01 - Proxmox VE Hypervisor (10.0.0.201)"
Traefik[Traefik Reverse Proxy<br/>:80, :443] subgraph "Heimdall VM (10.0.0.151)"
Komodo[Komodo Core<br/>Container Orchestrator] Traefik[Traefik Reverse Proxy<br/>:80, :443]
Gitea[Gitea<br/>Self-Hosted Git] Komodo[Komodo Core<br/>Container Orchestrator]
Redis[Redis Cache] Gitea[Gitea<br/>Self-Hosted Git]
Redis[Redis Cache]
Trek[Trek]
Vault[Vaultwarden]
end
end end
subgraph "Waldorf (Physical Server - 10.0.0.251)" subgraph "Waldorf - Physical Server (10.0.0.251)"
Plex[Plex Media Server<br/>GPU Transcoding] Plex[Plex Media Server<br/>GPU Transcoding]
Tunarr[Tunarr<br/>IPTV Channels] Tunarr[Tunarr<br/>IPTV Channels]
GPU[NVIDIA GTX 1060] GPU[NVIDIA GTX 1060 Mobile<br/>6GB VRAM]
KomodoW[Komodo Periphery]
end end
subgraph "Watchtower (Raspberry Pi 5 - 10.0.0.200)" subgraph "Watchtower - Raspberry Pi 5 (10.0.0.200)"
Periphery[Komodo Periphery<br/>Remote Agent] Ansible[Ansible Control Node<br/>Infrastructure Automation]
KomodoP[Komodo Periphery]
VSCode[VS Code Server]
end end
subgraph "TerraMaster NAS (10.0.0.250)" subgraph "TerraMaster NAS (10.0.0.250)"
NFS[NFS Storage<br/>/Volume1/appdata] NFS[NFS Storage<br/>Volume1: /appdata<br/>Volume2: /media]
end end
CF -->|HTTPS| Traefik CF -->|HTTPS| Traefik
@ -54,32 +64,36 @@ graph TB
Traefik --> Plex Traefik --> Plex
Traefik --> Tunarr Traefik --> Tunarr
Komodo <-->|WebSocket| Periphery Komodo <-->|WebSocket| KomodoW
Komodo <-->|WebSocket| KomodoP
Gitea -->|Webhook| Komodo Gitea -->|Webhook| Komodo
Ansible -.->|SSH| PVE01
Ansible -.->|SSH| Heimdall
Ansible -.->|SSH| Waldorf
Plex --> GPU Plex --> GPU
Tunarr --> GPU Tunarr --> GPU
Heimdall -.->|NFSv3| NFS Heimdall -.->|NFS v4| NFS
Waldorf -.->|NFSv3| NFS Waldorf -.->|NFS v4| NFS
Watchtower -.->|NFSv3| NFS Watchtower -.->|NFS v3| NFS
style Traefik fill:#326ce5,color:#fff
style Komodo fill:#ff6b6b,color:#fff
style GPU fill:#76b900,color:#fff
style NFS fill:#f9a825,color:#000 style NFS fill:#f9a825,color:#000
style PVE01 fill:#e57000,color:#fff
``` ```
--- ---
## 📦 Infrastructure Inventory ## 📦 Infrastructure Inventory
| Node | IP | Hardware | Role | Services | | Node | IP | Hardware | Platform/OS | Role | Services |
|------|------|----------|------|----------| |------|------|----------|----------|------|----------|
| **Heimdall** | `10.0.0.151` | Proxmox VM<br/>Intel N100, 16GB RAM | Core Services | Komodo, Gitea, Traefik, Redis | | **PVE01** | `10.0.0.201` | Physical Server<br/>Intel i5-13500T (14c), 15GB RAM | Proxmox VE 9.1.7 | Hypervisor | Hosts Heimdall VM |
| **Waldorf** | `10.0.0.251` | Physical Server<br/>i7-7820HQ, GTX 1060, 16GB | Media Processing | Plex, Tunarr (GPU transcoding) | | **Heimdall** | `10.0.0.151` | Proxmox VM on PVE01<br/>Intel N100 (4c), 15GB RAM | Ubuntu 24.04 | Core Services | Komodo Core, Gitea, Traefik, Redis, Trek, Vaultwarden |
| **Watchtower** | `10.0.0.200` | Raspberry Pi 5<br/>ARM Cortex-A76, 16GB | Periphery Node | Komodo Agent | | **Waldorf** | `10.0.0.251` | Physical Server<br/>i7-7820HQ (8c), GTX 1060, 16GB | Ubuntu 24.04 | Media Processing | Plex, Tunarr (GPU transcoding), Komodo Periphery |
| **TerraMaster** | `10.0.0.250` | NAS | Shared Storage | NFSv3 (`/Volume1/appdata`) | | **Watchtower** | `10.0.0.200` | Raspberry Pi 5<br/>ARM Cortex-A76 (4c), 16GB | Debian Trixie | Control Plane | Ansible, Komodo Periphery, VS Code Server |
| **TerraMaster** | `10.0.0.250` | NAS | TOS | Shared Storage | NFS (Volume1: `/appdata`, Volume2: `/media`) |
--- ---
@ -109,10 +123,49 @@ docker compose up -d
### Automated GitOps Workflow ### Automated GitOps Workflow
1. **Edit** `nodes/{node}/{service}/compose.yaml` 1. **Edit** `nodes/{node}/{service}/compose.yaml` locally
2. **Commit** and push to `main` branch 2. **Commit** and push to Gitea: `git add . && git commit -m "feat: update service" && git push`
3. **Webhook** triggers Komodo pull 3. **Webhook** triggers Komodo Core (heimdall)
4. **Auto-deploy** updates running containers 4. **Auto-deploy** pulls latest code and restarts containers
5. **Monitor** via Komodo UI at `http://10.0.0.151:9000`
---
## ⚙️ Automation
### Ansible Control Plane
**Watchtower** (10.0.0.200) manages all infrastructure via Ansible:
```bash
# SSH into control node
ssh chester@10.0.0.200
cd ~/homelab/ansible
# Test connectivity to all nodes
ansible all -m ping
# Gather live system facts
ansible-playbook playbooks/gather-node-facts.yml
# Deploy Proxmox post-install config
ansible-playbook playbooks/onboard-proxmox.yml --limit pve01
# Run commands across node groups
ansible docker_nodes -m command -a "docker ps"
ansible proxmox_cluster -m command -a "pveversion"
```
### Managed Node Groups
```yaml
control_plane: watchtower
docker_nodes: heimdall, waldorf
proxmox_cluster: pve01
nfs_clients: heimdall, waldorf
core_services: heimdall
media_services: waldorf
```
--- ---
@ -122,13 +175,13 @@ docker compose up -d
| Status | Mission | Details | | Status | Mission | Details |
|--------|---------|---------| |--------|---------|---------|
| 🟢 | **GitOps Migration** | All production stacks migrated to Git-based deployment | | 🟢 | **Komodo GitOps** | All stacks migrated to Git sources with webhook automation |
| 🟢 | **Webhook Automation** | Gitea webhooks trigger auto-deploy on push | | 🟢 | **GPU Transcoding** | GTX 1060 Mobile accessible in Plex/Tunarr containers |
| 🟢 | **GPU Passthrough** | NVIDIA GTX 1060 accessible in Plex/Tunarr containers |
| 🟢 | **Documentation Structure** | KBAs and SOPs organized in `documentation/` | | 🟢 | **Documentation Structure** | KBAs and SOPs organized in `documentation/` |
| 🟢 | **Ansible Automation** | All 4 nodes onboarded and managed by Ansible from Watchtower |
| 🟢 | **Proxmox Post-Install** | PVE01 configured: subscription nag removed, repos optimized |
| 🟡 | **Hardware Transcoding Validation** | Monitor Plex for `(hw)` indicator during active streams | | 🟡 | **Hardware Transcoding Validation** | Monitor Plex for `(hw)` indicator during active streams |
| 🟢 | **NFS Mount Stability** | NFSv3 forced on Raspberry Pi to prevent ID-domain errors | | 🟢 | **NFS Mount Stability** | NFSv3 on Pi, NFSv4 on x86 nodes |
| 🟢 | **Credential Security** | Secrets managed via Komodo Environment Variables (not Git) |
--- ---
@ -136,20 +189,36 @@ docker compose up -d
``` ```
homelab/ homelab/
├── ansible/ # Ansible automation (active)
│ ├── inventory/ # Managed hosts and groups
│ │ ├── hosts.ini # 4-node inventory
│ │ └── host_vars/ # Per-node configuration
│ ├── playbooks/ # Automation workflows
│ │ ├── onboard-nodes.yml # Node SSH key deployment
│ │ ├── onboard-proxmox.yml # Proxmox post-install
│ │ └── gather-node-facts.yml # System discovery
│ ├── roles/ # Reusable automation
│ │ └── proxmox_post_install/ # Nag removal, repo config
│ └── group_vars/ # Global variables
├── nodes/ # Service definitions per node ├── nodes/ # Service definitions per node
│ ├── heimdall/ # Core infrastructure (VM) │ ├── heimdall/ # Core infrastructure (VM on PVE01)
│ │ ├── core/ # Komodo, Traefik, Redis │ │ ├── core/ # Komodo, Traefik, Redis
│ │ └── gitea/ # Self-hosted Git │ │ ├── trek/ # Trek service
│ │ ├── vaultwarden/ # Password manager
│ │ └── (gitea via Komodo) # Self-hosted Git
│ ├── waldorf/ # Media services (Physical) │ ├── waldorf/ # Media services (Physical)
│ │ ├── plex/ # Media server + GPU │ │ ├── plex/ # Media server + GPU
│ │ └── tunarr/ # IPTV channels + GPU │ │ └── tunarr/ # IPTV channels + GPU
│ └── watchtower/ # Periphery agent (Pi 5) │ └── watchtower/ # Control plane (Pi 5)
│ └── vscode/ # Remote development
├── documentation/ # Technical knowledge base ├── documentation/ # Technical knowledge base
│ ├── KBAs/ # Troubleshooting guides │ ├── KBAs/ # Troubleshooting guides
│ ├── SOPs/ # Operational procedures │ ├── SOPs/ # Operational procedures
│ ├── plans/ # Implementation roadmaps
│ └── TECHNICAL_RUNBOOK.md # Emergency reference │ └── TECHNICAL_RUNBOOK.md # Emergency reference
├── ansible/ # (Future) Automated provisioning
└── scripts/ # Utility scripts └── scripts/ # Utility scripts
├── bootstrap.sh # Day-0 node initialization
└── lib/ # Shared function libraries
``` ```
--- ---
@ -231,10 +300,11 @@ Example:
```yaml ```yaml
# In Git (compose.yaml) # In Git (compose.yaml)
environment: environment:
- PLEX_CLAIM=${PLEX_CLAIM} # Placeholder - PUID=1000
- PGID=1000
- API_KEY=${PLEX_API_KEY} # Injected by Komodo
# In Komodo UI → Stack → Environment Variables # In Komodo UI: Set PLEX_API_KEY in Environment Variables
PLEX_CLAIM=claim-xxxxxxxxx
``` ```
### NFS Mount Configuration ### NFS Mount Configuration
@ -242,10 +312,34 @@ PLEX_CLAIM=claim-xxxxxxxxx
**Critical:** Raspberry Pi requires NFSv3 (not v4) due to ID-domain mismatches: **Critical:** Raspberry Pi requires NFSv3 (not v4) due to ID-domain mismatches:
```bash ```bash
# /etc/fstab on Watchtower # /etc/fstab on Watchtower (Pi 5)
10.0.0.250:/Volume1/appdata /mnt/appdata nfs rw,nfsvers=3,hard,intr,x-systemd.automount,nolock 0 0 10.0.0.250:/Volume1/appdata /mnt/appdata nfs nfsvers=3,rw,sync 0 0
# /etc/fstab on Heimdall/Waldorf (x86 Ubuntu)
10.0.0.250:/Volume1/appdata /mnt/appdata nfs4 rw,sync 0 0
``` ```
### Backup Strategy
- **Git Repository:** Daily backups via Gitea's built-in backup feature
- **Docker Volumes:** Weekly snapshots to `/mnt/appdata/backups/`
- **Proxmox VMs:** Daily snapshots with 7-day retention
- **Configuration Files:** Tracked in Git under `nodes/{hostname}/`
---
## 📊 Stats
- **Total Nodes:** 5 (1 hypervisor + 3 compute + 1 storage)
- **Automation:** Ansible managing 4 active nodes from Watchtower
- **Container Orchestration:** Komodo v2.1.2
- **Active Services:** 12+ (Traefik, Plex, Tunarr, Gitea, Trek, Vaultwarden, etc.)
- **Total RAM:** 62GB (15GB PVE01 + 15GB Heimdall + 16GB Waldorf + 16GB Watchtower)
- **Total CPU Cores:** 30 physical (14c i5-13500T + 8c i7-7820HQ + 4c N100 + 4c ARM)
- **Virtualization:** Proxmox VE 9.1.7 hosting 1 VM (expandable)
- **GPU Acceleration:** NVIDIA GTX 1060 Mobile (6GB VRAM)
- **Storage:** TerraMaster NAS (NFSv3/v4)
--- ---
## 🔥 Emergency Procedures ## 🔥 Emergency Procedures
@ -297,17 +391,6 @@ This is a personal homelab, but documentation improvements and issue reports are
--- ---
## 📊 Stats
- **Total Nodes:** 4 (3 compute + 1 storage)
- **Container Orchestration:** Komodo v2.1.2
- **Active Services:** 8+ (Traefik, Plex, Tunarr, Gitea, etc.)
- **Total RAM:** 48GB (across compute nodes)
- **GPU Acceleration:** NVIDIA GTX 1060 Mobile (6GB)
- **Storage:** TerraMaster NAS + local node storage
---
## 📜 License ## 📜 License
Personal infrastructure configuration. Documentation licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/). Personal infrastructure configuration. Documentation licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/).
@ -315,5 +398,6 @@ Personal infrastructure configuration. Documentation licensed under [CC BY-SA 4.
--- ---
**Maintained by:** Nathan Castaldi **Maintained by:** Nathan Castaldi
**Last Updated:** April 12, 2026 **Last Updated:** April 13, 2026
**Status:** 🟢 Operational **Status:** 🟢 Operational
**Automation Status:** 🟢 Ansible Fully Deployed