feat: Complete Docker deployment environment for PyGuardian v2.1.0
Some checks reported errors
continuous-integration/drone/push Build encountered an error
Some checks reported errors
continuous-integration/drone/push Build encountered an error
🐳 DOCKER DEPLOYMENT INFRASTRUCTURE: ## New Docker Files: - deployment/docker/Dockerfile.optimized - Multi-stage optimized builds - docker-compose.prod.yml - Production cluster deployment - docker-compose.dev.yml - Development environment - deploy-docker.sh - One-command deployment script - Makefile.docker - Advanced management commands - .env.docker - Environment configuration template - DOCKER_DEPLOYMENT.md - Complete deployment guide ## Container Images: - pyguardian:controller - Cluster management (200MB) - pyguardian:agent - Security monitoring (180MB) - pyguardian:standalone - All-in-one deployment (220MB) - pyguardian:development - Dev tools + Jupyter (350MB) ## Deployment Modes: - Standalone: Single container with all features - Cluster: Controller + scalable agents with JWT auth - Production: Enterprise deployment with monitoring - Development: Hot reload + debugging tools ## Key Features: ✅ Multi-stage Docker builds for optimization ✅ Privileged containers for system monitoring ✅ Host networking for firewall integration ✅ Volume persistence for data/logs/config ✅ Health checks and auto-restart ✅ Prometheus monitoring integration ✅ SSL/TLS support with custom certificates ✅ Automated backup and restore ✅ CI/CD ready builds ## Quick Commands: ./deploy-docker.sh standalone # Quick start ./deploy-docker.sh cluster --scale 3 # Production cluster make -f Makefile.docker prod-up # Advanced management make -f Makefile.docker health # Health checks Ready for enterprise Docker deployment! 🚀
This commit is contained in:
550
.history/DOCKER_DEPLOYMENT_20251126042542.md
Normal file
550
.history/DOCKER_DEPLOYMENT_20251126042542.md
Normal file
@@ -0,0 +1,550 @@
|
||||
# 🐳 PyGuardian Docker Deployment Guide
|
||||
|
||||
Complete containerized deployment solution for PyGuardian v2.1.0 enterprise security system.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### One-Command Deployment
|
||||
|
||||
```bash
|
||||
# Standalone deployment (recommended for single server)
|
||||
./deploy-docker.sh standalone
|
||||
|
||||
# Production cluster with 3 agents
|
||||
./deploy-docker.sh cluster --scale 3 --monitoring
|
||||
|
||||
# Development environment
|
||||
./deploy-docker.sh development
|
||||
```
|
||||
|
||||
### Using Makefile (Advanced)
|
||||
|
||||
```bash
|
||||
# Setup environment and start production
|
||||
make -f Makefile.docker setup-env
|
||||
make -f Makefile.docker prod-up
|
||||
|
||||
# Development environment
|
||||
make -f Makefile.docker dev-up
|
||||
|
||||
# Check status
|
||||
make -f Makefile.docker status
|
||||
```
|
||||
|
||||
## 📋 Prerequisites
|
||||
|
||||
### System Requirements
|
||||
|
||||
- **Docker**: 20.10+
|
||||
- **Docker Compose**: 2.0+
|
||||
- **Memory**: 2GB+ RAM
|
||||
- **Disk**: 10GB+ available space
|
||||
- **OS**: Linux (Ubuntu 20.04+, CentOS 8+, etc.)
|
||||
|
||||
### Install Docker
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
curl -fsSL https://get.docker.com -o get-docker.sh
|
||||
sudo sh get-docker.sh
|
||||
sudo usermod -aG docker $USER
|
||||
|
||||
# Enable and start
|
||||
sudo systemctl enable docker
|
||||
sudo systemctl start docker
|
||||
```
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
### Container Images
|
||||
|
||||
| Image | Purpose | Size | Target |
|
||||
|-------|---------|------|--------|
|
||||
| `pyguardian:controller` | Cluster controller | ~200MB | Production |
|
||||
| `pyguardian:agent` | Security agent | ~180MB | Production |
|
||||
| `pyguardian:standalone` | All-in-one | ~220MB | Single server |
|
||||
| `pyguardian:development` | Dev tools | ~350MB | Development |
|
||||
|
||||
### Network Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Host Network │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ │
|
||||
│ │ Controller │ │ Agent 1 │ │ Agent 2 │ │
|
||||
│ │ Port: 8443 │ │ (monitoring) │ │ (monitoring) │ │
|
||||
│ │ │◄─┤ │◄─┤ │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └──────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Copy and customize the environment file:
|
||||
|
||||
```bash
|
||||
cp .env.docker .env
|
||||
nano .env
|
||||
```
|
||||
|
||||
#### Essential Variables
|
||||
|
||||
```bash
|
||||
# Telegram integration
|
||||
TELEGRAM_BOT_TOKEN=1234567890:ABCDEFGHIJKLMNOPQRSTUVWXYZ
|
||||
|
||||
# Security secrets (generate with: openssl rand -hex 32)
|
||||
CLUSTER_SECRET=your_32_byte_hex_secret
|
||||
JWT_SECRET=your_32_byte_jwt_secret
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=INFO # DEBUG, INFO, WARNING, ERROR
|
||||
```
|
||||
|
||||
#### Advanced Configuration
|
||||
|
||||
```bash
|
||||
# Performance tuning
|
||||
CONTAINER_MEMORY_LIMIT=512m
|
||||
CONTAINER_CPU_LIMIT=1.0
|
||||
|
||||
# Monitoring
|
||||
PROMETHEUS_ENABLED=true
|
||||
HEALTH_CHECK_INTERVAL=30
|
||||
|
||||
# Security
|
||||
FIREWALL_ENABLED=true
|
||||
IDS_ENABLED=true
|
||||
```
|
||||
|
||||
## 🏭 Deployment Modes
|
||||
|
||||
### 1. Standalone Mode
|
||||
|
||||
**Best for**: Single server deployments, testing, small environments
|
||||
|
||||
```bash
|
||||
# Quick start
|
||||
./deploy-docker.sh standalone
|
||||
|
||||
# With custom config
|
||||
./deploy-docker.sh standalone --env .env.custom
|
||||
|
||||
# Manual Docker command
|
||||
docker run -d \
|
||||
--name pyguardian-standalone \
|
||||
--privileged \
|
||||
--network host \
|
||||
--restart unless-stopped \
|
||||
--env-file .env \
|
||||
-v /opt/pyguardian/data:/opt/pyguardian/data \
|
||||
-v /var/log:/var/log:ro \
|
||||
pyguardian:standalone
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- ✅ Complete security monitoring
|
||||
- ✅ Telegram notifications
|
||||
- ✅ Web API (port 8443)
|
||||
- ✅ Firewall management
|
||||
- ✅ Intrusion detection
|
||||
|
||||
### 2. Cluster Mode
|
||||
|
||||
**Best for**: Multi-server environments, high availability
|
||||
|
||||
```bash
|
||||
# Controller + 2 agents
|
||||
./deploy-docker.sh cluster --scale 2
|
||||
|
||||
# With monitoring stack
|
||||
./deploy-docker.sh cluster --scale 3 --monitoring
|
||||
|
||||
# Using docker-compose directly
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
**Architecture**:
|
||||
```
|
||||
Controller (Server 1) ←── Agent (Server 2)
|
||||
←── Agent (Server 3)
|
||||
←── Agent (Server N)
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- ✅ Centralized management
|
||||
- ✅ JWT-based authentication
|
||||
- ✅ Real-time agent communication
|
||||
- ✅ Scalable to 100+ agents
|
||||
- ✅ Health monitoring
|
||||
|
||||
### 3. Production Mode
|
||||
|
||||
**Best for**: Enterprise deployments, 24/7 operations
|
||||
|
||||
```bash
|
||||
# Full production stack
|
||||
./deploy-docker.sh production --monitoring
|
||||
|
||||
# Manual with all features
|
||||
make -f Makefile.docker prod-up monitoring-up
|
||||
```
|
||||
|
||||
**Includes**:
|
||||
- 🔒 **Enhanced security**: SSL/TLS, secrets management
|
||||
- 📊 **Monitoring**: Prometheus, health checks
|
||||
- 💾 **Data persistence**: Volume management
|
||||
- 🔄 **Auto-restart**: unless-stopped policy
|
||||
- 📝 **Logging**: Structured logs, rotation
|
||||
|
||||
### 4. Development Mode
|
||||
|
||||
**Best for**: Development, testing, debugging
|
||||
|
||||
```bash
|
||||
# Development environment
|
||||
./deploy-docker.sh development
|
||||
|
||||
# Access development tools
|
||||
make -f Makefile.docker dev-shell
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- 🔧 **Hot reload**: Code changes reflected live
|
||||
- 🧪 **Testing tools**: pytest, coverage, linting
|
||||
- 📔 **Jupyter Lab**: http://localhost:8888
|
||||
- 🐛 **Debug mode**: Verbose logging
|
||||
- 🗄️ **Test database**: PostgreSQL + Redis
|
||||
|
||||
## 🔧 Management Commands
|
||||
|
||||
### Using deploy-docker.sh
|
||||
|
||||
```bash
|
||||
# Deployment
|
||||
./deploy-docker.sh standalone # Single container
|
||||
./deploy-docker.sh cluster --scale 3 # 3-agent cluster
|
||||
./deploy-docker.sh production # Production ready
|
||||
|
||||
# Build options
|
||||
./deploy-docker.sh standalone --build --no-cache
|
||||
|
||||
# Custom environment
|
||||
./deploy-docker.sh cluster --env .env.production
|
||||
```
|
||||
|
||||
### Using Makefile
|
||||
|
||||
```bash
|
||||
# Environment setup
|
||||
make -f Makefile.docker setup-env # Create .env file
|
||||
make -f Makefile.docker generate-secrets # Generate secure secrets
|
||||
|
||||
# Production operations
|
||||
make -f Makefile.docker prod-up # Start production
|
||||
make -f Makefile.docker prod-down # Stop production
|
||||
make -f Makefile.docker prod-restart # Restart production
|
||||
make -f Makefile.docker prod-logs # View logs
|
||||
|
||||
# Development operations
|
||||
make -f Makefile.docker dev-up # Start development
|
||||
make -f Makefile.docker dev-shell # Access container shell
|
||||
make -f Makefile.docker dev-logs # View dev logs
|
||||
|
||||
# Cluster management
|
||||
make -f Makefile.docker cluster-up # Start cluster
|
||||
make -f Makefile.docker cluster-scale AGENTS=5 # Scale to 5 agents
|
||||
make -f Makefile.docker cluster-status # Check cluster
|
||||
|
||||
# Maintenance
|
||||
make -f Makefile.docker backup # Create data backup
|
||||
make -f Makefile.docker clean # Clean containers
|
||||
make -f Makefile.docker health # Health check
|
||||
```
|
||||
|
||||
## 📊 Monitoring & Logs
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Container health
|
||||
docker ps --format "table {{.Names}}\t{{.Status}}"
|
||||
|
||||
# Application health
|
||||
curl -k https://localhost:8443/health
|
||||
|
||||
# Detailed status
|
||||
make -f Makefile.docker health
|
||||
```
|
||||
|
||||
### Log Management
|
||||
|
||||
```bash
|
||||
# Real-time logs
|
||||
docker logs -f pyguardian-controller
|
||||
docker logs -f pyguardian-agent-1
|
||||
|
||||
# Production logs
|
||||
make -f Makefile.docker prod-logs
|
||||
|
||||
# Development logs
|
||||
make -f Makefile.docker dev-logs
|
||||
|
||||
# Log analysis
|
||||
docker exec pyguardian-controller tail -f /opt/pyguardian/logs/pyguardian.log
|
||||
```
|
||||
|
||||
### Prometheus Monitoring
|
||||
|
||||
When monitoring is enabled:
|
||||
|
||||
```bash
|
||||
# Start with monitoring
|
||||
./deploy-docker.sh production --monitoring
|
||||
|
||||
# Access Prometheus
|
||||
open http://localhost:9090
|
||||
|
||||
# Key metrics
|
||||
- pyguardian_agents_connected
|
||||
- pyguardian_security_incidents
|
||||
- pyguardian_system_cpu_percent
|
||||
- pyguardian_system_memory_percent
|
||||
```
|
||||
|
||||
## 🗄️ Data Management
|
||||
|
||||
### Volume Structure
|
||||
|
||||
```
|
||||
/opt/pyguardian/
|
||||
├── controller/
|
||||
│ ├── data/ # SQLite database, auth keys
|
||||
│ ├── logs/ # Application logs
|
||||
│ └── config/ # Configuration files
|
||||
├── agent1/
|
||||
│ ├── data/ # Agent data, cache
|
||||
│ ├── logs/ # Agent logs
|
||||
│ └── config/ # Agent configuration
|
||||
└── backups/ # Automated backups
|
||||
```
|
||||
|
||||
### Backup & Restore
|
||||
|
||||
```bash
|
||||
# Create backup
|
||||
make -f Makefile.docker backup
|
||||
|
||||
# Restore from backup
|
||||
make -f Makefile.docker restore BACKUP=pyguardian_backup_20231125_143022.tar.gz
|
||||
|
||||
# Manual backup
|
||||
docker run --rm \
|
||||
-v pyguardian_controller_data:/source \
|
||||
-v $(pwd)/backups:/backup \
|
||||
alpine tar czf /backup/manual_backup.tar.gz -C /source .
|
||||
```
|
||||
|
||||
### Database Access
|
||||
|
||||
```bash
|
||||
# SQLite database access
|
||||
docker exec -it pyguardian-controller \
|
||||
sqlite3 /opt/pyguardian/data/pyguardian.db
|
||||
|
||||
# View agent registrations
|
||||
docker exec pyguardian-controller \
|
||||
python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/opt/pyguardian/data/pyguardian.db')
|
||||
cursor = conn.cursor()
|
||||
cursor.execute('SELECT * FROM agent_auth')
|
||||
print(cursor.fetchall())
|
||||
"
|
||||
```
|
||||
|
||||
## 🔐 Security
|
||||
|
||||
### SSL/TLS Configuration
|
||||
|
||||
```bash
|
||||
# Generate SSL certificates
|
||||
mkdir -p ssl
|
||||
openssl req -x509 -newkey rsa:4096 -keyout ssl/key.pem -out ssl/cert.pem -days 365 -nodes
|
||||
|
||||
# Update environment
|
||||
echo "SSL_ENABLED=true" >> .env
|
||||
echo "SSL_CERT_PATH=/opt/pyguardian/ssl/cert.pem" >> .env
|
||||
echo "SSL_KEY_PATH=/opt/pyguardian/ssl/key.pem" >> .env
|
||||
```
|
||||
|
||||
### Secrets Management
|
||||
|
||||
```bash
|
||||
# Generate secure secrets
|
||||
make -f Makefile.docker generate-secrets
|
||||
|
||||
# Docker secrets (for Swarm)
|
||||
echo "your_secret" | docker secret create cluster_secret -
|
||||
echo "your_jwt_secret" | docker secret create jwt_secret -
|
||||
```
|
||||
|
||||
### Firewall Integration
|
||||
|
||||
```bash
|
||||
# Container needs privileged mode for iptables
|
||||
--privileged
|
||||
|
||||
# Custom iptables rules
|
||||
docker exec pyguardian-controller \
|
||||
sudo iptables -A INPUT -p tcp --dport 22 -j DROP
|
||||
```
|
||||
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### 1. Permission Denied
|
||||
|
||||
```bash
|
||||
# Fix data directory permissions
|
||||
sudo chown -R $USER:$USER /opt/pyguardian
|
||||
chmod -R 755 /opt/pyguardian
|
||||
```
|
||||
|
||||
#### 2. Port Already in Use
|
||||
|
||||
```bash
|
||||
# Check what's using port 8443
|
||||
sudo lsof -i :8443
|
||||
sudo netstat -tulpn | grep 8443
|
||||
|
||||
# Kill conflicting process
|
||||
sudo kill -9 <PID>
|
||||
```
|
||||
|
||||
#### 3. Container Health Check Failed
|
||||
|
||||
```bash
|
||||
# Check container logs
|
||||
docker logs pyguardian-controller
|
||||
|
||||
# Manual health check
|
||||
docker exec pyguardian-controller python3 -c "import requests; print(requests.get('http://localhost:8443/health').text)"
|
||||
|
||||
# Restart unhealthy container
|
||||
docker restart pyguardian-controller
|
||||
```
|
||||
|
||||
#### 4. Agent Connection Issues
|
||||
|
||||
```bash
|
||||
# Check network connectivity
|
||||
docker exec pyguardian-agent-1 curl -k https://controller:8443/health
|
||||
|
||||
# Verify cluster secret
|
||||
docker exec pyguardian-controller env | grep CLUSTER_SECRET
|
||||
docker exec pyguardian-agent-1 env | grep CLUSTER_SECRET
|
||||
|
||||
# Check agent logs
|
||||
docker logs pyguardian-agent-1 | grep -i error
|
||||
```
|
||||
|
||||
### Debug Commands
|
||||
|
||||
```bash
|
||||
# Container resource usage
|
||||
docker stats
|
||||
|
||||
# Inspect container configuration
|
||||
docker inspect pyguardian-controller
|
||||
|
||||
# Network debugging
|
||||
docker exec pyguardian-controller ip addr show
|
||||
docker exec pyguardian-controller ss -tulpn
|
||||
|
||||
# System debugging inside container
|
||||
docker exec -it pyguardian-controller bash
|
||||
ps aux
|
||||
netstat -tulpn
|
||||
tail -f /opt/pyguardian/logs/pyguardian.log
|
||||
```
|
||||
|
||||
### Performance Tuning
|
||||
|
||||
```bash
|
||||
# Increase memory limit
|
||||
echo "CONTAINER_MEMORY_LIMIT=1g" >> .env
|
||||
|
||||
# Optimize for production
|
||||
echo "PYGUARDIAN_LOG_LEVEL=WARNING" >> .env
|
||||
echo "WORKER_PROCESSES=4" >> .env
|
||||
```
|
||||
|
||||
## 📚 Advanced Usage
|
||||
|
||||
### Multi-Host Cluster
|
||||
|
||||
For deploying across multiple servers:
|
||||
|
||||
```bash
|
||||
# Server 1 (Controller)
|
||||
./deploy-docker.sh production
|
||||
echo "CONTROLLER_HOST=$(hostname -I | awk '{print $1}')" >> .env
|
||||
|
||||
# Server 2+ (Agents)
|
||||
export CONTROLLER_HOST=<controller_ip>
|
||||
./deploy-docker.sh agent --env .env.agent
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
```bash
|
||||
# Build for CI
|
||||
docker build -f deployment/docker/Dockerfile.optimized --target controller .
|
||||
|
||||
# Test deployment
|
||||
make -f Makefile.docker test-build
|
||||
|
||||
# Automated deployment
|
||||
./deploy-docker.sh production --build --no-cache
|
||||
```
|
||||
|
||||
### Custom Images
|
||||
|
||||
```bash
|
||||
# Build custom controller
|
||||
docker build -f deployment/docker/Dockerfile.optimized \
|
||||
--target controller \
|
||||
--build-arg PYGUARDIAN_VERSION=2.1.0-custom \
|
||||
-t pyguardian:controller-custom .
|
||||
|
||||
# Use custom image
|
||||
sed -i 's/pyguardian:controller/pyguardian:controller-custom/g' docker-compose.prod.yml
|
||||
```
|
||||
|
||||
## 📞 Support
|
||||
|
||||
- **Documentation**: `/documentation/`
|
||||
- **Issues**: GitHub Issues
|
||||
- **Logs**: Check `/opt/pyguardian/*/logs/`
|
||||
- **Health**: `https://localhost:8443/health`
|
||||
|
||||
## 🎯 Quick Reference
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| **Quick Start** | `./deploy-docker.sh standalone` |
|
||||
| **Production** | `./deploy-docker.sh production --monitoring` |
|
||||
| **Development** | `./deploy-docker.sh development` |
|
||||
| **Scale Cluster** | `make cluster-scale AGENTS=5` |
|
||||
| **View Logs** | `make prod-logs` |
|
||||
| **Health Check** | `make health` |
|
||||
| **Backup** | `make backup` |
|
||||
| **Clean Up** | `make clean` |
|
||||
|
||||
---
|
||||
|
||||
🚀 **PyGuardian v2.1.0** - Enterprise Security Made Simple!
|
||||
Reference in New Issue
Block a user