trevor/TG_autoposter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c849866fbd04a7eee86f8d9b23ce1118fd3bdb3e
TG_autoposter/docs/PRODUCTION_DEPLOYMENT.md
Andrew K. Choi a6817e487e init commit
2025-12-18 05:55:32 +09:00

537 lines
11 KiB
Markdown
Raw Blame History

# Production Deployment Guide
## Pre-Deployment Checklist
- [ ] Environment variables configured in `.env`
- [ ] PostgreSQL database created and migrated
- [ ] Redis running and accessible
- [ ] Telegram credentials verified
- [ ] SSL certificates prepared (if needed)
- [ ] Log rotation configured
- [ ] Monitoring and alerts set up
- [ ] Backups configured
- [ ] Health checks tested
## Deployment Methods
### 1. Docker Compose on VPS
#### 1.1 Prepare Server
```bash
# Update system
sudo apt update && sudo apt upgrade -y
# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
# Create non-root user for Docker
sudo usermod -aG docker $USER
newgrp docker
```
#### 1.2 Deploy Application
```bash
# Clone repository
mkdir -p /home/bot
cd /home/bot
git clone https://github.com/yourusername/TG_autoposter.git
cd TG_autoposter
# Create environment file
nano .env
# Fill in production values
# Start services
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
# Verify services
docker-compose ps
```
#### 1.3 Database Migrations
```bash
# Run migrations
docker-compose exec bot alembic upgrade head
# Verify
docker-compose exec bot alembic current
```
#### 1.4 Monitoring
```bash
# View logs
docker-compose logs -f
# Monitor specific service
docker-compose logs -f bot
docker-compose logs -f celery_worker_send
# Check health
docker-compose ps
```
### 2. Kubernetes Deployment
#### 2.1 Create Kubernetes Manifests
```bash
# Create namespace
kubectl create namespace telegram-bot
kubectl config set-context --current --namespace=telegram-bot
# Create ConfigMap for environment variables
kubectl create configmap bot-config --from-env-file=.env.prod
# Create Secrets for sensitive data
kubectl create secret generic bot-secrets \
--from-literal=telegram-bot-token=$TELEGRAM_BOT_TOKEN \
--from-literal=db-password=$DB_PASSWORD \
--from-literal=redis-password=$REDIS_PASSWORD
```
#### 2.2 Deploy Services
See `k8s/` directory for manifests:
- `postgres-deployment.yaml`
- `redis-deployment.yaml`
- `bot-deployment.yaml`
- `celery-worker-deployment.yaml`
- `celery-beat-deployment.yaml`
- `flower-deployment.yaml`
```bash
# Apply manifests
kubectl apply -f k8s/
# Monitor deployment
kubectl get pods
kubectl logs -f deployment/bot
```
### 3. Using Systemd Service
#### 3.1 Create Service File
```bash
sudo tee /etc/systemd/system/tg-autoposter.service > /dev/null <<EOF
[Unit]
Description=TG Autoposter Bot
After=network.target docker.service
Requires=docker.service
[Service]
Type=simple
User=bot
WorkingDirectory=/home/bot/TG_autoposter
ExecStart=/usr/local/bin/docker-compose up
ExecStop=/usr/local/bin/docker-compose down
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
```
#### 3.2 Enable Service
```bash
sudo systemctl daemon-reload
sudo systemctl enable tg-autoposter
sudo systemctl start tg-autoposter
# Check status
sudo systemctl status tg-autoposter
# View logs
sudo journalctl -u tg-autoposter -f
```
## Configuration for Production
### 1. Environment Variables
Create `.env` with production values:
```env
# Core
TELEGRAM_BOT_TOKEN=your_production_token
TELEGRAM_API_ID=your_api_id
TELEGRAM_API_HASH=your_api_hash
ADMIN_ID=your_admin_id
# Database (use managed service like RDS, CloudSQL)
DB_HOST=db.example.com
DB_PORT=5432
DB_USER=bot_user
DB_PASSWORD=secure_password_here
DB_NAME=tg_autoposter_prod
DB_SSL_MODE=require
# Redis (use managed service like ElastiCache, Redis Cloud)
REDIS_HOST=redis.example.com
REDIS_PORT=6379
REDIS_DB=0
REDIS_PASSWORD=secure_redis_password
# Logging
LOG_LEVEL=INFO
LOG_FILE=/var/log/tg-autoposter/bot.log
# Security
SECRET_KEY=your_secret_key_here
DEBUG=False
ALLOWED_HOSTS=yourdomain.com,www.yourdomain.com
```
### 2. Database Configuration
```yaml
# docker-compose.prod.yml
services:
postgres:
image: postgres:15-alpine
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_DB: ${DB_NAME}
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
interval: 10s
timeout: 5s
retries: 5
```
### 3. Redis Configuration
```yaml
# docker-compose.prod.yml
services:
redis:
image: redis:7-alpine
command: redis-server --requirepass ${REDIS_PASSWORD}
volumes:
- redis_data:/data
ports:
- "6379:6379"
healthcheck:
test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
interval: 10s
timeout: 5s
retries: 5
```
## Scaling Considerations
### 1. Horizontal Scaling
```yaml
# docker-compose.prod.yml - Multiple workers
services:
celery_worker_send_1:
# ... worker configuration
celery_worker_send_2:
# ... worker configuration
celery_worker_send_3:
# ... worker configuration
# Use load balancer or Docker Swarm for orchestration
```
### 2. Performance Tuning
```python
# app/settings.py
CELERY_WORKER_PREFETCH_MULTIPLIER = 1
CELERY_WORKER_MAX_TASKS_PER_CHILD = 1000
CELERY_TASK_TIME_LIMIT = 30 * 60 # 30 minutes
CELERY_TASK_SOFT_TIME_LIMIT = 25 * 60 # 25 minutes
# Database connection pooling
DB_POOL_SIZE = 20
DB_MAX_OVERFLOW = 0
DB_POOL_TIMEOUT = 30
DB_POOL_RECYCLE = 3600
```
### 3. Caching Strategy
```python
# Use Redis for caching
CACHE_TTL = 300 # 5 minutes
BROADCAST_CACHE_TTL = 3600 # 1 hour
MEMBERS_CACHE_TTL = 1800 # 30 minutes
```
## Monitoring & Logging
### 1. Logging Setup
```bash
# Create log directory
sudo mkdir -p /var/log/tg-autoposter
sudo chown bot:bot /var/log/tg-autoposter
# Configure log rotation
sudo tee /etc/logrotate.d/tg-autoposter > /dev/null <<EOF
/var/log/tg-autoposter/*.log {
daily
rotate 14
compress
delaycompress
notifempty
create 0640 bot bot
sharedscripts
postrotate
systemctl reload tg-autoposter > /dev/null 2>&1 || true
endscript
}
EOF
```
### 2. Prometheus Metrics
```python
# app/metrics.py
from prometheus_client import Counter, Histogram, Gauge
message_sent = Counter('messages_sent_total', 'Total messages sent')
message_failed = Counter('messages_failed_total', 'Total failed messages')
send_duration = Histogram('message_send_duration_seconds', 'Message send duration')
queue_size = Gauge('celery_queue_size', 'Celery queue size')
```
### 3. Monitoring with ELK Stack
```yaml
# docker-compose.prod.yml
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.0.0
kibana:
image: docker.elastic.co/kibana/kibana:8.0.0
ports:
- "5601:5601"
logstash:
image: docker.elastic.co/logstash/logstash:8.0.0
volumes:
- ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf
```
## Backup & Recovery
### 1. Database Backup
```bash
#!/bin/bash
# backup-db.sh
BACKUP_DIR="/backups/postgres"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="$BACKUP_DIR/tg_autoposter_$TIMESTAMP.sql"
mkdir -p $BACKUP_DIR
# Backup
docker-compose exec -T postgres pg_dump -U $DB_USER $DB_NAME > $BACKUP_FILE
# Compress
gzip $BACKUP_FILE
# Remove old backups (keep 7 days)
find $BACKUP_DIR -name "*.sql.gz" -mtime +7 -delete
echo "Backup completed: $BACKUP_FILE.gz"
```
### 2. Redis Snapshot
```bash
#!/bin/bash
# backup-redis.sh
BACKUP_DIR="/backups/redis"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR
# Create snapshot
docker-compose exec -T redis redis-cli BGSAVE
# Copy snapshot
docker-compose exec -T redis cp /data/dump.rdb /data/dump_$TIMESTAMP.rdb
echo "Redis backup completed"
```
### 3. Restore Database
```bash
# Drop and recreate database
docker-compose exec -T postgres dropdb -U $DB_USER $DB_NAME
docker-compose exec -T postgres createdb -U $DB_USER $DB_NAME
# Restore from backup
gunzip < /backups/postgres/tg_autoposter_*.sql.gz | \
docker-compose exec -T postgres psql -U $DB_USER $DB_NAME
```
## Security Best Practices
### 1. Environment Hardening
```bash
# Restrict file permissions
chmod 600 .env
chmod 700 /var/log/tg-autoposter
chmod 700 /backups
# Set ownership
sudo chown bot:bot /home/bot/TG_autoposter -R
```
### 2. Network Security
```yaml
# docker-compose.prod.yml
services:
bot:
networks:
- backend
expose:
- 8000
postgres:
networks:
- backend
expose:
- 5432
networks:
backend:
driver: bridge
driver_opts:
com.docker.network.bridge.name: br_backend
```
### 3. SSL/TLS
```bash
# Generate SSL certificate
certbot certonly --standalone -d yourdomain.com
# Configure in docker-compose.prod.yml
services:
nginx:
image: nginx:latest
volumes:
- /etc/letsencrypt:/etc/letsencrypt
- ./nginx.conf:/etc/nginx/nginx.conf
ports:
- "443:443"
```
## Troubleshooting Production Issues
### Issue: Memory Leaks
```bash
# Monitor memory usage
docker stats
# Restart worker
docker-compose restart celery_worker_send
# Check logs for errors
docker-compose logs celery_worker_send | grep -i error
```
### Issue: Database Connection Timeouts
```bash
# Increase pool size in settings
DB_POOL_SIZE = 30
# Check database status
docker-compose exec postgres psql -U bot -d tg_autoposter -c "SELECT datname, pid FROM pg_stat_activity;"
# Restart database
docker-compose restart postgres
```
### Issue: High CPU Usage
```bash
# Identify problematic tasks
docker-compose exec flower curl -s http://localhost:5555/api/stats | python -m json.tool
# Reduce worker concurrency
CELERY_WORKER_CONCURRENCY = 2
```
## CI/CD Integration
### GitHub Actions Example
```yaml
# .github/workflows/deploy.yml
name: Deploy to Production
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Deploy to VPS
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.VPS_HOST }}
username: ${{ secrets.VPS_USER }}
key: ${{ secrets.VPS_SSH_KEY }}
script: |
cd /home/bot/TG_autoposter
git pull origin main
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
docker-compose exec bot alembic upgrade head
```
## Support and Monitoring Links
- **Flower Dashboard**: http://yourserver.com:5555
- **PostgreSQL Monitoring**: pgAdmin (if enabled)
- **Application Logs**: `/var/log/tg-autoposter/`
- **Health Check Endpoint**: `/health` (if implemented)
## Maintenance Schedule
- **Daily**: Check logs for errors
- **Weekly**: Review resource usage
- **Monthly**: Security updates, dependency updates
- **Quarterly**: Performance analysis, capacity planning
## Contact & Support
For issues or questions:
1. Check logs and error messages
2. Review GitHub issues
3. Contact team lead
4. Escalate to DevOps team if needed
Reference in New Issue View Git Blame Copy Permalink