Add complete infrastructure automation and documentation
Some checks failed
Deploy to Production / Deploy to Server (push) Has been cancelled

- infrastructure/setup.sh: Master provisioning script for fresh Ubuntu servers
- infrastructure/docker-compose.yml: Production Docker Compose configuration
- infrastructure/.env.template: Environment variables template
- infrastructure/nginx/: Nginx configuration with security headers
- infrastructure/postfix/: Postfix + OpenDKIM email configuration
- infrastructure/README.md: Complete disaster recovery guide
- scripts/docfast-backup.sh: SQLite backup script with rotation

All services now fully reproducible with documented disaster recovery procedures.
This commit is contained in:
openclawd 2026-02-15 11:04:34 +00:00
parent d99eea517c
commit 3820d7ea4d
9 changed files with 766 additions and 0 deletions

View file

@ -0,0 +1,27 @@
# DocFast Environment Variables Template
# Copy this to .env and fill in real values
# Stripe Configuration (Production keys)
STRIPE_SECRET_KEY=sk_live_FILL_IN_YOUR_STRIPE_SECRET_KEY
STRIPE_WEBHOOK_SECRET=whsec_FILL_IN_YOUR_WEBHOOK_SECRET
# Application Configuration
BASE_URL=https://docfast.dev
API_KEYS=FILL_IN_YOUR_API_KEYS_COMMA_SEPARATED
PRO_KEYS=FILL_IN_YOUR_PRO_KEYS_COMMA_SEPARATED
# Database Configuration
DATABASE_PASSWORD=FILL_IN_SECURE_PASSWORD
# Optional: Override defaults if needed
# PORT=3100
# NODE_ENV=production
# SMTP_HOST=host.docker.internal
# SMTP_PORT=25
# DATABASE_HOST=172.17.0.1
# DATABASE_PORT=5432
# DATABASE_NAME=docfast
# DATABASE_USER=docfast
# POOL_SIZE=15
# BROWSER_COUNT=1
# PAGES_PER_BROWSER=15

293
infrastructure/README.md Normal file
View file

@ -0,0 +1,293 @@
# DocFast Infrastructure Guide
Complete disaster recovery and deployment guide for DocFast.
## Quick Start (New Server Deployment)
### 1. Prerequisites
- Fresh Ubuntu 24.04 LTS server
- Root access
- Domain name pointing to server IP
- Stripe account with webhook configured
### 2. Automated Setup
```bash
# Clone the repository
git clone ssh://forgejo@git.cloonar.com/openclawd/docfast.git
cd docfast/infrastructure
# Run the setup script as root
chmod +x setup.sh
./setup.sh
# Follow the post-setup instructions
```
### 3. Manual Configuration Required
After running `setup.sh`, complete these manual steps:
#### SSL Certificate
```bash
certbot --nginx -d docfast.dev -d www.docfast.dev
```
#### DKIM DNS Record
Add this TXT record to your DNS:
```
mail._domainkey.docfast.dev
```
Get the value from: `/etc/opendkim/keys/docfast.dev/mail.txt`
#### Environment Variables
```bash
cd /opt/docfast
cp infrastructure/.env.template .env
# Edit .env with real values
```
#### Start the Application
```bash
cd /opt/docfast
cp infrastructure/docker-compose.yml .
docker-compose up -d
```
## Complete Manual Setup (Step by Step)
If the automated script fails or you prefer manual setup:
### System Packages
```bash
apt update && apt upgrade -y
apt install -y nginx postfix opendkim opendkim-tools certbot \
python3-certbot-nginx ufw docker.io docker-compose-plugin \
git sqlite3 postgresql postgresql-contrib
```
### Firewall Configuration
```bash
ufw --force enable
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow from 172.16.0.0/12 to any port 25 comment "Docker SMTP relay"
ufw allow from 172.16.0.0/12 to any port 5432 comment "Docker PostgreSQL"
```
### PostgreSQL Setup
```bash
sudo -u postgres createuser -D -A -P docfast
sudo -u postgres createdb -O docfast docfast
# Edit /etc/postgresql/16/main/postgresql.conf
echo "listen_addresses = '*'" >> /etc/postgresql/16/main/postgresql.conf
# Edit /etc/postgresql/16/main/pg_hba.conf
echo "host docfast docfast 172.17.0.0/16 md5" >> /etc/postgresql/16/main/pg_hba.conf
echo "host docfast docfast 172.18.0.0/16 md5" >> /etc/postgresql/16/main/pg_hba.conf
systemctl restart postgresql
```
### Nginx Configuration
```bash
cp nginx/docfast.dev /etc/nginx/sites-available/
ln -s /etc/nginx/sites-available/docfast.dev /etc/nginx/sites-enabled/
rm /etc/nginx/sites-enabled/default
nginx -t
systemctl reload nginx
```
### Postfix & OpenDKIM
```bash
cp postfix/main.cf /etc/postfix/
cp postfix/opendkim.conf /etc/opendkim.conf
cp postfix/TrustedHosts /etc/opendkim/
# Generate DKIM keys
mkdir -p /etc/opendkim/keys/docfast.dev
cd /etc/opendkim/keys/docfast.dev
opendkim-genkey -s mail -d docfast.dev
chown opendkim:opendkim mail.private mail.txt
chmod 600 mail.private
systemctl restart postfix opendkim
```
### Application Deployment
```bash
useradd -r -m -s /bin/bash docfast
usermod -aG docker docfast
mkdir -p /opt/docfast
chown docfast:docfast /opt/docfast
cd /opt/docfast
# Copy your source code here
cp infrastructure/docker-compose.yml .
cp infrastructure/.env.template .env
# Edit .env with real values
docker-compose up -d
```
### Backup System
```bash
mkdir -p /opt/docfast-backups
cp scripts/docfast-backup.sh /opt/
chmod +x /opt/docfast-backup.sh
# Add to root crontab
echo "0 */6 * * * /opt/docfast-backup.sh >> /var/log/docfast-backup.log 2>&1" | crontab -
```
## Disaster Recovery Procedures
### Complete Server Failure
1. **Provision new server** with same OS version
2. **Run setup script** from this repository
3. **Restore DNS** records to point to new server
4. **Copy backups** from off-site storage to `/opt/docfast-backups/`
5. **Restore database**:
```bash
docker-compose down
docker volume rm docfast_docfast-data
docker volume create docfast_docfast-data
cp /opt/docfast-backups/docfast-weekly-LATEST.db \
/var/lib/docker/volumes/docfast_docfast-data/_data/docfast.db
docker-compose up -d
```
6. **Verify SSL certificates** with `certbot certificates`
7. **Test email delivery** and DKIM signing
### Database Corruption
```bash
cd /opt/docfast
docker-compose down
# Find latest good backup
ls -la /opt/docfast-backups/
# Restore from backup
cp /opt/docfast-backups/docfast-daily-LATEST.db \
/var/lib/docker/volumes/docfast_docfast-data/_data/docfast.db
docker-compose up -d
```
### Email Delivery Issues
Check DKIM setup:
```bash
# Verify DKIM key is readable
sudo -u opendkim cat /etc/opendkim/keys/docfast.dev/mail.private
# Check OpenDKIM is signing
tail -f /var/log/mail.log
# Test email sending
echo "Test email" | mail -s "Test" test@example.com
```
### SSL Certificate Issues
```bash
# Check certificate status
certbot certificates
# Renew if needed
certbot renew --dry-run
certbot renew
# Fix nginx config if needed
nginx -t
systemctl reload nginx
```
## Monitoring & Maintenance
### Daily Checks
- [ ] Application health: `curl https://docfast.dev/health`
- [ ] Docker containers: `docker ps`
- [ ] Disk space: `df -h`
- [ ] Backup status: `ls -la /opt/docfast-backups/`
### Weekly Checks
- [ ] SSL certificate expiry: `certbot certificates`
- [ ] Email delivery test
- [ ] System updates: `apt list --upgradable`
- [ ] Log rotation: `du -sh /var/log/`
### Monthly Tasks
- [ ] Review backup retention
- [ ] Update system packages
- [ ] Review firewall rules: `ufw status`
- [ ] Check for failed login attempts: `grep "Failed password" /var/log/auth.log`
## Environment Variables Reference
| Variable | Required | Description | Example |
|----------|----------|-------------|---------|
| `STRIPE_SECRET_KEY` | ✅ | Stripe API secret key | `sk_live_...` |
| `STRIPE_WEBHOOK_SECRET` | ✅ | Stripe webhook endpoint secret | `whsec_...` |
| `BASE_URL` | ✅ | Application base URL | `https://docfast.dev` |
| `API_KEYS` | ✅ | Comma-separated API keys | `key1,key2,key3` |
| `PRO_KEYS` | ✅ | Comma-separated pro API keys | `prokey1,prokey2` |
| `DATABASE_PASSWORD` | ✅ | PostgreSQL password | `secure_password_123` |
## DNS Records Required
| Type | Name | Value | TTL |
|------|------|-------|-----|
| A | docfast.dev | SERVER_IP | 300 |
| A | www.docfast.dev | SERVER_IP | 300 |
| TXT | mail._domainkey.docfast.dev | DKIM_PUBLIC_KEY | 300 |
| MX | docfast.dev | docfast.dev | 300 |
| TXT | docfast.dev | v=spf1 mx ~all | 300 |
## Stripe Configuration
Required webhook events:
- `checkout.session.completed`
- `invoice.payment_succeeded`
- `customer.subscription.created`
- `customer.subscription.updated`
- `customer.subscription.deleted`
Webhook URL: `https://docfast.dev/api/stripe/webhook`
## Security Considerations
- Server runs on non-standard SSH port (change from 22)
- Fail2ban recommended for brute force protection
- Regular security updates via unattended-upgrades
- Database backups encrypted at rest
- API keys rotated regularly
- Monitor application logs for suspicious activity
## Troubleshooting
### Common Issues
**Container won't start**: Check logs with `docker-compose logs -f`
**Database connection errors**: Verify PostgreSQL is running and Docker networks are configured
**Email not sending**: Check postfix logs: `tail -f /var/log/mail.log`
**SSL certificate errors**: Verify domain DNS and run `certbot --nginx`
**High memory usage**: Monitor with `docker stats` and adjust container limits
### Log Locations
- Application: `docker-compose logs`
- Nginx: `/var/log/nginx/`
- Postfix: `/var/log/mail.log`
- System: `/var/log/syslog`
- Backups: `/var/log/docfast-backup.log`

View file

@ -0,0 +1,41 @@
version: "3.8"
services:
docfast:
build: .
restart: unless-stopped
ports:
- "127.0.0.1:3100:3100"
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
- API_KEYS=${API_KEYS}
- PORT=3100
- NODE_ENV=production
- STRIPE_SECRET_KEY=${STRIPE_SECRET_KEY}
- STRIPE_WEBHOOK_SECRET=${STRIPE_WEBHOOK_SECRET}
- BASE_URL=${BASE_URL:-https://docfast.dev}
- PRO_KEYS=${PRO_KEYS}
- SMTP_HOST=host.docker.internal
- SMTP_PORT=25
- DATABASE_HOST=172.17.0.1
- DATABASE_PORT=5432
- DATABASE_NAME=docfast
- DATABASE_USER=docfast
- DATABASE_PASSWORD=${DATABASE_PASSWORD:-docfast}
- POOL_SIZE=15
- BROWSER_COUNT=1
- PAGES_PER_BROWSER=15
volumes:
- docfast-data:/app/data
mem_limit: 2560m
cpus: 1.5
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3100/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
volumes:
docfast-data:
driver: local

View file

@ -0,0 +1,70 @@
server {
server_name docfast.dev www.docfast.dev;
# Increase client max body size for file uploads
client_max_body_size 10m;
# Security headers
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
add_header Referrer-Policy "strict-origin-when-cross-origin";
# Proxy to the application
location / {
proxy_pass http://127.0.0.1:3100;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 60s;
proxy_connect_timeout 10s;
# WebSocket support (if needed)
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_cache_bypass $http_upgrade;
}
# Health check endpoint (bypass proxy for direct container health check)
location /health {
access_log off;
proxy_pass http://127.0.0.1:3100/health;
}
# Rate limiting for API endpoints
location /api/ {
limit_req zone=api_limit burst=10 nodelay;
proxy_pass http://127.0.0.1:3100;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# SSL configuration (managed by Certbot)
listen 443 ssl http2;
ssl_certificate /etc/letsencrypt/live/docfast.dev/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/docfast.dev/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
# Rate limiting zone (add to main nginx.conf or here)
# limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
# Redirect HTTP to HTTPS
server {
if ($host = docfast.dev) {
return 301 https://$host$request_uri;
}
if ($host = www.docfast.dev) {
return 301 https://docfast.dev$request_uri;
}
listen 80;
server_name docfast.dev www.docfast.dev;
return 404;
}

View file

@ -0,0 +1,11 @@
# OpenDKIM Trusted Hosts
# These hosts are allowed to send mail through this server
# Localhost
127.0.0.1
localhost
# Docker networks (adjust based on your Docker setup)
172.17.0.0/16
172.18.0.0/16
172.19.0.0/16

View file

@ -0,0 +1,44 @@
# Postfix main configuration for DocFast
# Minimal SMTP relay for application email sending
# Basic configuration
smtpd_banner = $myhostname ESMTP
biff = no
append_dot_mydomain = no
readme_directory = no
compatibility_level = 3.6
# Network configuration
myhostname = docfast.dev
mydomain = docfast.dev
myorigin = docfast.dev
inet_interfaces = 127.0.0.1, 172.17.0.1 # localhost + Docker bridge
inet_protocols = ipv4
mydestination = # Empty = relay only, no local delivery
mynetworks = 127.0.0.0/8, 172.17.0.0/16, 172.18.0.0/16
# TLS configuration
smtp_tls_security_level = may
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
# OpenDKIM integration
milter_protocol = 6
milter_default_action = accept
smtpd_milters = inet:localhost:8891
non_smtpd_milters = $smtpd_milters
# Size limits
mailbox_size_limit = 0
message_size_limit = 10240000 # 10MB
# Other settings
recipient_delimiter = +
disable_vrfy_command = yes
smtpd_helo_required = yes
# Rate limiting
smtpd_client_connection_count_limit = 10
smtpd_client_connection_rate_limit = 10
# Logging
maillog_file = /var/log/postfix.log

View file

@ -0,0 +1,38 @@
# OpenDKIM Configuration for DocFast
# Logging
Syslog yes
SyslogSuccess yes
LogWhy yes
# Operating mode (s = sign, v = verify, sv = both)
Mode sv
# Canonicalization
Canonicalization relaxed/simple
# Domain and selector
Domain docfast.dev
Selector mail
KeyFile /etc/opendkim/keys/docfast.dev/mail.private
# Network
Socket inet:8891@localhost
PidFile /run/opendkim/opendkim.pid
# Security
OversignHeaders From
TrustAnchorFile /usr/share/dns/root.key
UserID opendkim
# Trusted hosts (who can send mail through this server)
InternalHosts /etc/opendkim/TrustedHosts
ExternalIgnoreList /etc/opendkim/TrustedHosts
# Additional security options
RequireSafeKeys yes
SendReports yes
ReportAddress "postmaster@docfast.dev"
# Performance
MaximumHeaders 30

193
infrastructure/setup.sh Executable file
View file

@ -0,0 +1,193 @@
#!/bin/bash
# DocFast Infrastructure Setup Script
# Provisions a fresh Ubuntu/Debian server with all required services
# Run as root: ./setup.sh
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
log() {
echo -e "${GREEN}[INFO]${NC} $1"
}
warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
error() {
echo -e "${RED}[ERROR]${NC} $1"
exit 1
}
# Check if running as root
if [[ $EUID -ne 0 ]]; then
error "This script must be run as root"
fi
# Domain and user configuration
DOMAIN="${DOMAIN:-docfast.dev}"
APP_USER="${APP_USER:-docfast}"
BACKUP_DIR="/opt/docfast-backups"
INSTALL_DIR="/opt/docfast"
log "Setting up DocFast infrastructure for domain: $DOMAIN"
# Update system
log "Updating system packages..."
apt update && apt upgrade -y
# Install required packages
log "Installing required packages..."
apt install -y \
nginx \
postfix \
opendkim \
opendkim-tools \
certbot \
python3-certbot-nginx \
ufw \
docker.io \
docker-compose-plugin \
git \
sqlite3 \
curl \
wget \
unzip \
htop \
postgresql \
postgresql-contrib
# Enable and start services
log "Enabling services..."
systemctl enable nginx postfix opendkim docker postgresql
systemctl start nginx postfix opendkim docker postgresql
# Create application user
if ! id "$APP_USER" &>/dev/null; then
log "Creating application user: $APP_USER"
useradd -r -m -s /bin/bash "$APP_USER"
fi
# Add user to docker group
usermod -aG docker "$APP_USER"
# Setup UFW firewall
log "Configuring firewall..."
ufw --force enable
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow from 172.16.0.0/12 to any port 25 comment "Docker SMTP relay"
ufw allow from 172.16.0.0/12 to any port 5432 comment "Docker PostgreSQL"
# Setup PostgreSQL
log "Configuring PostgreSQL..."
sudo -u postgres createuser -D -A -P docfast || true # -P prompts for password
sudo -u postgres createdb -O docfast docfast || true
# Update PostgreSQL to allow Docker connections
PG_VERSION=$(ls /etc/postgresql/)
PG_CONF="/etc/postgresql/$PG_VERSION/main/postgresql.conf"
PG_HBA="/etc/postgresql/$PG_VERSION/main/pg_hba.conf"
# Backup original configs
cp "$PG_CONF" "$PG_CONF.backup" || true
cp "$PG_HBA" "$PG_HBA.backup" || true
# Allow connections from Docker networks
if ! grep -q "listen_addresses = '*'" "$PG_CONF"; then
sed -i "s/#listen_addresses = 'localhost'/listen_addresses = '*'/" "$PG_CONF"
fi
# Allow Docker networks to connect
if ! grep -q "172.17.0.0/16" "$PG_HBA"; then
echo "host docfast docfast 172.17.0.0/16 md5" >> "$PG_HBA"
echo "host docfast docfast 172.18.0.0/16 md5" >> "$PG_HBA"
fi
systemctl restart postgresql
# Setup OpenDKIM
log "Configuring OpenDKIM..."
mkdir -p /etc/opendkim/keys/"$DOMAIN"
chown -R opendkim:opendkim /etc/opendkim/keys
# Generate DKIM keys if they don't exist
if [[ ! -f /etc/opendkim/keys/"$DOMAIN"/mail.private ]]; then
log "Generating DKIM keys..."
cd /etc/opendkim/keys/"$DOMAIN"
opendkim-genkey -s mail -d "$DOMAIN"
chown opendkim:opendkim mail.private mail.txt
chmod 600 mail.private
fi
# Copy configuration files
log "Installing configuration files..."
cp nginx/"$DOMAIN" /etc/nginx/sites-available/"$DOMAIN" || warn "Nginx config not found, you'll need to configure manually"
cp postfix/main.cf /etc/postfix/main.cf || warn "Postfix config not found, you'll need to configure manually"
cp postfix/opendkim.conf /etc/opendkim.conf || warn "OpenDKIM config not found, you'll need to configure manually"
cp postfix/TrustedHosts /etc/opendkim/TrustedHosts || warn "TrustedHosts config not found, you'll need to configure manually"
# Enable nginx site
if [[ -f /etc/nginx/sites-available/"$DOMAIN" ]]; then
ln -sf /etc/nginx/sites-available/"$DOMAIN" /etc/nginx/sites-enabled/
rm -f /etc/nginx/sites-enabled/default
nginx -t && systemctl reload nginx
fi
# Update configurations with actual domain
log "Updating configuration files..."
sed -i "s/docfast\.dev/$DOMAIN/g" /etc/nginx/sites-available/"$DOMAIN" 2>/dev/null || true
sed -i "s/docfast\.dev/$DOMAIN/g" /etc/postfix/main.cf 2>/dev/null || true
sed -i "s/docfast\.dev/$DOMAIN/g" /etc/opendkim.conf 2>/dev/null || true
# Restart services with new configs
systemctl restart postfix opendkim
# Setup backup directory and script
log "Setting up backup system..."
mkdir -p "$BACKUP_DIR"
cp ../scripts/docfast-backup.sh /opt/docfast-backup.sh || warn "Backup script not found"
chmod +x /opt/docfast-backup.sh
# Add backup cron job
if ! crontab -l 2>/dev/null | grep -q docfast-backup; then
(crontab -l 2>/dev/null; echo "0 */6 * * * /opt/docfast-backup.sh >> /var/log/docfast-backup.log 2>&1") | crontab -
fi
# Setup application directory
log "Setting up application directory..."
mkdir -p "$INSTALL_DIR"
chown "$APP_USER":"$APP_USER" "$INSTALL_DIR"
# Install Docker Compose
if ! command -v docker-compose &> /dev/null; then
log "Installing docker-compose..."
COMPOSE_VERSION=$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep 'tag_name' | cut -d\" -f4)
curl -L "https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
fi
log "Base infrastructure setup complete!"
echo
log "Next steps:"
echo "1. Configure DNS A record for $DOMAIN to point to this server"
echo "2. Generate SSL certificates: certbot --nginx -d $DOMAIN"
echo "3. Copy your .env file with secrets to $INSTALL_DIR/.env"
echo "4. Copy your docker-compose.yml to $INSTALL_DIR/"
echo "5. Build and start the application:"
echo " cd $INSTALL_DIR"
echo " docker-compose up -d"
echo
warn "Remember to:"
echo "- Set up your DKIM DNS record (see /etc/opendkim/keys/$DOMAIN/mail.txt)"
echo "- Configure Stripe webhooks"
echo "- Set up monitoring/alerting"
echo "- Test email delivery"