install script update

INSTALL.md

@@ -0,0 +1,339 @@
+# Torrent Gateway Installation Guide
+
+This guide covers complete installation and setup of the Torrent Gateway on a fresh server.
+
+## Prerequisites
+
+- Ubuntu/Debian Linux server with root access
+- 2GB+ RAM (4GB recommended for transcoding)
+- 20GB+ disk space
+- Open ports 80, 443 (for web access)
+- Optional: Domain name pointed to server IP
+
+## Quick Installation
+
+### Basic Installation (HTTP only)
+```bash
+sudo ./scripts/install_native.sh
+```
+
+### Full Installation with SSL/HTTPS
+```bash
+sudo ./scripts/install_native.sh --domain gateway.example.com --email admin@example.com
+```
+
+### With Monitoring Stack
+```bash
+sudo ./scripts/install_native.sh --domain gateway.example.com --email admin@example.com --with-monitoring
+```
+
+## Installation Options
+
+| Option | Description |
+|--------|-------------|
+| `--domain DOMAIN` | Your domain name (e.g., gateway.example.com) |
+| `--email EMAIL` | Email for SSL certificate notifications |
+| `--skip-ssl` | Skip SSL setup (HTTP only, even with domain) |
+| `--with-monitoring` | Install Prometheus & Grafana monitoring |
+| `--skip-build` | Use existing binary (for reinstalls) |
+| `--help` | Show all options |
+
+## Domain Setup Instructions
+
+### 1. Configure DNS
+Before running the install script with `--domain`, set up your DNS:
+
+**For subdomain (recommended):**
+- Type: `A`
+- Name: `gateway` (or your preferred subdomain)
+- Value: `YOUR_SERVER_IP`
+- TTL: `300` (5 minutes)
+
+**For root domain:**
+- Type: `A`
+- Name: `@`
+- Value: `YOUR_SERVER_IP`
+- TTL: `300`
+
+### 2. Verify DNS Propagation
+```bash
+# Check if your domain points to the server
+dig +short gateway.example.com
+nslookup gateway.example.com
+
+# Should return your server's IP address
+```
+
+### 3. Install with Domain
+```bash
+sudo ./scripts/install_native.sh --domain gateway.example.com --email your@email.com
+```
+
+The installer will:
+- Configure nginx reverse proxy
+- Obtain SSL certificate via Let's Encrypt
+- Set up automatic certificate renewal
+- Configure HTTPS redirects
+
+## What Gets Installed
+
+### System Packages
+- `golang-go` - Go compiler
+- `nginx` - Web server/reverse proxy
+- `certbot` - SSL certificate management
+- `redis-server` - Caching layer
+- `sqlite3` - Database
+- `ffmpeg` - Video transcoding
+- Various utilities (curl, jq, bc, htop, etc.)
+
+### Directory Structure
+```
+/opt/torrent-gateway/
+├── bin/gateway              # Main binary
+├── configs/                 # Configuration files
+├── data/                    # Application data
+│   ├── blobs/              # Small file storage
+│   ├── chunks/             # Chunked file pieces
+│   └── metadata.db         # SQLite database
+├── logs/                    # Application logs
+├── scripts/                 # Management scripts
+└── backups/                # Automated backups
+```
+
+### Services Created
+- `torrent-gateway` - Main application service
+- `nginx` - Web server (configured as reverse proxy)
+- `redis-server` - Caching service
+- `prometheus` - Metrics collection (optional)
+- `grafana-server` - Metrics dashboard (optional)
+
+## Post-Installation
+
+### 1. Verify Installation
+```bash
+# Check service status
+sudo systemctl status torrent-gateway
+sudo systemctl status nginx
+
+# Run health checks
+sudo /opt/torrent-gateway/scripts/health_check.sh
+
+# Test web access
+curl http://localhost/api/health
+# or with domain:
+curl https://gateway.example.com/api/health
+```
+
+### 2. Access Web Interface
+- **With domain:** https://gateway.example.com
+- **Without domain:** http://YOUR_SERVER_IP
+- **Admin panel:** https://gateway.example.com/admin
+- **API:** https://gateway.example.com/api/
+
+### 3. Monitoring (if installed)
+- **Prometheus:** http://localhost:9090
+- **Grafana:** http://localhost:3000 (admin/admin)
+
+## Configuration
+
+### Main Config
+Edit `/opt/torrent-gateway/configs/config.yaml` and restart:
+```bash
+sudo nano /opt/torrent-gateway/configs/config.yaml
+sudo systemctl restart torrent-gateway
+```
+
+### Nginx Config
+For custom nginx settings:
+```bash
+sudo nano /etc/nginx/sites-available/torrent-gateway
+sudo nginx -t
+sudo systemctl restart nginx
+```
+
+## Management Commands
+
+### Service Management
+```bash
+# Start/stop/restart
+sudo systemctl start torrent-gateway
+sudo systemctl stop torrent-gateway
+sudo systemctl restart torrent-gateway
+
+# View logs
+sudo journalctl -u torrent-gateway -f
+sudo tail -f /var/log/nginx/error.log
+```
+
+### SSL Certificate Management
+```bash
+# Check certificate status
+sudo certbot certificates
+
+# Renew certificates manually
+sudo certbot renew
+
+# Test renewal process
+sudo certbot renew --dry-run
+```
+
+### Backup & Restore
+```bash
+# Create backup
+sudo /opt/torrent-gateway/scripts/backup.sh
+
+# List backups
+ls -la /opt/torrent-gateway/backups/
+
+# Restore from backup
+sudo /opt/torrent-gateway/scripts/restore.sh TIMESTAMP
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**SSL Certificate Failed:**
+```bash
+# Check DNS propagation
+dig +short gateway.example.com
+
+# Check firewall
+sudo ufw status
+sudo ufw allow 80/tcp
+sudo ufw allow 443/tcp
+
+# Manually get certificate
+sudo certbot --nginx -d gateway.example.com
+```
+
+**Service Won't Start:**
+```bash
+# Check logs
+sudo journalctl -u torrent-gateway -n 50
+
+# Check config syntax
+/opt/torrent-gateway/bin/gateway -config /opt/torrent-gateway/configs/config.yaml -check
+
+# Check permissions
+sudo chown -R torrent-gateway:torrent-gateway /opt/torrent-gateway/data
+```
+
+**Web Interface Not Loading:**
+```bash
+# Check nginx
+sudo nginx -t
+sudo systemctl status nginx
+
+# Check proxy connection
+curl http://localhost:9877/api/health
+```
+
+### Log Locations
+- Application logs: `journalctl -u torrent-gateway`
+- Nginx logs: `/var/log/nginx/`
+- SSL logs: `/var/log/letsencrypt/`
+- System logs: `/var/log/syslog`
+
+## Firewall Configuration
+
+The installer configures UFW with these rules:
+- Port 22 (SSH)
+- Port 80 (HTTP)
+- Port 443 (HTTPS)
+- Monitoring ports (localhost only)
+
+### Manual Firewall Setup
+```bash
+sudo ufw allow ssh
+sudo ufw allow 80/tcp
+sudo ufw allow 443/tcp
+sudo ufw enable
+```
+
+## Security Features
+
+- Dedicated service user (non-root)
+- Systemd security hardening
+- Rate limiting for uploads/downloads
+- SSL/TLS encryption (when domain configured)
+- Security headers via nginx
+- Firewall configuration
+- Log rotation and cleanup
+
+## Performance Tuning
+
+### For High Traffic
+Edit `/opt/torrent-gateway/configs/config.yaml`:
+```yaml
+rate_limiting:
+  download:
+    requests_per_second: 100.0
+    burst_size: 200
+
+transcoding:
+  concurrent_jobs: 4  # Adjust based on CPU cores
+```
+
+### For Large Files
+Edit `/etc/nginx/sites-available/torrent-gateway`:
+```nginx
+client_max_body_size 50G;  # Increase as needed
+proxy_read_timeout 600s;   # For large uploads
+```
+
+## Monitoring & Maintenance
+
+### Automated Tasks
+The installer sets up cron jobs for:
+- Daily backups (2 AM)
+- Database maintenance (3 AM)
+- Health checks (every 5 minutes)
+- Log cleanup (weekly)
+
+### Manual Maintenance
+```bash
+# Check disk usage
+df -h /opt/torrent-gateway/
+
+# Clean old files
+sudo /opt/torrent-gateway/scripts/cleanup.sh
+
+# Update application
+cd /path/to/source
+git pull
+sudo ./scripts/install_native.sh --skip-build=false
+```
+
+## Uninstalling
+
+```bash
+# Stop services
+sudo systemctl stop torrent-gateway nginx
+sudo systemctl disable torrent-gateway
+
+# Remove files (WARNING: This deletes all data!)
+sudo rm -rf /opt/torrent-gateway
+sudo rm /etc/systemd/system/torrent-gateway.service
+sudo rm /etc/nginx/sites-available/torrent-gateway
+sudo rm /etc/nginx/sites-enabled/torrent-gateway
+
+# Remove user
+sudo userdel torrent-gateway
+
+# Remove packages (optional)
+sudo apt remove golang-go nginx certbot python3-certbot-nginx
+
+# Clean systemd
+sudo systemctl daemon-reload
+```
+
+## Support
+
+For issues, check:
+1. This installation guide
+2. `/docs/troubleshooting.md`
+3. System logs and service status
+4. GitHub issues for known problems
+
+Installation complete! Your Torrent Gateway should now be running and accessible.

scripts/install_native.sh

@@ -21,6 +21,9 @@ fi
 # Parse arguments
 ENABLE_MONITORING=false
 SKIP_BUILD=false
+DOMAIN=""
+EMAIL=""
+SKIP_SSL=false
 
 while [[ $# -gt 0 ]]; do
     case $1 in
@@ -32,13 +35,31 @@ while [[ $# -gt 0 ]]; do
             SKIP_BUILD=true
             shift
             ;;
+        --domain)
+            DOMAIN="$2"
+            shift 2
+            ;;
+        --email)
+            EMAIL="$2"
+            shift 2
+            ;;
+        --skip-ssl)
+            SKIP_SSL=true
+            shift
+            ;;
         --help)
             echo "Usage: $0 [OPTIONS]"
             echo ""
             echo "Options:"
             echo "  --with-monitoring    Install Prometheus, Grafana, and AlertManager"
             echo "  --skip-build         Skip building the application (use existing binary)"
+            echo "  --domain DOMAIN      Domain name for SSL certificate (e.g., gateway.example.com)"
+            echo "  --email EMAIL        Email for Let's Encrypt certificate notifications"
+            echo "  --skip-ssl           Skip SSL/HTTPS setup (HTTP only)"
             echo "  --help               Show this help message"
+            echo ""
+            echo "Example:"
+            echo "  $0 --domain gateway.example.com --email admin@example.com"
             exit 0
             ;;
         *)
@@ -52,6 +73,9 @@ done
 echo "Configuration:"
 echo "  Monitoring: $ENABLE_MONITORING"
 echo "  Skip build: $SKIP_BUILD"
+echo "  Domain: ${DOMAIN:-'Not set (HTTP only)'}"
+echo "  Email: ${EMAIL:-'Not set'}"
+echo "  Skip SSL: $SKIP_SSL"
 echo ""
 
 cd "$PROJECT_ROOT"
@@ -65,6 +89,8 @@ apt-get install -y \
     sqlite3 \
     redis-server \
     nginx \
+    certbot \
+    python3-certbot-nginx \
     logrotate \
     curl \
     jq \
@@ -72,7 +98,8 @@ apt-get install -y \
     htop \
     tree \
     unzip \
-    wget
+    wget \
+    ffmpeg
 
 # Verify Go installation
 if ! command -v go &> /dev/null; then
@@ -167,7 +194,121 @@ EOF
     echo "✅ Node Exporter installed and started"
 fi
 
-# Step 6: Configure firewall
+# Step 6: Configure nginx
+echo "🌐 Configuring nginx..."
+
+# Create nginx configuration
+cat > /etc/nginx/sites-available/torrent-gateway << EOF
+server {
+    listen 80;
+    server_name ${DOMAIN:-'_'};
+    
+    # Security headers
+    add_header X-Frame-Options "SAMEORIGIN" always;
+    add_header X-XSS-Protection "1; mode=block" always;
+    add_header X-Content-Type-Options "nosniff" always;
+    add_header Referrer-Policy "no-referrer-when-downgrade" always;
+    
+    # Main application proxy
+    location / {
+        proxy_pass http://127.0.0.1:9877;
+        proxy_set_header Host \$host;
+        proxy_set_header X-Real-IP \$remote_addr;
+        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto \$scheme;
+        
+        # WebSocket support
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade \$http_upgrade;
+        proxy_set_header Connection "upgrade";
+        
+        # Timeouts for large uploads
+        proxy_connect_timeout 60s;
+        proxy_send_timeout 300s;
+        proxy_read_timeout 300s;
+    }
+    
+    # Health check endpoint
+    location /health {
+        access_log off;
+        proxy_pass http://127.0.0.1:9877/api/health;
+    }
+    
+    # Increase client max body size for file uploads
+    client_max_body_size 10G;
+    client_body_timeout 300s;
+    client_header_timeout 300s;
+    
+    # Enable gzip compression
+    gzip on;
+    gzip_vary on;
+    gzip_min_length 1024;
+    gzip_proxied any;
+    gzip_comp_level 6;
+    gzip_types
+        text/plain
+        text/css
+        text/xml
+        text/javascript
+        application/json
+        application/javascript
+        application/xml+rss
+        application/atom+xml
+        image/svg+xml;
+}
+EOF
+
+# Enable the site
+ln -sf /etc/nginx/sites-available/torrent-gateway /etc/nginx/sites-enabled/
+rm -f /etc/nginx/sites-enabled/default
+
+# Test nginx configuration
+nginx -t
+if [ $? -ne 0 ]; then
+    echo "❌ Nginx configuration error"
+    exit 1
+fi
+
+echo "✅ Nginx configured"
+
+# Step 7: Setup SSL with certbot (if domain provided)
+if [ -n "$DOMAIN" ] && [ "$SKIP_SSL" = false ]; then
+    echo "🔐 Setting up SSL certificate for $DOMAIN..."
+    
+    if [ -z "$EMAIL" ]; then
+        echo "❌ Email is required for SSL certificate"
+        echo "Please provide --email argument or use --skip-ssl"
+        exit 1
+    fi
+    
+    # Start nginx to respond to HTTP challenges
+    systemctl start nginx
+    
+    # Get SSL certificate
+    certbot --nginx --non-interactive --agree-tos --email "$EMAIL" -d "$DOMAIN"
+    
+    if [ $? -eq 0 ]; then
+        echo "✅ SSL certificate obtained successfully"
+        
+        # Setup auto-renewal
+        (crontab -l 2>/dev/null; echo "0 12 * * * /usr/bin/certbot renew --quiet") | crontab -
+        echo "✅ SSL auto-renewal configured"
+    else
+        echo "⚠️  SSL certificate setup failed. Continuing with HTTP only."
+        echo "You may need to:"
+        echo "  1. Ensure $DOMAIN points to this server's IP"
+        echo "  2. Check firewall allows ports 80/443"
+        echo "  3. Run: sudo certbot --nginx -d $DOMAIN"
+    fi
+else
+    if [ -n "$DOMAIN" ]; then
+        echo "⏭️  Skipping SSL setup (--skip-ssl specified)"
+    else
+        echo "⏭️  Skipping SSL setup (no domain provided)"
+    fi
+fi
+
+# Step 8: Configure firewall
 echo "🔒 Configuring firewall..."
 if command -v ufw &> /dev/null; then
     # Allow SSH
@@ -194,7 +335,7 @@ else
     echo "⚠️  UFW not available, skipping firewall configuration"
 fi
 
-# Step 7: Create maintenance scripts
+# Step 9: Create maintenance scripts
 echo "🛠️  Creating maintenance scripts..."
 
 # Create backup cron job
@@ -244,7 +385,7 @@ chmod +x /opt/torrent-gateway/scripts/cleanup.sh
 # Add weekly cleanup to cron
 echo "0 4 * * 0 root /opt/torrent-gateway/scripts/cleanup.sh > /var/log/torrent-gateway-cleanup.log 2>&1" >> /etc/cron.d/torrent-gateway
 
-# Step 8: Final service startup
+# Step 10: Final service startup
 echo "🚀 Starting all services..."
 
 # Start dependencies first
@@ -261,7 +402,11 @@ fi
 
 # Wait for service to be ready
 echo "⏳ Waiting for services to be ready..."
-timeout 60 bash -c 'until curl -sf http://localhost/api/health; do sleep 2; done'
+timeout 60 bash -c 'until curl -sf http://localhost:9877/api/health; do sleep 2; done'
+
+# Test nginx proxy
+echo "🧪 Testing nginx proxy..."
+timeout 30 bash -c 'until curl -sf http://localhost/api/health; do sleep 2; done'
 
 # Run health checks
 echo "🏥 Running health checks..."
@@ -277,8 +422,19 @@ if [ $? -eq 0 ]; then
     echo "   Config: /opt/torrent-gateway/"
     echo ""
     echo "🌐 Access URLs:"
-    echo "   Gateway API: http://localhost/api/"
+    if [ -n "$DOMAIN" ] && [ "$SKIP_SSL" = false ]; then
-    echo "   Admin Panel: http://localhost/admin"
+        echo "   Gateway: https://$DOMAIN"
+        echo "   API: https://$DOMAIN/api/"
+        echo "   Admin Panel: https://$DOMAIN/admin"
+    elif [ -n "$DOMAIN" ]; then
+        echo "   Gateway: http://$DOMAIN"
+        echo "   API: http://$DOMAIN/api/"
+        echo "   Admin Panel: http://$DOMAIN/admin"
+    else
+        echo "   Gateway: http://localhost (or http://YOUR_SERVER_IP)"
+        echo "   API: http://localhost/api/"
+        echo "   Admin Panel: http://localhost/admin"
+    fi
     if [ "$ENABLE_MONITORING" = true ]; then
         echo "   Prometheus: http://localhost:9090"
         echo "   Grafana: http://localhost:3000 (admin/admin)"
@@ -289,6 +445,7 @@ if [ $? -eq 0 ]; then
     echo "   Stop:    sudo systemctl stop torrent-gateway"
     echo "   Restart: sudo systemctl restart torrent-gateway"
     echo "   Status:  sudo systemctl status torrent-gateway"
+    echo "   Nginx:   sudo systemctl restart nginx"
     echo ""
     echo "💾 Backup & Restore:"
     echo "   Backup:  sudo /opt/torrent-gateway/scripts/backup.sh"
@@ -296,8 +453,31 @@ if [ $? -eq 0 ]; then
     echo ""
     echo "📝 Logs and Monitoring:"
     echo "   App logs:    sudo journalctl -u torrent-gateway -f"
+    echo "   Nginx logs:  sudo tail -f /var/log/nginx/error.log"
     echo "   System logs: sudo tail -f /var/log/syslog"
     echo "   Health:      sudo /opt/torrent-gateway/scripts/health_check.sh"
+    echo ""
+    if [ -n "$DOMAIN" ]; then
+        echo "🔐 SSL Certificate:"
+        echo "   Status:      sudo certbot certificates"
+        echo "   Renew:       sudo certbot renew"
+        echo "   Test Renew:  sudo certbot renew --dry-run"
+        echo ""
+    fi
+    echo "🛠️  Domain Setup Instructions:"
+    echo ""
+    echo "   To use a custom domain, you need to:"
+    echo "   1. Point your domain's DNS A record to this server's IP"
+    echo "   2. Ensure ports 80 and 443 are open in your firewall"
+    echo "   3. Re-run install with: sudo $0 --domain yourdomain.com --email your@email.com"
+    echo ""
+    echo "   Example DNS setup:"
+    echo "   - Type: A"
+    echo "   - Name: gateway (or @ for root domain)"
+    echo "   - Value: $(curl -s https://api.ipify.org || echo 'YOUR_SERVER_IP')"
+    echo "   - TTL: 300 (5 minutes)"
+    echo ""
+    echo "   After DNS propagation (usually 5-60 minutes), SSL will be automatically configured."
 else
     echo "❌ Installation completed but health checks failed"
     echo "Check logs: journalctl -u torrent-gateway"