torrent-gateway/docs/backup_restore.md

Backup and Restore Procedures

Overview

This guide covers comprehensive backup and restore procedures for the Torrent Gateway, including data, configuration, and disaster recovery scenarios.

Backup Strategy

Automatic Backups

Daily Backup (via cron):

# Configured automatically during installation
# Runs daily at 2 AM
0 2 * * * root /opt/torrent-gateway/scripts/backup.sh

Components Backed Up:

• Database (SQLite file + SQL dump)
• File storage (blobs and chunks)
• Configuration files
• Application logs
• Docker volumes (Docker deployment)

Manual Backup

Create immediate backup:

# Standard backup
./scripts/backup.sh

# Emergency backup with custom name
./scripts/backup.sh emergency

# Backup with specific timestamp
./scripts/backup.sh $(date +%Y%m%d_%H%M%S)

Backup Contents:

• gateway_backup_YYYYMMDD_HHMMSS.tar.gz - Complete system backup
• database_YYYYMMDD_HHMMSS.sql - Database SQL dump
• Stored in ./backups/ directory

Restore Procedures

Standard Restore

List available backups:

ls -la backups/gateway_backup_*.tar.gz

Restore from backup:

# Restore specific backup
./scripts/restore.sh 20240816_143022

# The script will:
# 1. Stop running services
# 2. Create restore point of current state
# 3. Extract backup data
# 4. Restore database from SQL dump
# 5. Start services
# 6. Run health checks

Emergency Recovery

Complete System Failure:

# 1. Boot from rescue media if needed
# 2. Mount data drive
# 3. Navigate to project directory
cd /path/to/torrent-gateway

# 4. Install dependencies
sudo apt-get install sqlite3 curl

# 5. Restore from latest backup
sudo ./scripts/restore.sh $(ls backups/ | grep gateway_backup | tail -1 | sed 's/gateway_backup_\(.*\).tar.gz/\1/')

# 6. Verify restoration
./scripts/health_check.sh

Partial Recovery

Database Only:

# Stop gateway
sudo systemctl stop torrent-gateway

# Backup current database
cp data/metadata.db data/metadata.db.corrupted

# Restore database from SQL backup
sqlite3 data/metadata.db < backups/database_YYYYMMDD_HHMMSS.sql

# Start gateway
sudo systemctl start torrent-gateway

Configuration Only:

# Extract configs from backup
tar -xzf backups/gateway_backup_YYYYMMDD_HHMMSS.tar.gz configs/

# Restart to apply new config
sudo systemctl restart torrent-gateway

Backup Verification

Automated Verification

The backup script automatically verifies:

• Archive integrity (checksum)
• Database dump validity
• File count consistency

Manual Verification

Test backup integrity:

# Test archive
tar -tzf backups/gateway_backup_YYYYMMDD_HHMMSS.tar.gz > /dev/null
echo "Archive integrity: $?"

# Test database dump
sqlite3 :memory: < backups/database_YYYYMMDD_HHMMSS.sql
echo "Database dump validity: $?"

Verify backup contents:

# List backup contents
tar -tzf backups/gateway_backup_YYYYMMDD_HHMMSS.tar.gz

# Check database schema
sqlite3 data/metadata.db ".schema" > current_schema.sql
sqlite3 :memory: < backups/database_YYYYMMDD_HHMMSS.sql
sqlite3 :memory: ".schema" > backup_schema.sql
diff current_schema.sql backup_schema.sql

Backup Retention

Automatic Cleanup

Retention Policy (configured in cleanup script):

• Daily backups: Keep 30 days
• Weekly backups: Keep 12 weeks
• Monthly backups: Keep 12 months

Manual Cleanup:

# Remove backups older than 30 days
find backups/ -name "gateway_backup_*.tar.gz" -mtime +30 -delete

# Remove old database dumps
find backups/ -name "database_*.sql" -mtime +30 -delete

Archive to Cold Storage

For long-term retention:

# Compress older backups
find backups/ -name "*.tar.gz" -mtime +7 -exec gzip -9 {} \;

# Move to archive location
find backups/ -name "*.tar.gz.gz" -mtime +30 -exec mv {} /archive/location/ \;

Disaster Recovery

Complete Site Recovery

Recovery from offsite backup:

1. Provision new hardware/VM
2. Install operating system (Ubuntu 20.04+ recommended)
3. Restore from backup:

   # Download backup from offsite storage
   wget/scp/rsync your-backup-location/gateway_backup_YYYYMMDD_HHMMSS.tar.gz
   
   # Install Torrent Gateway
   git clone <repository-url>
   cd torrent-gateway
   sudo ./scripts/install_native.sh --skip-build
   
   # Restore data
   sudo ./scripts/restore.sh YYYYMMDD_HHMMSS
   

Data Migration

Moving to new server:

1. Create backup on old server:

   ./scripts/backup.sh migration
   

2. Transfer backup:

   scp backups/gateway_backup_*.tar.gz newserver:/tmp/
   

3. Install on new server:

   # On new server
   sudo ./scripts/install_native.sh
   sudo ./scripts/restore.sh <timestamp>
   

4. Update DNS/load balancer to point to new server

Database Migration

Upgrading SQLite to PostgreSQL:

1. Export data:

   # Export to SQL
   sqlite3 data/metadata.db .dump > export.sql
   
   # Convert SQLite SQL to PostgreSQL format
   sed -i 's/INTEGER PRIMARY KEY AUTOINCREMENT/SERIAL PRIMARY KEY/g' export.sql
   sed -i 's/datetime()/NOW()/g' export.sql
   

2. Import to PostgreSQL:

   createdb torrent_gateway
   psql torrent_gateway < export.sql
   

Backup Testing

Regular Testing Schedule

Monthly restore test:

#!/bin/bash
# test_restore.sh

# Create test environment
mkdir -p /tmp/restore_test
cd /tmp/restore_test

# Copy latest backup
cp /opt/torrent-gateway/backups/gateway_backup_*.tar.gz ./

# Extract and verify
tar -xzf gateway_backup_*.tar.gz
sqlite3 data/metadata.db "PRAGMA integrity_check;"
sqlite3 data/metadata.db "SELECT COUNT(*) FROM files;"

echo "✅ Restore test completed successfully"

Backup Monitoring

Monitor backup success:

# Check last backup status
tail -20 /var/log/torrent-gateway-backup.log

# Verify recent backups exist
ls -la /opt/torrent-gateway/backups/ | head -10

# Check backup sizes (should be consistent)
du -sh /opt/torrent-gateway/backups/gateway_backup_*.tar.gz | tail -5

Offsite Backup Configuration

AWS S3 Integration

# Install AWS CLI
apt-get install awscli

# Configure backup upload
cat >> /opt/torrent-gateway/scripts/backup.sh << 'EOF'

# Upload to S3 after backup creation
if [ -n "$AWS_S3_BUCKET" ]; then
    aws s3 cp "$BACKUP_FILE" "s3://$AWS_S3_BUCKET/backups/"
    aws s3 cp "$DB_BACKUP_FILE" "s3://$AWS_S3_BUCKET/backups/"
    echo "✅ Backup uploaded to S3"
fi
EOF

rsync to Remote Server

# Add to backup script
cat >> /opt/torrent-gateway/scripts/backup.sh << 'EOF'

# Sync to remote server
if [ -n "$BACKUP_REMOTE_HOST" ]; then
    rsync -av --compress backups/ "$BACKUP_REMOTE_HOST:/backups/torrent-gateway/"
    echo "✅ Backup synced to remote server"
fi
EOF

Security Considerations

Backup Encryption

Encrypt sensitive backups:

# Create encrypted backup
./scripts/backup.sh
gpg --symmetric --cipher-algo AES256 backups/gateway_backup_*.tar.gz

# Decrypt for restore
gpg --decrypt backups/gateway_backup_*.tar.gz.gpg > /tmp/backup.tar.gz

Access Control

Backup file permissions:

# Restrict backup access
chmod 600 backups/*.tar.gz
chown root:root backups/*.tar.gz

Secure backup storage:

• Use encrypted storage for offsite backups
• Implement access logging for backup access
• Regular audit of backup access permissions

Recovery Time Objectives

Target Recovery Times

RTO (Recovery Time Objective):

• Database only: < 5 minutes
• Full service: < 15 minutes
• Complete disaster recovery: < 2 hours

RPO (Recovery Point Objective):

• Maximum data loss: 24 hours (daily backups)
• Database transactions: < 1 hour (with WAL mode)

Improving Recovery Times

Reduce RTO:

• Keep hot spare server ready
• Implement automated failover
• Use faster storage for backups
• Optimize restore scripts

Reduce RPO:

• Increase backup frequency
• Implement continuous replication
• Use database WAL mode
• Stream backups to offsite storage