torrent-gateway/docs/backup_restore.md

# Backup and Restore Procedures

## Overview

This guide covers comprehensive backup and restore procedures for the Torrent Gateway, including data, configuration, and disaster recovery scenarios.

## Backup Strategy

### Automatic Backups

**Daily Backup (via cron):**
```bash
# Configured automatically during installation
# Runs daily at 2 AM
0 2 * * * root /opt/torrent-gateway/scripts/backup.sh
```

**Components Backed Up:**
- Database (SQLite file + SQL dump)
- File storage (blobs and chunks)
- Configuration files
- Application logs
- Docker volumes (Docker deployment)

### Manual Backup

**Create immediate backup:**
```bash
# Standard backup
./scripts/backup.sh

# Emergency backup with custom name
./scripts/backup.sh emergency

# Backup with specific timestamp
./scripts/backup.sh $(date +%Y%m%d_%H%M%S)
```

**Backup Contents:**
- `gateway_backup_YYYYMMDD_HHMMSS.tar.gz` - Complete system backup
- `database_YYYYMMDD_HHMMSS.sql` - Database SQL dump
- Stored in `./backups/` directory

## Restore Procedures

### Standard Restore

**List available backups:**
```bash
ls -la backups/gateway_backup_*.tar.gz
```

**Restore from backup:**
```bash
# Restore specific backup
./scripts/restore.sh 20240816_143022

# The script will:
# 1. Stop running services
# 2. Create restore point of current state
# 3. Extract backup data
# 4. Restore database from SQL dump
# 5. Start services
# 6. Run health checks
```

### Emergency Recovery

**Complete System Failure:**
```bash
# 1. Boot from rescue media if needed
# 2. Mount data drive
# 3. Navigate to project directory
cd /path/to/torrent-gateway

# 4. Install dependencies
sudo apt-get install sqlite3 curl

# 5. Restore from latest backup
sudo ./scripts/restore.sh $(ls backups/ | grep gateway_backup | tail -1 | sed 's/gateway_backup_\(.*\).tar.gz/\1/')

# 6. Verify restoration
./scripts/health_check.sh
```

### Partial Recovery

**Database Only:**
```bash
# Stop gateway
sudo systemctl stop torrent-gateway

# Backup current database
cp data/metadata.db data/metadata.db.corrupted

# Restore database from SQL backup
sqlite3 data/metadata.db < backups/database_YYYYMMDD_HHMMSS.sql

# Start gateway
sudo systemctl start torrent-gateway
```

**Configuration Only:**
```bash
# Extract configs from backup
tar -xzf backups/gateway_backup_YYYYMMDD_HHMMSS.tar.gz configs/

# Restart to apply new config
sudo systemctl restart torrent-gateway
```

## Backup Verification

### Automated Verification

The backup script automatically verifies:
- Archive integrity (checksum)
- Database dump validity
- File count consistency

### Manual Verification

**Test backup integrity:**
```bash
# Test archive
tar -tzf backups/gateway_backup_YYYYMMDD_HHMMSS.tar.gz > /dev/null
echo "Archive integrity: $?"

# Test database dump
sqlite3 :memory: < backups/database_YYYYMMDD_HHMMSS.sql
echo "Database dump validity: $?"
```

**Verify backup contents:**
```bash
# List backup contents
tar -tzf backups/gateway_backup_YYYYMMDD_HHMMSS.tar.gz

# Check database schema
sqlite3 data/metadata.db ".schema" > current_schema.sql
sqlite3 :memory: < backups/database_YYYYMMDD_HHMMSS.sql
sqlite3 :memory: ".schema" > backup_schema.sql
diff current_schema.sql backup_schema.sql
```

## Backup Retention

### Automatic Cleanup

**Retention Policy (configured in cleanup script):**
- Daily backups: Keep 30 days
- Weekly backups: Keep 12 weeks  
- Monthly backups: Keep 12 months

**Manual Cleanup:**
```bash
# Remove backups older than 30 days
find backups/ -name "gateway_backup_*.tar.gz" -mtime +30 -delete

# Remove old database dumps
find backups/ -name "database_*.sql" -mtime +30 -delete
```

### Archive to Cold Storage

**For long-term retention:**
```bash
# Compress older backups
find backups/ -name "*.tar.gz" -mtime +7 -exec gzip -9 {} \;

# Move to archive location
find backups/ -name "*.tar.gz.gz" -mtime +30 -exec mv {} /archive/location/ \;
```

## Disaster Recovery

### Complete Site Recovery

**Recovery from offsite backup:**

1. **Provision new hardware/VM**
2. **Install operating system** (Ubuntu 20.04+ recommended)
3. **Restore from backup:**
   ```bash
   # Download backup from offsite storage
   wget/scp/rsync your-backup-location/gateway_backup_YYYYMMDD_HHMMSS.tar.gz
   
   # Install Torrent Gateway
   git clone <repository-url>
   cd torrent-gateway
   sudo ./scripts/install_native.sh --skip-build
   
   # Restore data
   sudo ./scripts/restore.sh YYYYMMDD_HHMMSS
   ```

### Data Migration

**Moving to new server:**

1. **Create backup on old server:**
   ```bash
   ./scripts/backup.sh migration
   ```

2. **Transfer backup:**
   ```bash
   scp backups/gateway_backup_*.tar.gz newserver:/tmp/
   ```

3. **Install on new server:**
   ```bash
   # On new server
   sudo ./scripts/install_native.sh
   sudo ./scripts/restore.sh <timestamp>
   ```

4. **Update DNS/load balancer** to point to new server

### Database Migration

**Upgrading SQLite to PostgreSQL:**

1. **Export data:**
   ```bash
   # Export to SQL
   sqlite3 data/metadata.db .dump > export.sql
   
   # Convert SQLite SQL to PostgreSQL format
   sed -i 's/INTEGER PRIMARY KEY AUTOINCREMENT/SERIAL PRIMARY KEY/g' export.sql
   sed -i 's/datetime()/NOW()/g' export.sql
   ```

2. **Import to PostgreSQL:**
   ```bash
   createdb torrent_gateway
   psql torrent_gateway < export.sql
   ```

## Backup Testing

### Regular Testing Schedule

**Monthly restore test:**
```bash
#!/bin/bash
# test_restore.sh

# Create test environment
mkdir -p /tmp/restore_test
cd /tmp/restore_test

# Copy latest backup
cp /opt/torrent-gateway/backups/gateway_backup_*.tar.gz ./

# Extract and verify
tar -xzf gateway_backup_*.tar.gz
sqlite3 data/metadata.db "PRAGMA integrity_check;"
sqlite3 data/metadata.db "SELECT COUNT(*) FROM files;"

echo "✅ Restore test completed successfully"
```

### Backup Monitoring

**Monitor backup success:**
```bash
# Check last backup status
tail -20 /var/log/torrent-gateway-backup.log

# Verify recent backups exist
ls -la /opt/torrent-gateway/backups/ | head -10

# Check backup sizes (should be consistent)
du -sh /opt/torrent-gateway/backups/gateway_backup_*.tar.gz | tail -5
```

## Offsite Backup Configuration

### AWS S3 Integration

```bash
# Install AWS CLI
apt-get install awscli

# Configure backup upload
cat >> /opt/torrent-gateway/scripts/backup.sh << 'EOF'

# Upload to S3 after backup creation
if [ -n "$AWS_S3_BUCKET" ]; then
    aws s3 cp "$BACKUP_FILE" "s3://$AWS_S3_BUCKET/backups/"
    aws s3 cp "$DB_BACKUP_FILE" "s3://$AWS_S3_BUCKET/backups/"
    echo "✅ Backup uploaded to S3"
fi
EOF
```

### rsync to Remote Server

```bash
# Add to backup script
cat >> /opt/torrent-gateway/scripts/backup.sh << 'EOF'

# Sync to remote server
if [ -n "$BACKUP_REMOTE_HOST" ]; then
    rsync -av --compress backups/ "$BACKUP_REMOTE_HOST:/backups/torrent-gateway/"
    echo "✅ Backup synced to remote server"
fi
EOF
```

## Security Considerations

### Backup Encryption

**Encrypt sensitive backups:**
```bash
# Create encrypted backup
./scripts/backup.sh
gpg --symmetric --cipher-algo AES256 backups/gateway_backup_*.tar.gz

# Decrypt for restore
gpg --decrypt backups/gateway_backup_*.tar.gz.gpg > /tmp/backup.tar.gz
```

### Access Control

**Backup file permissions:**
```bash
# Restrict backup access
chmod 600 backups/*.tar.gz
chown root:root backups/*.tar.gz
```

**Secure backup storage:**
- Use encrypted storage for offsite backups
- Implement access logging for backup access
- Regular audit of backup access permissions

## Recovery Time Objectives

### Target Recovery Times

**RTO (Recovery Time Objective):**
- Database only: < 5 minutes
- Full service: < 15 minutes
- Complete disaster recovery: < 2 hours

**RPO (Recovery Point Objective):**
- Maximum data loss: 24 hours (daily backups)
- Database transactions: < 1 hour (with WAL mode)

### Improving Recovery Times

**Reduce RTO:**
- Keep hot spare server ready
- Implement automated failover
- Use faster storage for backups
- Optimize restore scripts

**Reduce RPO:**
- Increase backup frequency
- Implement continuous replication
- Use database WAL mode
- Stream backups to offsite storage