torrent-gateway/README.md

# BitTorrent Gateway

A comprehensive unified content distribution system that seamlessly integrates BitTorrent protocol, WebSeed technology, DHT peer discovery, built-in tracker, and Nostr announcements. This gateway provides intelligent content distribution by automatically selecting the optimal delivery method based on file size and network conditions.

## Architecture Overview

The BitTorrent Gateway operates as a unified system with multiple specialized components working together:

### Core Components

**1. Gateway HTTP API Server (Port 9877)**
- Main web interface and API endpoints
- File upload/download management
- Smart proxy for reassembling chunked content
- WebSeed implementation with advanced LRU caching
- Rate limiting and abuse prevention

**2. Embedded Blossom Server (Port 8082)**
- Nostr-compatible blob storage protocol
- Direct blob storage for small files (<100MB)
- Integration with gateway for seamless operation

**3. DHT Node (Port 6883)**
- Distributed peer discovery
- BitTorrent DHT protocol implementation
- Bootstrap connectivity with major DHT networks
- Automatic torrent announcement and peer sharing

**4. Built-in BitTorrent Tracker**
- Full BitTorrent tracker implementation
- Announce/scrape protocol support
- P2P coordination and peer ranking
- Client compatibility optimizations for qBittorrent, Transmission, WebTorrent, Deluge, uTorrent

### Smart Storage Strategy

The system uses an intelligent dual-storage approach:

- **Small Files (<100MB)**: Stored directly as blobs using Blossom protocol
- **Large Files (≥100MB)**: Automatically chunked into 2MB pieces, stored as torrents with WebSeed fallback

### P2P Coordination System

A sophisticated P2P coordinator manages all networking components:

- **Unified Peer Discovery**: Aggregates peers from tracker, DHT, and WebSeed sources
- **Smart Peer Ranking**: Geographic proximity and performance-based peer selection
- **Load Balancing**: Distributes load across multiple peer sources
- **Health Monitoring**: Real-time monitoring of all P2P components with automatic alerting

## Installation

### Prerequisites

- Go 1.21 or later
- SQLite3
- 10MB+ available storage
- Linux/macOS/Windows

### Quick Start (Standalone)

```bash
# Clone repository
git clone https://git.sovbit.dev/enki/torrentGateway.git
cd torrentGateway

# Build the gateway
go build -o gateway ./cmd/gateway

# Run with default configuration
./gateway
```

The web interface will be available at http://localhost:9877

### Production Deployment (No Docker)

For production deployment without Docker, use the native installation script:

```bash
# Make installation script executable
chmod +x scripts/install_native.sh

# Install as system service
sudo ./scripts/install_native.sh

# Start the service
sudo systemctl start torrent-gateway
sudo systemctl enable torrent-gateway

# Check status
sudo systemctl status torrent-gateway
```

### Nginx Reverse Proxy Configuration

For production deployments, use Nginx as a reverse proxy:

```nginx
server {
    listen 80;
    server_name your-domain.com;
    
    # Redirect HTTP to HTTPS
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name your-domain.com;
    
    # SSL Configuration
    ssl_certificate /path/to/your/certificate.pem;
    ssl_certificate_key /path/to/your/private-key.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;
    
    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "no-referrer-when-downgrade" always;
    add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
    
    # Gateway API and Web Interface
    location / {
        proxy_pass http://127.0.0.1:9877;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket support for real-time features
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        # Increase timeouts for large file uploads
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }
    
    # Blossom Server (optional, if running separately)
    location /blossom/ {
        proxy_pass http://127.0.0.1:8082/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    
    # Increase client max body size for file uploads
    client_max_body_size 10G;
    
    # Enable gzip compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/atom+xml image/svg+xml;
}
```

### SystemD Service Configuration

The native installer creates a systemd service. Manual configuration:

```bash
# Create service file
sudo tee /etc/systemd/system/torrent-gateway.service > /dev/null <<EOF
[Unit]
Description=Torrent Gateway Service
After=network.target

[Service]
Type=simple
User=torrent-gateway
Group=torrent-gateway
WorkingDirectory=/opt/torrent-gateway
ExecStart=/opt/torrent-gateway/gateway -config /etc/torrent-gateway/config.yaml
Restart=always
RestartSec=3
LimitNOFILE=65536

# Security settings
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/torrent-gateway/data
ReadWritePaths=/var/log/torrent-gateway

[Install]
WantedBy=multi-user.target
EOF

# Create dedicated user
sudo useradd --system --home /opt/torrent-gateway --shell /bin/false torrent-gateway

# Create directories and set permissions
sudo mkdir -p /opt/torrent-gateway/{data/{blobs,chunks},logs}
sudo mkdir -p /etc/torrent-gateway
sudo mkdir -p /var/log/torrent-gateway
sudo chown -R torrent-gateway:torrent-gateway /opt/torrent-gateway
sudo chown -R torrent-gateway:torrent-gateway /var/log/torrent-gateway

# Copy binary and config
sudo cp gateway /opt/torrent-gateway/
sudo cp configs/config.yaml /etc/torrent-gateway/
sudo chmod +x /opt/torrent-gateway/gateway

# Enable and start service
sudo systemctl daemon-reload
sudo systemctl enable torrent-gateway
sudo systemctl start torrent-gateway
```

### Configuration

The default configuration is in `configs/config.yaml`. Customize settings there:

```yaml
# Unified Blossom-BitTorrent Gateway Configuration
mode: unified  # unified (all services), gateway-only, blossom-only, dht-only

# Gateway HTTP API server
gateway:
  enabled: true
  port: 9877
  max_upload_size: 10GB

# Embedded Blossom server
blossom_server:
  enabled: true
  port: 8082
  max_blob_size: 100MB
  rate_limit:
    requests_per_minute: 100
    burst_size: 20

# DHT node configuration
dht:
  enabled: true
  port: 6883
  bootstrap_nodes:
    - "router.bittorrent.com:6881"
    - "dht.transmissionbt.com:6881"
    - "router.utorrent.com:6881"
    - "dht.libtorrent.org:25401"
  announce_interval: 900s  # 15 minutes
  max_torrents: 10000
  max_peers_per_torrent: 200

# Shared storage configuration
storage:
  blob_threshold: 104857600    # 100MB in bytes
  chunk_size: 2097152          # 2MB chunks for large files  
  metadata_db: "./data/metadata.db"
  blob_storage: "./data/blobs"
  chunk_storage: "./data/chunks"

# Built-in BitTorrent tracker
tracker:
  enabled: true
  announce_interval: 1800      # 30 minutes
  min_interval: 900            # 15 minutes  
  default_numwant: 50          # peers to return
  max_numwant: 100             # maximum peers

# Nostr relay configuration
nostr:
  relays:
    - "wss://freelay.sovbit.host"

# Admin configuration
admin:
  enabled: true
  pubkeys:
    - "your_admin_pubkey_here"  # Replace with actual admin pubkey
  default_user_storage_limit: "10GB"

# Rate limiting configuration
rate_limiting:
  upload:
    requests_per_second: 1.0    # Max uploads per second per IP
    burst_size: 5               # Burst allowance
    max_file_size: "3GB"        # Maximum file size
  download:
    requests_per_second: 50.0   # Global download rate limit
    burst_size: 100             # Global burst allowance
  stream:
    requests_per_second: 10.0   # Max streams per second per file
    burst_size: 20              # Stream burst allowance
    max_concurrent: 50          # Max concurrent streams
  auth:
    login_attempts_per_minute: 10  # Login attempts per IP per minute
    burst_size: 5                  # Login burst allowance
```

## API Reference

### Authentication

All endpoints support Nostr-based authentication via:
- **NIP-07**: Browser extension (Alby, nos2x)
- **NIP-46**: Remote signer/bunker URL

```bash
# Get challenge
curl http://localhost:9877/api/auth/challenge

# Login (requires signed Nostr event)
curl -X POST http://localhost:9877/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"auth_type": "nip07", "auth_event": "..."}'
```

### File Operations

```bash
# Upload file
curl -X POST http://localhost:9877/api/upload \
  -F "file=@example.mp4" \
  -F "announce_dht=true"

# Download file
curl http://localhost:9877/api/download/[hash] -o downloaded_file

# Get torrent
curl http://localhost:9877/api/torrent/[hash] -o file.torrent

# Stream video (HLS)
curl http://localhost:9877/api/stream/[hash]/playlist.m3u8
```

### User Management

```bash
# Get user stats (requires auth)
curl http://localhost:9877/api/users/me/stats \
  -H "Authorization: Bearer [session_token]"

# List user files
curl http://localhost:9877/api/users/me/files \
  -H "Authorization: Bearer [session_token]"

# Delete file
curl -X DELETE http://localhost:9877/api/users/me/files/[hash] \
  -H "Authorization: Bearer [session_token]"
```

## Nostr Integration

The BitTorrent Gateway seamlessly integrates with the Nostr network to enable decentralized content discovery and social features:

### How Nostr Integration Works

1. **Content Announcements**: When files are uploaded, they're automatically announced to configured Nostr relays as structured events
2. **Decentralized Discovery**: Other nodes and users can discover new content by subscribing to these announcement events
3. **Social Layer**: Users can comment, react, and share content using standard Nostr clients
4. **No Central Authority**: Content discovery happens through the distributed Nostr relay network

### Technical Implementation

- **Event Type**: Uses kind `2003` per NIP-35 specification for torrent announcements
- **Content Addressing**: Files are identified by cryptographic hashes (SHA-256)
- **Metadata**: File size, name, magnet links, and WebSeed URLs included in events
- **Relay Redundancy**: Announces to multiple relays for reliability

### Nostr Event Structure

When a file is uploaded, the gateway creates a Nostr event like this:

```json
{
  "kind": 2003,
  "content": "New torrent: example-video.mp4",
  "tags": [
    ["title", "Example Video"],
    ["x", "d1c5a0f85a4e4f3d8e7d8f9c6b5a4e3f2d1c0b9a8e7d6f5c4b3a2e1d0c9b8a7f6"],
    ["file", "example-video.mp4", "157286400"],
    ["magnet", "magnet:?xt=urn:btih:..."],
    ["webseed", "https://gateway.example.com/api/webseed/..."],
    ["blossom", "sha256_blob_hash"],
    ["stream", "https://gateway.example.com/api/stream/hash"],
    ["hls", "https://gateway.example.com/api/stream/hash/playlist.m3u8"],
    ["duration", "3600"],
    ["video", "1920x1080", "30fps", "h264"],
    ["m", "video/mp4"],
    ["t", "torrent"],
    ["t", "video"],
    ["t", "streaming"],
    ["tcat", "video,streaming"]
  ]
}
```

### Benefits

- **Censorship Resistance**: No single point of control for content discovery
- **Network Effects**: Content spreads naturally through the Nostr social graph
- **Interoperability**: Standard Nostr clients can display and interact with content
- **Privacy**: Users maintain control over their identity and data

## Performance & Scaling

### Optimization Features
- **Concurrent Downloads**: Multiple parallel piece downloads
- **Geographic Peer Selection**: Prioritizes nearby peers for faster transfers  
- **Smart Caching**: LRU eviction with configurable cache sizes
- **Rate Limiting**: Prevents abuse while maintaining performance
- **Connection Pooling**: Efficient resource utilization

### Monitoring & Alerting
- **Component Health Scores**: 0-100 scoring for all P2P components
- **Performance Metrics**: Response times, throughput, error rates
- **Automatic Alerts**: Configurable thresholds for degraded performance
- **Diagnostic Endpoints**: Detailed system introspection

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes with comprehensive tests
4. Submit a pull request

## License

[Add your license information here]

## Support

- **Issues**: Report bugs and feature requests via GitHub issues
- **Documentation**: Additional documentation in `/docs`
- **Community**: [Add community links if available]