torrent-gateway/README.md

BitTorrent Gateway

A comprehensive unified content distribution system that seamlessly integrates BitTorrent protocol, WebSeed technology, DHT peer discovery, built-in tracker, and Nostr announcements. This gateway provides intelligent content distribution by automatically selecting the optimal delivery method based on file size and network conditions.

Architecture Overview

The BitTorrent Gateway operates as a unified system with multiple specialized components working together:

Core Components

1. Gateway HTTP API Server (Port 9877)

• Main web interface and API endpoints
• File upload/download management
• Smart proxy for reassembling chunked content
• WebSeed implementation with advanced LRU caching
• Rate limiting and abuse prevention

2. Embedded Blossom Server (Port 8082)

• Nostr-compatible blob storage protocol
• Direct blob storage for small files (<100MB)
• Integration with gateway for seamless operation

3. DHT Node (Port 6883)

• Distributed peer discovery
• BitTorrent DHT protocol implementation
• Bootstrap connectivity with major DHT networks
• Automatic torrent announcement and peer sharing

4. Built-in BitTorrent Tracker

• Full BitTorrent tracker implementation
• Announce/scrape protocol support
• P2P coordination and peer ranking
• Client compatibility optimizations for qBittorrent, Transmission, WebTorrent, Deluge, uTorrent

Smart Storage Strategy

The system uses an intelligent dual-storage approach:

• Small Files (<100MB): Stored directly as blobs using Blossom protocol
• Large Files (≥100MB): Automatically chunked into 2MB pieces, stored as torrents with WebSeed fallback

P2P Coordination System

A sophisticated P2P coordinator manages all networking components:

• Unified Peer Discovery: Aggregates peers from tracker, DHT, and WebSeed sources
• Smart Peer Ranking: Geographic proximity and performance-based peer selection
• Load Balancing: Distributes load across multiple peer sources
• Health Monitoring: Real-time monitoring of all P2P components with automatic alerting

Installation

Prerequisites

• Go 1.21 or later
• SQLite3
• 10MB+ available storage
• Linux/macOS/Windows

Quick Start (Standalone)

# Clone repository
git clone https://git.sovbit.dev/enki/torrentGateway.git
cd torrentGateway

# Build the gateway
go build -o gateway ./cmd/gateway

# Run with default configuration
./gateway

The web interface will be available at http://localhost:9877

Production Deployment (No Docker)

For production deployment without Docker, use the native installation script:

# Make installation script executable
chmod +x scripts/install_native.sh

# Install as system service
sudo ./scripts/install_native.sh

# Start the service
sudo systemctl start torrent-gateway
sudo systemctl enable torrent-gateway

# Check status
sudo systemctl status torrent-gateway

Nginx Reverse Proxy Configuration

For production deployments, use Nginx as a reverse proxy:

server {
    listen 80;
    server_name your-domain.com;
    
    # Redirect HTTP to HTTPS
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name your-domain.com;
    
    # SSL Configuration
    ssl_certificate /path/to/your/certificate.pem;
    ssl_certificate_key /path/to/your/private-key.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;
    
    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "no-referrer-when-downgrade" always;
    add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
    
    # Gateway API and Web Interface
    location / {
        proxy_pass http://127.0.0.1:9877;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket support for real-time features
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        # Increase timeouts for large file uploads
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }
    
    # Blossom Server (optional, if running separately)
    location /blossom/ {
        proxy_pass http://127.0.0.1:8082/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    
    # Increase client max body size for file uploads
    client_max_body_size 10G;
    
    # Enable gzip compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/atom+xml image/svg+xml;
}

SystemD Service Configuration

The native installer creates a systemd service. Manual configuration:

# Create service file
sudo tee /etc/systemd/system/torrent-gateway.service > /dev/null <<EOF
[Unit]
Description=Torrent Gateway Service
After=network.target

[Service]
Type=simple
User=torrent-gateway
Group=torrent-gateway
WorkingDirectory=/opt/torrent-gateway
ExecStart=/opt/torrent-gateway/gateway -config /etc/torrent-gateway/config.yaml
Restart=always
RestartSec=3
LimitNOFILE=65536

# Security settings
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/torrent-gateway/data
ReadWritePaths=/var/log/torrent-gateway

[Install]
WantedBy=multi-user.target
EOF

# Create dedicated user
sudo useradd --system --home /opt/torrent-gateway --shell /bin/false torrent-gateway

# Create directories and set permissions
sudo mkdir -p /opt/torrent-gateway/{data/{blobs,chunks},logs}
sudo mkdir -p /etc/torrent-gateway
sudo mkdir -p /var/log/torrent-gateway
sudo chown -R torrent-gateway:torrent-gateway /opt/torrent-gateway
sudo chown -R torrent-gateway:torrent-gateway /var/log/torrent-gateway

# Copy binary and config
sudo cp gateway /opt/torrent-gateway/
sudo cp configs/config.yaml /etc/torrent-gateway/
sudo chmod +x /opt/torrent-gateway/gateway

# Enable and start service
sudo systemctl daemon-reload
sudo systemctl enable torrent-gateway
sudo systemctl start torrent-gateway

Configuration

The default configuration is in configs/config.yaml. Customize settings there:

# Unified Blossom-BitTorrent Gateway Configuration
mode: unified  # unified (all services), gateway-only, blossom-only, dht-only

# Gateway HTTP API server
gateway:
  enabled: true
  port: 9877
  max_upload_size: 10GB

# Embedded Blossom server
blossom_server:
  enabled: true
  port: 8082
  max_blob_size: 100MB
  rate_limit:
    requests_per_minute: 100
    burst_size: 20

# DHT node configuration
dht:
  enabled: true
  port: 6883
  bootstrap_nodes:
    - "router.bittorrent.com:6881"
    - "dht.transmissionbt.com:6881"
    - "router.utorrent.com:6881"
    - "dht.libtorrent.org:25401"
  announce_interval: 900s  # 15 minutes
  max_torrents: 10000
  max_peers_per_torrent: 200

# Shared storage configuration
storage:
  blob_threshold: 104857600    # 100MB in bytes
  chunk_size: 2097152          # 2MB chunks for large files  
  metadata_db: "./data/metadata.db"
  blob_storage: "./data/blobs"
  chunk_storage: "./data/chunks"

# Built-in BitTorrent tracker
tracker:
  enabled: true
  announce_interval: 1800      # 30 minutes
  min_interval: 900            # 15 minutes  
  default_numwant: 50          # peers to return
  max_numwant: 100             # maximum peers

# Nostr relay configuration
nostr:
  relays:
    - "wss://freelay.sovbit.host"

# Admin configuration
admin:
  enabled: true
  pubkeys:
    - "your_admin_pubkey_here"  # Replace with actual admin pubkey
  default_user_storage_limit: "10GB"

# Rate limiting configuration
rate_limiting:
  upload:
    requests_per_second: 1.0    # Max uploads per second per IP
    burst_size: 5               # Burst allowance
    max_file_size: "3GB"        # Maximum file size
  download:
    requests_per_second: 50.0   # Global download rate limit
    burst_size: 100             # Global burst allowance
  stream:
    requests_per_second: 10.0   # Max streams per second per file
    burst_size: 20              # Stream burst allowance
    max_concurrent: 50          # Max concurrent streams
  auth:
    login_attempts_per_minute: 10  # Login attempts per IP per minute
    burst_size: 5                  # Login burst allowance

API Reference

Authentication

All endpoints support Nostr-based authentication via:

• NIP-07: Browser extension (Alby, nos2x)
• NIP-46: Remote signer/bunker URL

# Get challenge
curl http://localhost:9877/api/auth/challenge

# Login (requires signed Nostr event)
curl -X POST http://localhost:9877/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"auth_type": "nip07", "auth_event": "..."}'

File Operations

# Upload file
curl -X POST http://localhost:9877/api/upload \
  -F "file=@example.mp4" \
  -F "announce_dht=true"

# Download file
curl http://localhost:9877/api/download/[hash] -o downloaded_file

# Get torrent
curl http://localhost:9877/api/torrent/[hash] -o file.torrent

# Stream video (HLS)
curl http://localhost:9877/api/stream/[hash]/playlist.m3u8

User Management

# Get user stats (requires auth)
curl http://localhost:9877/api/users/me/stats \
  -H "Authorization: Bearer [session_token]"

# List user files
curl http://localhost:9877/api/users/me/files \
  -H "Authorization: Bearer [session_token]"

# Delete file
curl -X DELETE http://localhost:9877/api/users/me/files/[hash] \
  -H "Authorization: Bearer [session_token]"

Nostr Integration

The BitTorrent Gateway seamlessly integrates with the Nostr network to enable decentralized content discovery and social features:

How Nostr Integration Works

1. Content Announcements: When files are uploaded, they're automatically announced to configured Nostr relays as structured events
2. Decentralized Discovery: Other nodes and users can discover new content by subscribing to these announcement events
3. Social Layer: Users can comment, react, and share content using standard Nostr clients
4. No Central Authority: Content discovery happens through the distributed Nostr relay network

Technical Implementation

• Event Type: Uses kind 1063 for file/torrent announcements
• Content Addressing: Files are identified by cryptographic hashes (SHA-256)
• Metadata: File size, name, magnet links, and WebSeed URLs included in events
• Relay Redundancy: Announces to multiple relays for reliability

Nostr Event Structure

When a file is uploaded, the gateway creates a Nostr event like this:

{
  "kind": 1063,
  "content": "New torrent: example-file.zip",
  "tags": [
    ["magnet", "magnet:?xt=urn:btih:..."],
    ["size", "104857600"],
    ["name", "example-file.zip"],
    ["webseed", "https://gateway.example.com/api/webseed/..."]
  ]
}

Benefits

• Censorship Resistance: No single point of control for content discovery
• Network Effects: Content spreads naturally through the Nostr social graph
• Interoperability: Standard Nostr clients can display and interact with content
• Privacy: Users maintain control over their identity and data

Performance & Scaling

Optimization Features

• Concurrent Downloads: Multiple parallel piece downloads
• Geographic Peer Selection: Prioritizes nearby peers for faster transfers
• Smart Caching: LRU eviction with configurable cache sizes
• Rate Limiting: Prevents abuse while maintaining performance
• Connection Pooling: Efficient resource utilization

Monitoring & Alerting

• Component Health Scores: 0-100 scoring for all P2P components
• Performance Metrics: Response times, throughput, error rates
• Automatic Alerts: Configurable thresholds for degraded performance
• Diagnostic Endpoints: Detailed system introspection

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes with comprehensive tests
4. Submit a pull request

License

[Add your license information here]

Support

• Issues: Report bugs and feature requests via GitHub issues
• Documentation: Additional documentation in /docs
• Community: [Add community links if available]