# BitTorrent Gateway A comprehensive unified content distribution system that seamlessly integrates BitTorrent protocol, WebSeed technology, DHT peer discovery, built-in tracker, video transcoding, Blossom blob storage, and multi-relay Nostr announcements. This gateway provides intelligent content distribution by automatically selecting the optimal delivery method based on file size and network conditions, with automatic video transcoding, cleanup automation, and performance optimizations. ## Architecture Overview The BitTorrent Gateway operates as a unified system with multiple specialized components working together: ### Core Components **1. Gateway HTTP API Server (Port 9877)** - Main web interface and API endpoints - File upload/download management - Smart proxy for reassembling chunked content - WebSeed implementation with advanced LRU caching - Rate limiting and abuse prevention **2. Embedded Blossom Server (Port 8082)** - Nostr-compatible blob storage protocol - Direct blob storage for small files (<100MB) - Integration with gateway for seamless operation **3. DHT Node (Port 6883)** - Distributed peer discovery - BitTorrent DHT protocol implementation - Bootstrap connectivity with major DHT networks - Automatic torrent announcement and peer sharing **4. Built-in BitTorrent Tracker** - Full BitTorrent tracker implementation - Announce/scrape protocol support - P2P coordination and peer ranking - Client compatibility optimizations for qBittorrent, Transmission, WebTorrent, Deluge, uTorrent **5. Video Transcoding Engine** - Automatic H.264/AAC conversion for web compatibility - Smart serving: transcoded versions when ready, originals otherwise - Background processing with priority queuing - FFmpeg integration with progress tracking - Multiple quality profiles (1080p, 720p, 480p) with configurable bitrates **6. Automated Cleanup System** - Intelligent file lifecycle management - Automatic removal of old files (configurable age: 90d default) - Orphaned chunk detection and cleanup - Inactive user cleanup (180 days default) - Smart proxy cache auto-cleanup (10-minute intervals) - Database optimization with SQLite pragma tuning **7. Multi-Relay Nostr Integration** - NIP-35 compliant torrent announcements - NIP-71 video event publishing (horizontal/vertical detection) - WebTorrent extensions with WebSocket tracker support - Blossom hash cross-referencing - Configurable relay sets for different content types - Fault-tolerant publishing with success/failure tracking ### Smart Storage Strategy The system uses an intelligent dual-storage approach with video optimization: - **Small Files (<100MB)**: Stored directly as blobs using Blossom protocol - **Large Files (≥100MB)**: Automatically chunked into 2MB pieces, stored as torrents with WebSeed fallback - **Video Files**: Automatically queued for H.264/AAC transcoding to web-compatible MP4 format - **Smart Serving**: Transcoded versions served when ready, original chunks as fallback ### P2P Coordination System A sophisticated P2P coordinator manages all networking components: - **Unified Peer Discovery**: Aggregates peers from tracker, DHT, and WebSeed sources - **Smart Peer Ranking**: Geographic proximity and performance-based peer selection - **Load Balancing**: Distributes load across multiple peer sources - **Health Monitoring**: Real-time monitoring of all P2P components with automatic alerting ## Installation ### Prerequisites - Go 1.21 or later - SQLite3 - FFmpeg (for video transcoding, optional) - 10MB+ available storage - Linux/macOS/Windows ### Quick Start (Standalone) ```bash # Clone repository git clone https://git.sovbit.dev/enki/torrentGateway.git cd torrentGateway # Build the gateway go build -o gateway ./cmd/gateway # Run with default configuration ./gateway ``` The web interface will be available at http://localhost:9877 ### Production Deployment For production deployment, use the automated installation script: ```bash # Basic installation (HTTP only) sudo ./scripts/install_native.sh # Full installation with SSL/HTTPS sudo ./scripts/install_native.sh --domain gateway.example.com --email admin@example.com # With monitoring stack sudo ./scripts/install_native.sh --domain gateway.example.com --email admin@example.com --with-monitoring ``` This will automatically: - Install all dependencies (Go, nginx, Redis, FFmpeg, etc.) - Build and deploy the application - Configure systemd service - Set up nginx reverse proxy - Configure SSL certificates (if domain provided) - Start all services See `INSTALL.md` for detailed installation instructions and domain setup. ### Manual Installation (Development) For development or manual setup, you can run the application directly: ```bash # Build the application go build -o bin/gateway cmd/gateway/main.go # Create basic directories mkdir -p data/{blobs,chunks} # Run with default config ./bin/gateway # Or with custom config ./bin/gateway -config configs/config.yaml ``` For production deployments, use the automated installation script instead. ### Configuration The default configuration is in `configs/config.yaml`. Customize settings there: ```yaml # Unified Blossom-BitTorrent Gateway Configuration mode: unified # unified (all services), gateway-only, blossom-only, dht-only # Gateway HTTP API server gateway: enabled: true port: 9877 max_upload_size: 10GB # Embedded Blossom server blossom_server: enabled: true port: 8082 max_blob_size: 100MB rate_limit: requests_per_minute: 100 burst_size: 20 # DHT node configuration dht: enabled: true port: 6883 bootstrap_nodes: - "router.bittorrent.com:6881" - "dht.transmissionbt.com:6881" - "router.utorrent.com:6881" - "dht.libtorrent.org:25401" announce_interval: 900s # 15 minutes max_torrents: 10000 max_peers_per_torrent: 200 # Shared storage configuration storage: blob_threshold: 104857600 # 100MB in bytes chunk_size: 2097152 # 2MB chunks for large files metadata_db: "./data/metadata.db" blob_storage: "./data/blobs" chunk_storage: "./data/chunks" # Built-in BitTorrent tracker tracker: enabled: true announce_interval: 1800 # 30 minutes min_interval: 900 # 15 minutes default_numwant: 50 # peers to return max_numwant: 100 # maximum peers # Nostr relay configuration nostr: relays: # NIP-35 torrent announcements - "wss://freelay.sovbit.host" - "wss://relay.damus.io" - "wss://nos.lol" - "wss://relay.nostr.band" video_relays: # NIP-71 video events (can be different) - "wss://relay.damus.io" - "wss://nos.lol" - "wss://freelay.sovbit.host" publish_nip35: true # Enable NIP-35 torrent events publish_nip71: true # Enable NIP-71 video events auto_publish: true # Auto-publish on upload private_key: "" # Hex private key (auto-generate if empty) # Video transcoding configuration transcoding: enabled: true # Enable/disable transcoding (requires ffmpeg) concurrent_jobs: 2 # Number of parallel transcoding jobs work_dir: "./data/transcoded" # Directory for transcoded files auto_transcode: true # Automatically transcode uploaded videos min_file_size: 50MB # Don't transcode files smaller than this # Admin configuration admin: enabled: true pubkeys: - "your_admin_pubkey_here" # Replace with actual admin pubkey auto_cleanup: true # Enable automated cleanup cleanup_age: "90d" # Delete files older than 90 days max_file_age: "365d" # Maximum file age before forced deletion default_user_storage_limit: "10GB" # Rate limiting configuration rate_limiting: upload: requests_per_second: 1.0 # Max uploads per second per IP burst_size: 5 # Burst allowance max_file_size: "3GB" # Maximum file size download: requests_per_second: 50.0 # Global download rate limit burst_size: 100 # Global burst allowance stream: requests_per_second: 10.0 # Max streams per second per file burst_size: 20 # Stream burst allowance max_concurrent: 50 # Max concurrent streams auth: login_attempts_per_minute: 10 # Login attempts per IP per minute burst_size: 5 # Login burst allowance ``` ## API Reference ### Authentication All endpoints support Nostr-based authentication via: - **NIP-07**: Browser extension (Alby, nos2x) - **NIP-46**: Remote signer/bunker URL ```bash # Get challenge curl http://localhost:9877/api/auth/challenge # Login (requires signed Nostr event) curl -X POST http://localhost:9877/api/auth/login \ -H "Content-Type: application/json" \ -d '{"auth_type": "nip07", "auth_event": "..."}' ``` ### File Operations ```bash # Upload file curl -X POST http://localhost:9877/api/upload \ -F "file=@example.mp4" \ -F "announce_dht=true" # Download file curl http://localhost:9877/api/download/[hash] -o downloaded_file # Get torrent curl http://localhost:9877/api/torrent/[hash] -o file.torrent # Stream video (HLS) curl http://localhost:9877/api/stream/[hash]/playlist.m3u8 # Check transcoding status (requires auth) curl http://localhost:9877/api/users/me/files/[hash]/transcoding-status \ -H "Authorization: Bearer [session_token]" ``` ### User Management ```bash # Get user stats (requires auth) curl http://localhost:9877/api/users/me/stats \ -H "Authorization: Bearer [session_token]" # List user files curl http://localhost:9877/api/users/me/files \ -H "Authorization: Bearer [session_token]" # Delete file curl -X DELETE http://localhost:9877/api/users/me/files/[hash] \ -H "Authorization: Bearer [session_token]" ``` ## Nostr Integration The BitTorrent Gateway seamlessly integrates with the Nostr network to enable decentralized content discovery and social features: ### How Nostr Integration Works 1. **Content Announcements**: When files are uploaded, they're automatically announced to configured Nostr relays as structured events 2. **Decentralized Discovery**: Other nodes and users can discover new content by subscribing to these announcement events 3. **Social Layer**: Users can comment, react, and share content using standard Nostr clients 4. **No Central Authority**: Content discovery happens through the distributed Nostr relay network ### Technical Implementation - **Event Type**: Uses kind `2003` per NIP-35 specification for torrent announcements - **Content Addressing**: Files are identified by cryptographic hashes (SHA-256) - **Metadata**: File size, name, magnet links, and WebSeed URLs included in events - **Relay Redundancy**: Announces to multiple relays for reliability ### Nostr Event Structure When a file is uploaded, the gateway creates a Nostr event like this: ```json { "kind": 2003, "content": "New torrent: example-video.mp4", "tags": [ ["title", "Example Video"], ["x", "d1c5a0f85a4e4f3d8e7d8f9c6b5a4e3f2d1c0b9a8e7d6f5c4b3a2e1d0c9b8a7f6"], ["file", "example-video.mp4", "157286400"], ["magnet", "magnet:?xt=urn:btih:..."], ["webseed", "https://gateway.example.com/api/webseed/..."], ["blossom", "sha256_blob_hash"], ["stream", "https://gateway.example.com/api/stream/hash"], ["hls", "https://gateway.example.com/api/stream/hash/playlist.m3u8"], ["transcoded", "https://gateway.example.com/api/stream/hash?transcoded=true"], ["duration", "3600"], ["video", "1920x1080", "30fps", "h264"], ["m", "video/mp4"], ["t", "torrent"], ["t", "video"], ["t", "streaming"], ["tcat", "video,streaming"] ] } ``` ### Benefits - **Censorship Resistance**: No single point of control for content discovery - **Network Effects**: Content spreads naturally through the Nostr social graph - **Interoperability**: Standard Nostr clients can display and interact with content - **Privacy**: Users maintain control over their identity and data ## Performance & Scaling ### Optimization Features - **Concurrent Downloads**: Multiple parallel piece downloads - **Geographic Peer Selection**: Prioritizes nearby peers for faster transfers - **Smart Caching**: LRU eviction with configurable cache sizes - **Rate Limiting**: Prevents abuse while maintaining performance - **Connection Pooling**: Efficient resource utilization ### Monitoring & Alerting - **Component Health Scores**: 0-100 scoring for all P2P components - **Performance Metrics**: Response times, throughput, error rates - **Automatic Alerts**: Configurable thresholds for degraded performance - **Diagnostic Endpoints**: Detailed system introspection ## Contributing 1. Fork the repository 2. Create a feature branch 3. Make your changes with comprehensive tests 4. Submit a pull request ## License [Add your license information here] ## Support - **Issues**: Report bugs and feature requests via GitHub issues - **Documentation**: Additional documentation in `/docs` - **Community**: [Add community links if available]