A comprehensive unified content distribution system that seamlessly integrates BitTorrent protocol, WebSeed technology, DHT peer discovery, built-in tracker, video transcoding, Blossom blob storage, and multi-relay Nostr announcements. This gateway provides intelligent content distribution by automatically selecting the optimal delivery method based on file size and network conditions, with automatic video transcoding, cleanup automation, and performance optimizations.

Architecture Overview

The BitTorrent Gateway operates as a unified system with multiple specialized components working together:

Core Components

1. Gateway HTTP API Server (Port 9877)

Main web interface and API endpoints
File upload/download management
Smart proxy for reassembling chunked content
WebSeed implementation with advanced LRU caching
Rate limiting and abuse prevention

2. Embedded Blossom Server (Port 8082)

Nostr-compatible blob storage protocol
Direct blob storage for small files (<100MB)
Integration with gateway for seamless operation

3. DHT Node (Port 6883)

Distributed peer discovery
BitTorrent DHT protocol implementation
Bootstrap connectivity with major DHT networks
Automatic torrent announcement and peer sharing

4. Built-in BitTorrent Tracker

Full BitTorrent tracker implementation
Announce/scrape protocol support
P2P coordination and peer ranking
Client compatibility optimizations for qBittorrent, Transmission, WebTorrent, Deluge, uTorrent

5. Video Transcoding Engine

Automatic H.264/AAC conversion for web compatibility
Smart serving: transcoded versions when ready, originals otherwise
Background processing with priority queuing
FFmpeg integration with progress tracking
Multiple quality profiles (1080p, 720p, 480p) with configurable bitrates

6. Automated Cleanup System

Intelligent file lifecycle management
Automatic removal of old files (configurable age: 90d default)
Orphaned chunk detection and cleanup
Inactive user cleanup (180 days default)
Smart proxy cache auto-cleanup (10-minute intervals)
Database optimization with SQLite pragma tuning

7. Multi-Relay Nostr Integration

NIP-35 compliant torrent announcements
NIP-71 video event publishing (horizontal/vertical detection)
WebTorrent extensions with WebSocket tracker support
Blossom hash cross-referencing
Configurable relay sets for different content types
Fault-tolerant publishing with success/failure tracking

Smart Storage Strategy

The system uses an intelligent dual-storage approach with video optimization:

Small Files (<100MB): Stored directly as blobs using Blossom protocol
Large Files (≥100MB): Automatically chunked into 2MB pieces, stored as torrents with WebSeed fallback
Video Files: Automatically queued for H.264/AAC transcoding to web-compatible MP4 format
Smart Serving: Transcoded versions served when ready, original chunks as fallback

P2P Coordination System

A sophisticated P2P coordinator manages all networking components:

Unified Peer Discovery: Aggregates peers from tracker, DHT, and WebSeed sources
Smart Peer Ranking: Geographic proximity and performance-based peer selection
Load Balancing: Distributes load across multiple peer sources
Health Monitoring: Real-time monitoring of all P2P components with automatic alerting

Installation

Prerequisites

Go 1.21 or later
SQLite3
FFmpeg (for video transcoding, optional)
10MB+ available storage
Linux/macOS/Windows

Quick Start (Standalone)

# Clone repository
git clone https://git.sovbit.dev/enki/torrentGateway.git
cd torrentGateway

# Build the gateway
go build -o gateway ./cmd/gateway

# Run with default configuration
./gateway

The web interface will be available at http://localhost:9877

Production Deployment

For production deployment, use the automated installation script:

# Basic installation (HTTP only)
sudo ./scripts/install_native.sh

# Full installation with SSL/HTTPS
sudo ./scripts/install_native.sh --domain gateway.example.com --email admin@example.com

# With monitoring stack
sudo ./scripts/install_native.sh --domain gateway.example.com --email admin@example.com --with-monitoring

This will automatically:

Install all dependencies (Go, nginx, Redis, FFmpeg, etc.)
Build and deploy the application
Configure systemd service
Set up nginx reverse proxy
Configure SSL certificates (if domain provided)
Start all services

See INSTALL.md for detailed installation instructions and domain setup.

Manual Installation (Development)

For development or manual setup, you can run the application directly:

# Build the application
go build -o bin/gateway cmd/gateway/main.go

# Create basic directories
mkdir -p data/{blobs,chunks}

# Run with default config
./bin/gateway

# Or with custom config
./bin/gateway -config configs/config.yaml

For production deployments, use the automated installation script instead.

Configuration

The default configuration is in configs/config.yaml. Customize settings there:

# Unified Blossom-BitTorrent Gateway Configuration
mode: unified  # unified (all services), gateway-only, blossom-only, dht-only

# Gateway HTTP API server
gateway:
  enabled: true
  port: 9877
  max_upload_size: 10GB

# Embedded Blossom server
blossom_server:
  enabled: true
  port: 8082
  max_blob_size: 100MB
  rate_limit:
    requests_per_minute: 100
    burst_size: 20

# DHT node configuration
dht:
  enabled: true
  port: 6883
  bootstrap_nodes:
    - "router.bittorrent.com:6881"
    - "dht.transmissionbt.com:6881"
    - "router.utorrent.com:6881"
    - "dht.libtorrent.org:25401"
  announce_interval: 900s  # 15 minutes
  max_torrents: 10000
  max_peers_per_torrent: 200

# Shared storage configuration
storage:
  blob_threshold: 104857600    # 100MB in bytes
  chunk_size: 2097152          # 2MB chunks for large files  
  metadata_db: "./data/metadata.db"
  blob_storage: "./data/blobs"
  chunk_storage: "./data/chunks"

# Built-in BitTorrent tracker
tracker:
  enabled: true
  announce_interval: 1800      # 30 minutes
  min_interval: 900            # 15 minutes  
  default_numwant: 50          # peers to return
  max_numwant: 100             # maximum peers

# Nostr relay configuration
nostr:
  relays:                          # NIP-35 torrent announcements
    - "wss://freelay.sovbit.host"
    - "wss://relay.damus.io"
    - "wss://nos.lol"
    - "wss://relay.nostr.band"
  video_relays:                    # NIP-71 video events (can be different)
    - "wss://relay.damus.io"
    - "wss://nos.lol"
    - "wss://freelay.sovbit.host"
  publish_nip35: true             # Enable NIP-35 torrent events
  publish_nip71: true             # Enable NIP-71 video events
  auto_publish: true              # Auto-publish on upload
  private_key: ""                 # Hex private key (auto-generate if empty)

# Video transcoding configuration
transcoding:
  enabled: true                # Enable/disable transcoding (requires ffmpeg)
  concurrent_jobs: 2           # Number of parallel transcoding jobs
  work_dir: "./data/transcoded" # Directory for transcoded files
  auto_transcode: true         # Automatically transcode uploaded videos
  min_file_size: 50MB          # Don't transcode files smaller than this

# Admin configuration
admin:
  enabled: true
  pubkeys:
    - "your_admin_pubkey_here"  # Replace with actual admin pubkey
  auto_cleanup: true              # Enable automated cleanup
  cleanup_age: "90d"              # Delete files older than 90 days
  max_file_age: "365d"            # Maximum file age before forced deletion
  default_user_storage_limit: "10GB"

# Rate limiting configuration
rate_limiting:
  upload:
    requests_per_second: 1.0    # Max uploads per second per IP
    burst_size: 5               # Burst allowance
    max_file_size: "3GB"        # Maximum file size
  download:
    requests_per_second: 50.0   # Global download rate limit
    burst_size: 100             # Global burst allowance
  stream:
    requests_per_second: 10.0   # Max streams per second per file
    burst_size: 20              # Stream burst allowance
    max_concurrent: 50          # Max concurrent streams
  auth:
    login_attempts_per_minute: 10  # Login attempts per IP per minute
    burst_size: 5                  # Login burst allowance

API Reference

Authentication

All endpoints support Nostr-based authentication via:

NIP-07: Browser extension (Alby, nos2x)
NIP-46: Remote signer/bunker URL

# Get challenge
curl http://localhost:9877/api/auth/challenge

# Login (requires signed Nostr event)
curl -X POST http://localhost:9877/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"auth_type": "nip07", "auth_event": "..."}'

File Operations

# Upload file
curl -X POST http://localhost:9877/api/upload \
  -F "file=@example.mp4" \
  -F "announce_dht=true"

# Download file
curl http://localhost:9877/api/download/[hash] -o downloaded_file

# Get torrent
curl http://localhost:9877/api/torrent/[hash] -o file.torrent

# Stream video (HLS)
curl http://localhost:9877/api/stream/[hash]/playlist.m3u8

# Check transcoding status (requires auth)
curl http://localhost:9877/api/users/me/files/[hash]/transcoding-status \
  -H "Authorization: Bearer [session_token]"

User Management

# Get user stats (requires auth)
curl http://localhost:9877/api/users/me/stats \
  -H "Authorization: Bearer [session_token]"

# List user files
curl http://localhost:9877/api/users/me/files \
  -H "Authorization: Bearer [session_token]"

# Delete file
curl -X DELETE http://localhost:9877/api/users/me/files/[hash] \
  -H "Authorization: Bearer [session_token]"

Nostr Integration

The BitTorrent Gateway seamlessly integrates with the Nostr network to enable decentralized content discovery and social features:

How Nostr Integration Works

Content Announcements: When files are uploaded, they're automatically announced to configured Nostr relays as structured events
Decentralized Discovery: Other nodes and users can discover new content by subscribing to these announcement events
Social Layer: Users can comment, react, and share content using standard Nostr clients
No Central Authority: Content discovery happens through the distributed Nostr relay network

Technical Implementation

Event Type: Uses kind 2003 per NIP-35 specification for torrent announcements
Content Addressing: Files are identified by cryptographic hashes (SHA-256)
Metadata: File size, name, magnet links, and WebSeed URLs included in events
Relay Redundancy: Announces to multiple relays for reliability

Nostr Event Structure

When a file is uploaded, the gateway creates a Nostr event like this:

{
  "kind": 2003,
  "content": "New torrent: example-video.mp4",
  "tags": [
    ["title", "Example Video"],
    ["x", "d1c5a0f85a4e4f3d8e7d8f9c6b5a4e3f2d1c0b9a8e7d6f5c4b3a2e1d0c9b8a7f6"],
    ["file", "example-video.mp4", "157286400"],
    ["magnet", "magnet:?xt=urn:btih:..."],
    ["webseed", "https://gateway.example.com/api/webseed/..."],
    ["blossom", "sha256_blob_hash"],
    ["stream", "https://gateway.example.com/api/stream/hash"],
    ["hls", "https://gateway.example.com/api/stream/hash/playlist.m3u8"],
    ["transcoded", "https://gateway.example.com/api/stream/hash?transcoded=true"],
    ["duration", "3600"],
    ["video", "1920x1080", "30fps", "h264"],
    ["m", "video/mp4"],
    ["t", "torrent"],
    ["t", "video"],
    ["t", "streaming"],
    ["tcat", "video,streaming"]
  ]
}

Benefits

Censorship Resistance: No single point of control for content discovery
Network Effects: Content spreads naturally through the Nostr social graph
Interoperability: Standard Nostr clients can display and interact with content
Privacy: Users maintain control over their identity and data

Performance & Scaling

Optimization Features

Concurrent Downloads: Multiple parallel piece downloads
Geographic Peer Selection: Prioritizes nearby peers for faster transfers
Smart Caching: LRU eviction with configurable cache sizes
Rate Limiting: Prevents abuse while maintaining performance
Connection Pooling: Efficient resource utilization

Monitoring & Alerting

Component Health Scores: 0-100 scoring for all P2P components
Performance Metrics: Response times, throughput, error rates
Automatic Alerts: Configurable thresholds for degraded performance
Diagnostic Endpoints: Detailed system introspection

Contributing

Fork the repository
Create a feature branch
Make your changes with comprehensive tests
Submit a pull request

License

[Add your license information here]

Support

Issues: Report bugs and feature requests via GitHub issues
Documentation: Additional documentation in /docs
Community: [Add community links if available]

Description

A comprehensive unified content distribution system that seamlessly integrates BitTorrent protocol, WebSeed technology, DHT peer discovery, built-in tracker, and Nostr announcements. This gateway provides intelligent content distribution by automatically selecting the optimal delivery method based on file size and network conditions.

Readme 102 MiB

Languages

Go 68.7%

HTML 14.9%

Shell 7.7%

JavaScript 5.1%

CSS 3.6%