enki/torrent-gateway
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

docs and about updates
Some checks are pending
CI Pipeline / Run Tests (push) Waiting to run
Details
CI Pipeline / Lint Code (push) Waiting to run
Details
CI Pipeline / Security Scan (push) Waiting to run
Details
CI Pipeline / Build Docker Images (push) Blocked by required conditions
Details
CI Pipeline / E2E Tests (push) Blocked by required conditions
Details

Browse Source
This commit is contained in:
Enki 2025-08-18 00:50:56 -07:00
parent b3204ea07a
commit ae9557bacf
2 changed files with 330 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

265
README.md
View File

@ -55,8 +55,9 @@ A sophisticated P2P coordinator manages all networking components:
- Go 1.21 or later
- SQLite3
- 10MB+ available storage
- Linux/macOS/Windows
### Quick Start
### Quick Start (Standalone)
```bash
# Clone repository
@ -70,48 +71,222 @@ go build -o gateway ./cmd/gateway
./gateway
```
The web interface will be available at http://localhost:9876
The web interface will be available at http://localhost:9877
### Production Deployment (No Docker)
For production deployment without Docker, use the native installation script:
```bash
# Make installation script executable
chmod +x scripts/install_native.sh
# Install as system service
sudo ./scripts/install_native.sh
# Start the service
sudo systemctl start torrent-gateway
sudo systemctl enable torrent-gateway
# Check status
sudo systemctl status torrent-gateway
```
### Nginx Reverse Proxy Configuration
For production deployments, use Nginx as a reverse proxy:
```nginx
server {
listen 80;
server_name your-domain.com;
# Redirect HTTP to HTTPS
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name your-domain.com;
# SSL Configuration
ssl_certificate /path/to/your/certificate.pem;
ssl_certificate_key /path/to/your/private-key.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off;
# Security Headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
# Gateway API and Web Interface
location / {
proxy_pass http://127.0.0.1:9877;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket support for real-time features
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Increase timeouts for large file uploads
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
# Blossom Server (optional, if running separately)
location /blossom/ {
proxy_pass http://127.0.0.1:8082/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Increase client max body size for file uploads
client_max_body_size 10G;
# Enable gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/atom+xml image/svg+xml;
}
```
### SystemD Service Configuration
The native installer creates a systemd service. Manual configuration:
```bash
# Create service file
sudo tee /etc/systemd/system/torrent-gateway.service > /dev/null <<EOF
[Unit]
Description=Torrent Gateway Service
After=network.target
[Service]
Type=simple
User=torrent-gateway
Group=torrent-gateway
WorkingDirectory=/opt/torrent-gateway
ExecStart=/opt/torrent-gateway/gateway -config /etc/torrent-gateway/config.yaml
Restart=always
RestartSec=3
LimitNOFILE=65536
# Security settings
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/torrent-gateway/data
ReadWritePaths=/var/log/torrent-gateway
[Install]
WantedBy=multi-user.target
EOF
# Create dedicated user
sudo useradd --system --home /opt/torrent-gateway --shell /bin/false torrent-gateway
# Create directories and set permissions
sudo mkdir -p /opt/torrent-gateway/{data/{blobs,chunks},logs}
sudo mkdir -p /etc/torrent-gateway
sudo mkdir -p /var/log/torrent-gateway
sudo chown -R torrent-gateway:torrent-gateway /opt/torrent-gateway
sudo chown -R torrent-gateway:torrent-gateway /var/log/torrent-gateway
# Copy binary and config
sudo cp gateway /opt/torrent-gateway/
sudo cp configs/config.yaml /etc/torrent-gateway/
sudo chmod +x /opt/torrent-gateway/gateway
# Enable and start service
sudo systemctl daemon-reload
sudo systemctl enable torrent-gateway
sudo systemctl start torrent-gateway
```
### Configuration
The default configuration is in `configs/config.yaml`. Customize settings there:
```yaml
# Unified Blossom-BitTorrent Gateway Configuration
mode: unified # unified (all services), gateway-only, blossom-only, dht-only
# Gateway HTTP API server
gateway:
host: "0.0.0.0"
port: 9876
storage_path: "./storage"
blossom:
enabled: true
host: "0.0.0.0"
port: 8081
max_blob_size: 10485760 # 10MB
port: 9877
max_upload_size: 10GB
# Embedded Blossom server
blossom_server:
enabled: true
port: 8082
max_blob_size: 100MB
rate_limit:
requests_per_minute: 100
burst_size: 20
# DHT node configuration
dht:
enabled: true
port: 6882
port: 6883
bootstrap_nodes:
- "router.bittorrent.com:6881"
- "dht.transmissionbt.com:6881"
- "router.utorrent.com:6881"
- "dht.libtorrent.org:25401"
announce_interval: 900s # 15 minutes
max_torrents: 10000
max_peers_per_torrent: 200
database:
path: "./gateway.db"
# Shared storage configuration
storage:
blob_threshold: 104857600 # 100MB in bytes
chunk_size: 2097152 # 2MB chunks for large files
metadata_db: "./data/metadata.db"
blob_storage: "./data/blobs"
chunk_storage: "./data/chunks"
# Built-in BitTorrent tracker
tracker:
enabled: true
announce_interval: 1800 # 30 minutes
min_interval: 900 # 15 minutes
default_numwant: 50 # peers to return
max_numwant: 100 # maximum peers
# Nostr relay configuration
nostr:
relays:
- "wss://relay.damus.io"
- "wss://nos.lol"
- "wss://freelay.sovbit.host"
# Admin configuration
admin:
enabled: false
pubkeys: [] # Add admin Nostr pubkeys here
enabled: true
pubkeys:
- "your_admin_pubkey_here" # Replace with actual admin pubkey
default_user_storage_limit: "10GB"
# Rate limiting configuration
rate_limiting:
upload:
requests_per_second: 1.0 # Max uploads per second per IP
burst_size: 5 # Burst allowance
max_file_size: "100MB" # Maximum file size
max_file_size: "3GB" # Maximum file size
download:
requests_per_second: 50.0 # Global download rate limit
burst_size: 100 # Global burst allowance
@ -134,10 +309,10 @@ All endpoints support Nostr-based authentication via:
```bash
# Get challenge
curl http://localhost:9876/api/auth/challenge
curl http://localhost:9877/api/auth/challenge
# Login (requires signed Nostr event)
curl -X POST http://localhost:9876/api/auth/login \
curl -X POST http://localhost:9877/api/auth/login \
-H "Content-Type: application/json" \
-d '{"auth_type": "nip07", "auth_event": "..."}'
```
@ -146,58 +321,78 @@ curl -X POST http://localhost:9876/api/auth/login \
```bash
# Upload file
curl -X POST http://localhost:9876/api/upload \
curl -X POST http://localhost:9877/api/upload \
-F "file=@example.mp4" \
-F "announce_dht=true"
# Download file
curl http://localhost:9876/api/download/[hash] -o downloaded_file
curl http://localhost:9877/api/download/[hash] -o downloaded_file
# Get torrent
curl http://localhost:9876/api/torrent/[hash] -o file.torrent
curl http://localhost:9877/api/torrent/[hash] -o file.torrent
# Stream video (HLS)
curl http://localhost:9876/api/stream/[hash]/playlist.m3u8
curl http://localhost:9877/api/stream/[hash]/playlist.m3u8
```
### User Management
```bash
# Get user stats (requires auth)
curl http://localhost:9876/api/users/me/stats \
curl http://localhost:9877/api/users/me/stats \
-H "Authorization: Bearer [session_token]"
# List user files
curl http://localhost:9876/api/users/me/files \
curl http://localhost:9877/api/users/me/files \
-H "Authorization: Bearer [session_token]"
# Delete file
curl -X DELETE http://localhost:9876/api/users/me/files/[hash] \
curl -X DELETE http://localhost:9877/api/users/me/files/[hash] \
-H "Authorization: Bearer [session_token]"
```
## Nostr Integration
The system announces new content to configured Nostr relays:
The BitTorrent Gateway seamlessly integrates with the Nostr network to enable decentralized content discovery and social features:
- **Event Type**: Custom torrent announcement events
- **Content**: Torrent magnet links and metadata
- **Discovery**: Enables decentralized content discovery
- **Relay Configuration**: Multiple relays for redundancy
### How Nostr Integration Works
1. **Content Announcements**: When files are uploaded, they're automatically announced to configured Nostr relays as structured events
2. **Decentralized Discovery**: Other nodes and users can discover new content by subscribing to these announcement events
3. **Social Layer**: Users can comment, react, and share content using standard Nostr clients
4. **No Central Authority**: Content discovery happens through the distributed Nostr relay network
### Technical Implementation
- **Event Type**: Uses kind `1063` for file/torrent announcements
- **Content Addressing**: Files are identified by cryptographic hashes (SHA-256)
- **Metadata**: File size, name, magnet links, and WebSeed URLs included in events
- **Relay Redundancy**: Announces to multiple relays for reliability
### Nostr Event Structure
When a file is uploaded, the gateway creates a Nostr event like this:
Example Nostr event:
```json
{
"kind": 1063,
"content": "New torrent available",
"content": "New torrent: example-file.zip",
"tags": [
["magnet", "magnet:?xt=urn:btih:..."],
["size", "104857600"],
["name", "example-file.zip"]
["name", "example-file.zip"],
["webseed", "https://gateway.example.com/api/webseed/..."]
]
}
```
### Benefits
- **Censorship Resistance**: No single point of control for content discovery
- **Network Effects**: Content spreads naturally through the Nostr social graph
- **Interoperability**: Standard Nostr clients can display and interact with content
- **Privacy**: Users maintain control over their identity and data
## Performance & Scaling
### Optimization Features

102
internal/web/index.html
View File

@ -206,13 +206,13 @@
<div id="about-section" class="section active">
<div class="about-header">
<h2>⚡ BitTorrent Gateway</h2>
<p class="about-subtitle">Comprehensive Unified Content Distribution System</p>
<p class="about-subtitle">Decentralized Content Distribution with Nostr Integration</p>
</div>
<div class="about-content">
<div class="intro-section">
<h3>What This Platform Does</h3>
<p>The BitTorrent Gateway is a comprehensive unified content distribution system that seamlessly integrates BitTorrent protocol, WebSeed technology, DHT peer discovery, and Nostr announcements. It provides intelligent content distribution by automatically selecting the optimal delivery method:</p>
<p>The BitTorrent Gateway is a next-generation content distribution system that combines the reliability of traditional web hosting with the power of peer-to-peer networks and decentralized social discovery through Nostr. It automatically chooses the best way to store and distribute your files:</p>
<div class="storage-flow">
<div class="flow-item">
@ -231,6 +231,104 @@
</div>
</div>
</div>
<div class="key-benefits">
<h4>Why This Approach Works</h4>
<div class="benefit-list">
<div class="benefit-item">
<span class="benefit-icon">🚀</span>
<div class="benefit-content">
<strong>Always Available:</strong> Files are instantly accessible via WebSeed, with P2P peers providing additional bandwidth
</div>
</div>
<div class="benefit-item">
<span class="benefit-icon">🌐</span>
<div class="benefit-content">
<strong>Scales Automatically:</strong> Popular files get more peers, reducing server load naturally
</div>
</div>
<div class="benefit-item">
<span class="benefit-icon">🔒</span>
<div class="benefit-content">
<strong>Censorship Resistant:</strong> Content announced on Nostr can't be taken down by any single party
</div>
</div>
<div class="benefit-item">
<span class="benefit-icon">💰</span>
<div class="benefit-content">
<strong>Cost Effective:</strong> P2P distribution reduces bandwidth costs for large files
</div>
</div>
</div>
</div>
</div>
<div class="nostr-integration">
<h3>🌟 How Nostr Integration Works</h3>
<p class="nostr-intro">Nostr (Notes and Other Stuff Transmitted by Relays) is a decentralized protocol that enables censorship-resistant social networking. Here's how we integrate it for content distribution:</p>
<div class="nostr-flow">
<div class="nostr-step">
<div class="step-number">1</div>
<div class="step-content">
<h5>Content Upload</h5>
<p>You upload a file using your Nostr identity (no email or passwords needed)</p>
</div>
</div>
<div class="flow-arrow">↓</div>
<div class="nostr-step">
<div class="step-number">2</div>
<div class="step-content">
<h5>Automatic Processing</h5>
<p>System creates torrents, generates magnet links, and sets up WebSeed URLs</p>
</div>
</div>
<div class="flow-arrow">↓</div>
<div class="nostr-step">
<div class="step-number">3</div>
<div class="step-content">
<h5>Nostr Announcement</h5>
<p>File metadata is broadcast to Nostr relays as a structured event</p>
</div>
</div>
<div class="flow-arrow">↓</div>
<div class="nostr-step">
<div class="step-number">4</div>
<div class="step-content">
<h5>Social Discovery</h5>
<p>Other users can discover, comment on, and share your content through Nostr clients</p>
</div>
</div>
</div>
<div class="nostr-benefits">
<h4>Benefits of Nostr Integration:</h4>
<ul>
<li><strong>No Central Control:</strong> Content discovery happens through the distributed Nostr relay network</li>
<li><strong>Social Features:</strong> Users can comment, react, and share content using any Nostr client</li>
<li><strong>Network Effects:</strong> Content spreads naturally through social connections</li>
<li><strong>Privacy:</strong> You control your identity and data through cryptographic keys</li>
<li><strong>Interoperability:</strong> Works with existing Nostr apps and clients</li>
</ul>
</div>
<div class="nostr-example">
<h4>Example Nostr Event:</h4>
<div class="code-block">
<pre>{
"kind": 1063,
"content": "New file: example-video.mp4",
"tags": [
["magnet", "magnet:?xt=urn:btih:..."],
["size", "157286400"],
["name", "example-video.mp4"],
["webseed", "https://gateway.example.com/api/webseed/..."],
["mimetype", "video/mp4"]
]
}</pre>
</div>
<p class="code-explanation">This event gets distributed across Nostr relays, allowing anyone subscribed to file announcements to discover new content.</p>
</div>
</div>
<div class="architecture-section">