diff --git a/README.md b/README.md index 33daf7f..82b2075 100644 --- a/README.md +++ b/README.md @@ -364,7 +364,7 @@ The BitTorrent Gateway seamlessly integrates with the Nostr network to enable de ### Technical Implementation -- **Event Type**: Uses kind `1063` for file/torrent announcements +- **Event Type**: Uses kind `2003` per NIP-35 specification for torrent announcements - **Content Addressing**: Files are identified by cryptographic hashes (SHA-256) - **Metadata**: File size, name, magnet links, and WebSeed URLs included in events - **Relay Redundancy**: Announces to multiple relays for reliability @@ -375,13 +375,24 @@ When a file is uploaded, the gateway creates a Nostr event like this: ```json { - "kind": 1063, - "content": "New torrent: example-file.zip", + "kind": 2003, + "content": "New torrent: example-video.mp4", "tags": [ + ["title", "Example Video"], + ["x", "d1c5a0f85a4e4f3d8e7d8f9c6b5a4e3f2d1c0b9a8e7d6f5c4b3a2e1d0c9b8a7f6"], + ["file", "example-video.mp4", "157286400"], ["magnet", "magnet:?xt=urn:btih:..."], - ["size", "104857600"], - ["name", "example-file.zip"], - ["webseed", "https://gateway.example.com/api/webseed/..."] + ["webseed", "https://gateway.example.com/api/webseed/..."], + ["blossom", "sha256_blob_hash"], + ["stream", "https://gateway.example.com/api/stream/hash"], + ["hls", "https://gateway.example.com/api/stream/hash/playlist.m3u8"], + ["duration", "3600"], + ["video", "1920x1080", "30fps", "h264"], + ["m", "video/mp4"], + ["t", "torrent"], + ["t", "video"], + ["t", "streaming"], + ["tcat", "video,streaming"] ] } ``` diff --git a/TECHNICAL_OVERVIEW.md b/TECHNICAL_OVERVIEW.md index 2d6fcdb..833442f 100644 --- a/TECHNICAL_OVERVIEW.md +++ b/TECHNICAL_OVERVIEW.md @@ -339,7 +339,7 @@ When files are uploaded, they're announced to configured Nostr relays: ```go func (g *Gateway) announceToNostr(fileInfo *FileInfo, torrentInfo *TorrentInfo) error { event := nostr.Event{ - Kind: 1063, // Custom torrent announcement kind + Kind: 2003, // NIP-35 torrent announcement kind Content: fmt.Sprintf("New torrent: %s", fileInfo.Filename), CreatedAt: time.Now(), Tags: []nostr.Tag{ diff --git a/docs/STANDARDS_PROPOSAL_GUIDE.md b/docs/STANDARDS_PROPOSAL_GUIDE.md new file mode 100644 index 0000000..e266012 --- /dev/null +++ b/docs/STANDARDS_PROPOSAL_GUIDE.md @@ -0,0 +1,446 @@ +# Standards Proposal Guide + +This document walks you through submitting standards proposals to standardize the BitTorrent Gateway's innovative streaming and hybrid protocol features. + +## 🎯 Overview + +The BitTorrent Gateway implements pioneering features that deserve standardization: + +1. **NIP-35 Streaming Extensions** - Adds streaming metadata to Nostr torrent announcements +2. **Blossom BUD-09** - Streaming support for the Blossom protocol +3. **Hybrid Protocol Specification** - BitTorrent + Blossom integration patterns + +## 📋 Required Pull Requests + +### 1. NIP-35 Streaming Extensions + +**Repository**: https://github.com/nostr-protocol/nips +**Target File**: `35.md` +**Type**: Enhancement to existing NIP + +#### What to Propose + +Add streaming-specific tags to NIP-35 to support video/audio streaming announcements: + +**New Tags**: +- `stream`: Direct streaming URL +- `hls`: HLS playlist URL +- `duration`: Content duration in seconds +- `video`: Resolution, framerate, codec info +- `m`: MIME type (following NIP-94 convention) + +#### Proposed Addition to NIP-35 + +```markdown +## Streaming Extensions + +For content that supports streaming, the following additional tags MAY be included: + +- `stream` - Direct streaming URL for progressive download/streaming +- `hls` - HLS playlist URL (.m3u8) for adaptive streaming +- `duration` - Duration in seconds for time-based media +- `video` - Video metadata as ``, ``, `` +- `m` - MIME type (following NIP-94) + +### Example with Streaming + +```json +{ + "kind": 2003, + "content": "High-quality nature documentary", + "tags": [ + ["title", "Nature Documentary"], + ["x", "d1c5a0f85a4e4f3d8e7d8f9c6b5a4e3f2d1c0b9a8e7d6f5c4b3a2e1d0c9b8a7f6"], + ["file", "nature-doc.mp4", "2147483648"], + ["magnet", "magnet:?xt=urn:btih:..."], + ["webseed", "https://example.com/api/webseed/hash"], + ["stream", "https://example.com/api/stream/hash"], + ["hls", "https://example.com/api/stream/hash/playlist.m3u8"], + ["duration", "3600"], + ["video", "1920x1080", "30fps", "h264"], + ["m", "video/mp4"], + ["t", "video"], + ["t", "streaming"], + ["tcat", "video,documentary"] + ] +} +``` + +These extensions enable: +- **Stream Discovery**: Users can find streamable content +- **Player Integration**: Video players can directly consume streams +- **Quality Selection**: Multiple formats and qualities can be announced +- **Social Streaming**: Comments and reactions on streaming content +``` + +### 2. Blossom BUD-11: Streaming Support + +**Repository**: https://github.com/hzrd149/blossom +**Target Directory**: `buds/` +**File**: `buds/11.md` +**Type**: New BUD (Blossom Upgrade Document) + +#### What to Propose + +Create BUD-11 to define streaming endpoints and behavior for Blossom servers: + +**New Endpoints**: +- `GET /{hash}/stream` - Direct blob streaming with range support +- `GET /{hash}/playlist.m3u8` - HLS playlist generation +- `GET /{hash}/segment/{n}` - HLS segment delivery +- `GET /{hash}/info` - Media metadata extraction + +#### Complete BUD-11 Draft + +```markdown +# BUD-11: Streaming Support + +## Abstract + +This BUD defines streaming endpoints for Blossom servers to support direct video/audio streaming and HLS (HTTP Live Streaming) delivery from stored blobs. + +## Motivation + +As Blossom adoption grows for media storage, there's increasing need for streaming capability without requiring separate streaming infrastructure. This BUD enables: + +- Direct blob streaming with HTTP range support +- HLS playlist generation from single blobs +- Adaptive streaming for better user experience +- Integration with existing video players + +## Specification + +### Streaming Endpoint + +``` +GET //stream +``` + +Servers implementing BUD-09 MUST support: +- HTTP range requests (RFC 7233) +- Proper MIME type detection and headers +- CORS headers for web player compatibility + +**Headers**: +- `Accept-Ranges: bytes` +- `Content-Type: ` +- `Content-Length: ` +- `Access-Control-Allow-Origin: *` + +### HLS Playlist Endpoint + +``` +GET //playlist.m3u8 +``` + +For video/audio blobs, servers MAY generate HLS playlists: +- Virtual segmentation of the blob +- Standard M3U8 format compliance +- Segment duration typically 6-10 seconds + +**Example Response**: +``` +#EXTM3U +#EXT-X-VERSION:3 +#EXT-X-TARGETDURATION:7 +#EXT-X-MEDIA-SEQUENCE:0 +#EXT-X-PLAYLIST-TYPE:VOD +#EXTINF:6.000, +/segment/0 +#EXTINF:6.000, +/segment/1 +#EXT-X-ENDLIST +``` + +### HLS Segment Endpoint + +``` +GET //segment/ +``` + +Delivers virtual segments from the blob: +- Segment boundaries calculated from blob size +- Proper byte range extraction +- Video container format preservation + +### Media Info Endpoint + +``` +GET //info +``` + +Returns JSON metadata about media blobs: + +```json +{ + "hash": "sha256...", + "size": 104857600, + "mime_type": "video/mp4", + "duration": 3600, + "video": { + "width": 1920, + "height": 1080, + "codec": "h264", + "fps": "30" + }, + "streaming": { + "stream_url": "//stream", + "hls_url": "//playlist.m3u8" + } +} +``` + +## Implementation Notes + +- Servers SHOULD cache generated playlists for performance +- Range requests MUST be handled efficiently for large blobs +- Video codec compatibility varies by browser +- Consider implementing quality variants for adaptive streaming + +## References + +- RFC 8216: HTTP Live Streaming (HLS) +- RFC 7233: Range Requests +- BUD-01: Authorization +- BUD-02: Upload and List +``` + +### 3. Hybrid Protocol Specification + +**Repository**: Create new repository or add to existing project +**Target**: New specification document +**Type**: Protocol definition + +#### What to Propose + +Document the hybrid BitTorrent + Blossom approach as a reusable pattern: + +**Key Components**: +- Storage threshold rules (when to use Blossom vs BitTorrent) +- WebSeed integration patterns +- Chunk mapping strategies +- Fallback hierarchies +- P2P bootstrapping methods + +## 📝 Step-by-Step Submission Process + +### Step 1: Fork the Repositories + +```bash +# Fork NIP repository +git clone https://github.com/YOUR_USERNAME/nips.git +cd nips + +# Fork Blossom repository +git clone https://github.com/YOUR_USERNAME/blossom.git +cd blossom +``` + +### Step 2: Create Feature Branches + +```bash +# In nips repository +git checkout -b nip-35-streaming-extensions +git push -u origin nip-35-streaming-extensions + +# In blossom repository +git checkout -b bud-11-streaming-support +git push -u origin bud-11-streaming-support +``` + +### Step 3: Make the Changes + +#### NIP-35 Changes + +1. Edit `35.md` +2. Add streaming extensions section (see above) +3. Update examples to include new tags +4. Test with reference implementation + +```bash +cd nips +# Edit 35.md with streaming extensions +git add 35.md +git commit -m "Add streaming extensions to NIP-35 + +- Add stream, hls, duration, video tags for streaming media +- Include comprehensive example with streaming metadata +- Enable social streaming discovery and player integration +- Maintains backward compatibility with existing implementations" +``` + +#### BUD-11 Creation + +1. Create `buds/11.md` +2. Write complete BUD specification (see above) +3. Include implementation examples + +```bash +cd blossom +# Create buds/11.md with streaming specification +git add buds/11.md +git commit -m "Add BUD-11: Streaming Support + +- Define streaming endpoints for direct blob streaming +- Add HLS playlist generation from blobs +- Include media info endpoint for metadata +- Enable integration with standard video players" +``` + +### Step 4: Test Your Changes + +Before submitting, test the proposals: + +1. **Implementation Testing**: Verify the BitTorrent Gateway works with the new tags +2. **Compatibility Testing**: Ensure existing clients aren't broken +3. **Documentation Review**: Check all examples and references + +```bash +# In BitTorrent Gateway repository +go build -o gateway ./cmd/gateway +./gateway # Test with new NIP-35 tags +``` + +### Step 5: Submit Pull Requests + +#### NIP-35 Pull Request + +```bash +cd nips +git push origin nip-35-streaming-extensions +``` + +**PR Title**: `NIP-35: Add streaming extensions for video/audio content` + +**PR Description**: +```markdown +## Summary + +This PR extends NIP-35 to support streaming media by adding optional tags for direct streaming and HLS delivery. + +## Motivation + +Current NIP-35 only supports basic torrent metadata. With the growth of video/audio content on Nostr, there's a need for streaming-specific metadata that enables: + +- Direct video streaming in clients +- HLS playlist discovery +- Media player integration +- Social features around streaming content + +## Changes + +- Added `stream` tag for direct streaming URLs +- Added `hls` tag for HLS playlist URLs +- Added `duration` tag for time-based media +- Added `video` tag for resolution/codec metadata +- Added `m` tag for MIME type (consistent with NIP-94) +- Updated example to demonstrate streaming extensions + +## Implementation + +This extension is already implemented in the BitTorrent Gateway project: +https://github.com/YOUR_USERNAME/torrentGateway + +## Backward Compatibility + +All new tags are optional. Existing NIP-35 implementations continue to work unchanged. +``` + +#### BUD-11 Pull Request + +```bash +cd blossom +git push origin bud-11-streaming-support +``` + +**PR Title**: `BUD-11: Add streaming support for media blobs` + +**PR Description**: +```markdown +## Summary + +This BUD defines streaming endpoints that enable Blossom servers to serve video/audio content directly with support for HTTP range requests and HLS streaming. + +## Motivation + +Blossom is increasingly used for media storage, but lacks streaming capability. This BUD enables: + +- Direct blob streaming without separate infrastructure +- HLS support for adaptive streaming +- Better user experience for video/audio content +- Integration with standard video players + +## Specification + +- `GET //stream` - Direct streaming with range support +- `GET //playlist.m3u8` - HLS playlist generation +- `GET //segment/` - HLS segment delivery +- `GET //info` - Media metadata endpoint + +## Implementation + +Reference implementation available in: +https://github.com/YOUR_USERNAME/torrentGateway + +The implementation demonstrates practical usage and performance characteristics. +``` + +### Step 6: Follow Up and Iterate + +1. **Respond to Feedback**: Address reviewer comments promptly +2. **Update Implementation**: Make changes based on specification feedback +3. **Community Engagement**: Participate in discussions and decisions +4. **Documentation**: Update your implementation as specs evolve + +## 🤝 Community Engagement Tips + +### For NIP-35 (Nostr Protocol) + +- **Join Nostr Development**: Follow discussions on nostr-protocol GitHub +- **Test with Clients**: Verify compatibility with major Nostr clients +- **Use Cases**: Provide compelling examples of streaming use cases +- **Performance Data**: Share performance metrics from your implementation + +### For Blossom BUD-11 + +- **Implementation First**: Having working code makes proposals stronger +- **Server Compatibility**: Test with existing Blossom server implementations +- **Client Integration**: Show how existing clients benefit +- **Resource Usage**: Document server resource requirements + +## 📊 Success Metrics + +Your proposals will be considered successful if they achieve: + +1. **Technical Merit**: Solve real problems elegantly +2. **Backward Compatibility**: Don't break existing implementations +3. **Implementation Proof**: Working code demonstrates feasibility +4. **Community Support**: Other developers express interest +5. **Clear Specification**: Unambiguous implementation guidance + +## 🔗 Useful Resources + +### NIP-35 Resources +- [NIP-35 Current Specification](https://github.com/nostr-protocol/nips/blob/master/35.md) +- [Nostr Implementation Possibilities](https://github.com/nostr-protocol/nips) +- [NIP-35 Pull Request #1175](https://github.com/nostr-protocol/nips/pull/1175) + +### Blossom Resources +- [Blossom Protocol](https://github.com/hzrd149/blossom) +- [Existing BUDs](https://github.com/hzrd149/blossom/tree/master/buds) +- [Reference Server Implementation](https://github.com/hzrd149/blossom-server) + +### Standards Writing +- [RFC 2119: Key words for RFCs](https://tools.ietf.org/html/rfc2119) +- [RFC 8216: HTTP Live Streaming](https://tools.ietf.org/html/rfc8216) +- [RFC 7233: Range Requests](https://tools.ietf.org/html/rfc7233) + +--- + +## 🚀 Ready to Submit? + +With these proposals, you'll be pioneering the standardization of decentralized streaming protocols. The BitTorrent Gateway's innovative approach deserves to become the foundation for next-generation content distribution. + +**Remember**: Standards are adopted through implementation and usage. Your working code is the most compelling argument for adoption! + +Good luck with your standards proposals! 🎉 \ No newline at end of file diff --git a/internal/api/handlers.go b/internal/api/handlers.go index 0869581..07d64f1 100644 --- a/internal/api/handlers.go +++ b/internal/api/handlers.go @@ -494,12 +494,24 @@ func (g *Gateway) handleBlobUpload(w http.ResponseWriter, r *http.Request, file // Publish to Nostr for blobs var nostrEventID string if g.nostrPublisher != nil { + // Detect if this is a video file for streaming metadata + isVideo, mimeType := streaming.DetectMediaType(fileName) + eventData := nostr.TorrentEventData{ Title: fmt.Sprintf("File: %s", fileName), FileName: fileName, FileSize: metadata.Size, BlossomHash: metadata.Hash, Description: fmt.Sprintf("File '%s' (%.2f MB) available via Blossom blob storage", fileName, float64(metadata.Size)/1024/1024), + MimeType: mimeType, + } + + // Add streaming URLs for video files + if isVideo { + baseURL := g.getBaseURL() + eventData.StreamURL = fmt.Sprintf("%s/api/stream/%s", baseURL, metadata.Hash) + eventData.HLSPlaylistURL = fmt.Sprintf("%s/api/stream/%s/playlist.m3u8", baseURL, metadata.Hash) + eventData.Duration = int64(streaming.EstimateVideoDuration(metadata.Size, fileName)) } ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) @@ -704,6 +716,17 @@ func (g *Gateway) handleTorrentUpload(w http.ResponseWriter, r *http.Request, fi WebSeedURL: webSeedURL, BlossomHash: metadata.Hash, Description: fmt.Sprintf("File '%s' (%.2f MB) available via BitTorrent", fileName, float64(metadata.Size)/1024/1024), + MimeType: mimeType, + } + + // Add streaming information for video files + if streamingInfo != nil { + baseURL := g.getBaseURL() + eventData.StreamURL = fmt.Sprintf("%s/api/stream/%s", baseURL, metadata.Hash) + eventData.HLSPlaylistURL = fmt.Sprintf("%s/api/stream/%s/playlist.m3u8", baseURL, metadata.Hash) + eventData.Duration = int64(streamingInfo.Duration) + eventData.VideoCodec = "h264" // Default assumption + eventData.MimeType = streamingInfo.MimeType } ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) @@ -3231,4 +3254,11 @@ func RegisterTrackerRoutes(r *mux.Router, cfg *config.Config, storage *storage.B // GetGatewayFromRoutes returns a gateway instance for DHT integration func GetGatewayFromRoutes(cfg *config.Config, storage *storage.Backend) *Gateway { return NewGateway(cfg, storage) +} + +// getBaseURL returns the base URL for the gateway +func (g *Gateway) getBaseURL() string { + // TODO: This should be configurable or detected from request + // For now, use localhost with the configured port + return fmt.Sprintf("http://localhost:%d", g.config.Gateway.Port) } \ No newline at end of file diff --git a/internal/nostr/publisher.go b/internal/nostr/publisher.go index 2342a0b..216ad90 100644 --- a/internal/nostr/publisher.go +++ b/internal/nostr/publisher.go @@ -6,6 +6,7 @@ import ( "fmt" "log" "strconv" + "strings" "time" "github.com/nbd-wtf/go-nostr" @@ -32,6 +33,15 @@ type TorrentEventData struct { WebSeedURL string BlossomHash string Description string + // Streaming extensions + HLSPlaylistURL string + StreamURL string + Duration int64 // seconds + VideoWidth int + VideoHeight int + FrameRate string + VideoCodec string + MimeType string } // NewPublisher creates a new Nostr publisher @@ -72,7 +82,7 @@ func NewPublisher(privateKeyHex string, relays []string) (*Publisher, error) { }, nil } -// CreateTorrentEvent creates a NIP-35 compliant torrent announcement event +// CreateTorrentEvent creates a NIP-35 compliant torrent announcement event with streaming extensions func (p *Publisher) CreateTorrentEvent(data TorrentEventData) (*nostr.Event, error) { event := &nostr.Event{ Kind: KindTorrent, @@ -86,29 +96,76 @@ func (p *Publisher) CreateTorrentEvent(data TorrentEventData) (*nostr.Event, err event.Tags = event.Tags.AppendUnique(nostr.Tag{"title", data.Title}) } + // Required: BitTorrent info hash (x tag) if data.InfoHash != "" { event.Tags = event.Tags.AppendUnique(nostr.Tag{"x", data.InfoHash}) } + // Required: File information with size (file tag) if data.FileName != "" && data.FileSize > 0 { event.Tags = event.Tags.AppendUnique(nostr.Tag{"file", data.FileName, strconv.FormatInt(data.FileSize, 10)}) } + // Standard NIP-35 tags if data.WebSeedURL != "" { event.Tags = event.Tags.AppendUnique(nostr.Tag{"webseed", data.WebSeedURL}) } - if data.BlossomHash != "" { - event.Tags = event.Tags.AppendUnique(nostr.Tag{"blossom", data.BlossomHash}) - } - if data.MagnetLink != "" { event.Tags = event.Tags.AppendUnique(nostr.Tag{"magnet", data.MagnetLink}) } - // Add some additional useful tags + // Blossom integration (custom extension) + if data.BlossomHash != "" { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"blossom", data.BlossomHash}) + } + + // Streaming extensions (proposed for NIP-35) + if data.StreamURL != "" { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"stream", data.StreamURL}) + } + + if data.HLSPlaylistURL != "" { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"hls", data.HLSPlaylistURL}) + } + + if data.Duration > 0 { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"duration", strconv.FormatInt(data.Duration, 10)}) + } + + if data.VideoWidth > 0 && data.VideoHeight > 0 { + resolution := fmt.Sprintf("%dx%d", data.VideoWidth, data.VideoHeight) + if data.FrameRate != "" && data.VideoCodec != "" { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"video", resolution, data.FrameRate, data.VideoCodec}) + } else if data.FrameRate != "" { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"video", resolution, data.FrameRate}) + } else { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"video", resolution}) + } + } + + if data.MimeType != "" { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"m", data.MimeType}) + } + + // Content categorization tags event.Tags = event.Tags.AppendUnique(nostr.Tag{"t", "torrent"}) event.Tags = event.Tags.AppendUnique(nostr.Tag{"t", "blossom"}) + + // Determine content category based on mime type + if data.MimeType != "" { + if strings.HasPrefix(data.MimeType, "video/") { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"t", "video"}) + if data.StreamURL != "" || data.HLSPlaylistURL != "" { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"t", "streaming"}) + } + // Add video category path + event.Tags = event.Tags.AppendUnique(nostr.Tag{"tcat", "video,streaming"}) + } else if strings.HasPrefix(data.MimeType, "audio/") { + event.Tags = event.Tags.AppendUnique(nostr.Tag{"t", "audio"}) + event.Tags = event.Tags.AppendUnique(nostr.Tag{"tcat", "audio"}) + } + } // Sign the event err := event.Sign(p.privateKey) diff --git a/internal/web/index.html b/internal/web/index.html index 3073972..48ea830 100644 --- a/internal/web/index.html +++ b/internal/web/index.html @@ -316,14 +316,24 @@

Example Nostr Event:

{
-  "kind": 1063,
-  "content": "New file: example-video.mp4",
+  "kind": 2003,
+  "content": "New torrent: example-video.mp4",
   "tags": [
+    ["title", "Example Video"],
+    ["x", "d1c5a0f85a4e4f3d8e7d8f9c6b5a4e3f2d1c0b9a8e7d6f5c4b3a2e1d0c9b8a7f6"],
+    ["file", "example-video.mp4", "157286400"],
     ["magnet", "magnet:?xt=urn:btih:..."],
-    ["size", "157286400"],
-    ["name", "example-video.mp4"],
     ["webseed", "https://gateway.example.com/api/webseed/..."],
-    ["mimetype", "video/mp4"]
+    ["blossom", "sha256_blob_hash"],
+    ["stream", "https://gateway.example.com/api/stream/hash"],
+    ["hls", "https://gateway.example.com/api/stream/hash/playlist.m3u8"],
+    ["duration", "3600"],
+    ["video", "1920x1080", "30fps", "h264"],
+    ["m", "video/mp4"],
+    ["t", "torrent"],
+    ["t", "video"],
+    ["t", "streaming"],
+    ["tcat", "video,streaming"]
   ]
 }