Merge pull request #23 from asmogo/domain

Added .nostr domains
This commit is contained in:
asmogo 2024-07-28 20:23:16 +02:00 committed by GitHub
commit db232f8e16
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 573 additions and 46 deletions

View File

@ -1,32 +1,37 @@
# Nostr Web Services (NWS) # Nostr Web Services (NWS)
NWS replaces the IP layer in TCP transport using Nostr, enabling a secure connection between NWS replaces the IP layer in TCP transport using Nostr, enabling a secure connection between
clients and backend services. clients and backend services.
Exit nodes are reachable through their [nprofiles](https://nostr-nips.com/nip-19), which are combinations of a Nostr public key and multiple relays. Exit node [domain names](#nws-domain-names) make private services accessible to entry nodes.
### Prerequisites ### Prerequisites
- A list of Nostr relays that the exit node is connected to. - A list of Nostr relays that the exit node is connected to.
- The Nostr private key of the exit node. - The Nostr private key of the exit node.
The exit node utilizes the private key and relay list to generate an [nprofile](https://nostr-nips.com/nip-19), which is printed in the console on startup.
## Overview ## Overview
### NWS main components ### NWS main components
1. **Entry node**: It forwards tcp packets to the exit node using a SOCKS proxy and creates encrypted events for the public key of the exit node. 1. **Exit node**: It is a TCP reverse proxy that listens for incoming Nostr subscriptions and forwards the payload to your designated backend service.
2. **Exit node**: It is a TCP reverse proxy that listens for incoming Nostr subscriptions and forwards the payload to the designated backend service. 2. **Entry node**: It forwards tcp packets to the exit node using a SOCKS proxy and creates encrypted events for the exit node.
<img src="nws.png" width="900"/> <img src="nws.png" width="900"/>
### NWS domain names
There are two types of domain names resolved by NWS entry nodes:
1. `.nostr` domains have base32 encoded public key hostnames and base32 encoded relays as subdomains.
2. [nprofiles](https://nostr-nips.com/nip-19) are combinations of a Nostr public key and multiple relays.
Both types of domains will be generated and printed in the console on startup
## Quickstart ## Quickstart
Running NWS using Docker is recommended. For instructions on running NWS on your local machine, refer to the [Build from source](#build-from-source) section. Running NWS using Docker is recommended. For instructions on running NWS on your local machine, refer to the [Build from source](#build-from-source) section.
### Using Docker Compose ### Using Docker-Compose
Please navigate to the `docker-compose.yaml` file and set `NOSTR_PRIVATE_KEY` to your own private key. Please navigate to the `docker-compose.yaml` file and set `NOSTR_PRIVATE_KEY` to your own private key.
Leaving it empty will generate a new private key on startup. Leaving it empty will generate a new private key on startup.
@ -43,27 +48,28 @@ This will start an example environment, including:
* [Cashu Nutshell](https://github.com/cashubtc/nutshell) (backend service) * [Cashu Nutshell](https://github.com/cashubtc/nutshell) (backend service)
* [nostr-relay](https://github.com/scsibug/nostr-rs-relay) * [nostr-relay](https://github.com/scsibug/nostr-rs-relay)
You can run the following commands to receive your nprofiles: You can run the following commands to receive your NWS domain:
```bash ```bash
docker logs exit-https 2>&1 | awk -F'profile=' '{if ($2) print $2}' | awk '{print $1}' docker logs exit-https 2>&1 | awk -F'domain=' '{if ($2) print $2}' | awk '{print $1}'
``` ```
```bash ```bash
docker logs exit 2>&1 | awk -F'profile=' '{if ($2) print $2}' | awk '{print $1}` docker logs exit 2>&1 | awk -F'domain=' '{if ($2) print $2}' | awk '{print $1}`
``` ```
### Sending Requests to the Entry node ### Sending requests to the entry node
With the log information from the previous step, you can use the following command to send a request to the nprofile: With the log information from the previous step, you can use the following command to send a request to the exit node domain:
``` ```
curl -v -x socks5h://localhost:8882 http://"$(docker logs exit 2>&1 | awk -F'profile=' '{if ($2) print $2}' | awk '{print $1}' | tail -n 1)"/v1/info --insecure curl -v -x socks5h://localhost:8882 http://"$(docker logs exit 2>&1 | awk -F'domain=' '{if ($2) print $2}' | awk '{print $1}' | tail -n 1)"/v1/info --insecure
``` ```
If the nprofile supports TLS, you can choose to connect using https scheme If the exit node supports TLS, you can choose to connect using https scheme
``` ```
curl -v -x socks5h://localhost:8882 https://"$(docker logs exit-https 2>&1 | awk -F'profile=' '{if ($2) print $2}' | awk '{print $1}' | tail -n 1)"/v1/info --insecure curl -v -x socks5h://localhost:8882 https://"$(docker logs exit-https 2>&1 | awk -F'domain=' '{if ($2) print $2}' | awk '{print $1}' | tail -n 1)"/v1/info --insecure
``` ```
When using https, the entry node can be used as a service, since the operator will not be able to see the request data. When using https, the entry node can be used as a service, since the operator will not be able to see the request data.
@ -72,7 +78,7 @@ When using https, the entry node can be used as a service, since the operator wi
The exit node must be set up to make your services reachable via Nostr. The exit node must be set up to make your services reachable via Nostr.
### Exit node Configuration ### Exit node
Configuration should be completed using environment variables. Configuration should be completed using environment variables.
Alternatively, you can create a `.env` file in the current working directory with the following content: Alternatively, you can create a `.env` file in the current working directory with the following content:
@ -97,7 +103,7 @@ If your backend services support TLS, your service can now start using TLS encry
--- ---
### Entry node Configuration ### Entry node
To run an entry node for accessing NWS services behind exit nodes, use the following command: To run an entry node for accessing NWS services behind exit nodes, use the following command:
``` ```

View File

@ -6,15 +6,15 @@ import (
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
var httpsPort int32
var httpTarget string
const ( const (
usagePort = "set the https reverse proxy port" usagePort = "set the https reverse proxy port"
usageTarget = "set https reverse proxy target (your local service)" usageTarget = "set https reverse proxy target (your local service)"
) )
func main() { func main() {
var httpsPort int32
var httpTarget string
rootCmd := &cobra.Command{Use: "exit", Run: startExitNode} rootCmd := &cobra.Command{Use: "exit", Run: startExitNode}
rootCmd.Flags().Int32VarP(&httpsPort, "port", "p", 0, usagePort) rootCmd.Flags().Int32VarP(&httpsPort, "port", "p", 0, usagePort)
rootCmd.Flags().StringVarP(&httpTarget, "target", "t", "", usageTarget) rootCmd.Flags().StringVarP(&httpTarget, "target", "t", "", usageTarget)
@ -25,9 +25,19 @@ func main() {
} }
// updateConfigFlag updates the configuration with the provided flags. // updateConfigFlag updates the configuration with the provided flags.
func updateConfigFlag(cfg *config.ExitConfig) { func updateConfigFlag(cmd *cobra.Command, cfg *config.ExitConfig) error {
httpsPort, err := cmd.Flags().GetInt32("port")
if err != nil {
return err
}
httpTarget, err := cmd.Flags().GetString("target")
if err != nil {
return err
}
cfg.HttpsPort = httpsPort cfg.HttpsPort = httpsPort
cfg.HttpsTarget = httpTarget cfg.HttpsTarget = httpTarget
return nil
} }
func startExitNode(cmd *cobra.Command, args []string) { func startExitNode(cmd *cobra.Command, args []string) {
@ -37,7 +47,7 @@ func startExitNode(cmd *cobra.Command, args []string) {
if err != nil { if err != nil {
panic(err) panic(err)
} }
updateConfigFlag(cfg) updateConfigFlag(cmd, cfg)
ctx := cmd.Context() ctx := cmd.Context()
exitNode := exit.NewExit(ctx, cfg) exitNode := exit.NewExit(ctx, cfg)
exitNode.ListenAndServe(ctx) exitNode.ListenAndServe(ctx)

View File

@ -1,19 +1,24 @@
package exit package exit
import ( import (
"crypto/tls" "encoding/base32"
"encoding/hex"
"fmt" "fmt"
"log/slog"
"net"
"strings"
"github.com/asmogo/nws/config" "github.com/asmogo/nws/config"
"github.com/asmogo/nws/netstr" "github.com/asmogo/nws/netstr"
"github.com/asmogo/nws/protocol" "github.com/asmogo/nws/protocol"
"github.com/asmogo/nws/socks5" "github.com/asmogo/nws/socks5"
"github.com/btcsuite/btcd/btcec/v2"
"github.com/btcsuite/btcd/btcec/v2/schnorr"
"github.com/nbd-wtf/go-nostr" "github.com/nbd-wtf/go-nostr"
"github.com/nbd-wtf/go-nostr/nip04" "github.com/nbd-wtf/go-nostr/nip04"
"github.com/nbd-wtf/go-nostr/nip19" "github.com/nbd-wtf/go-nostr/nip19"
"github.com/puzpuzpuz/xsync/v3" "github.com/puzpuzpuz/xsync/v3"
"golang.org/x/net/context" "golang.org/x/net/context"
"log/slog"
"net"
) )
const ( const (
@ -100,9 +105,13 @@ func NewExit(ctx context.Context, exitNodeConfig *config.ExitConfig) *Exit {
} }
exit.relays = append(exit.relays, relay) exit.relays = append(exit.relays, relay)
fmt.Printf("added relay connection to %s\n", relayUrl) fmt.Printf("added relay connection to %s\n", relayUrl)
}
slog.Info("created exit node", "profile", profile) }
domain, err := exit.getDomain()
if err != nil {
panic(err)
}
slog.Info("created exit node", "profile", profile, "domain", domain)
// setup subscriptions // setup subscriptions
err = exit.setSubscriptions(ctx) err = exit.setSubscriptions(ctx)
if err != nil { if err != nil {
@ -111,6 +120,51 @@ func NewExit(ctx context.Context, exitNodeConfig *config.ExitConfig) *Exit {
return exit return exit
} }
// getDomain returns the domain string used by the Exit node for communication with the Nostr relays.
// It concatenates the relay URLs using base32 encoding with no padding, separated by dots.
// The resulting domain is then appended with the base32 encoded public key obtained using the configured Nostr private key.
// The final domain string is converted to lowercase and returned.
// If any errors occur during the process, they are returned along with an
func (e *Exit) getDomain() (string, error) {
var domain string
// first lets build the subdomains
for _, relayUrl := range e.config.NostrRelays {
if domain == "" {
domain = base32.HexEncoding.WithPadding(base32.NoPadding).EncodeToString([]byte(relayUrl))
} else {
domain = fmt.Sprintf("%s.%s", domain, base32.HexEncoding.WithPadding(base32.NoPadding).EncodeToString([]byte(relayUrl)))
}
}
// create base32 encoded public key
decoded, err := GetPublicKeyBase32(e.config.NostrPrivateKey)
if err != nil {
return "", err
}
// use public key as host. add TLD
domain = strings.ToLower(fmt.Sprintf("%s.%s.nostr", domain, decoded))
return domain, nil
}
// GetPublicKeyBase32 decodes the private key string from hexadecimal format
// and returns the base32 encoded public key obtained using the provided private key.
// The base32 encoding has no padding. If there is an error decoding the private key
// or generating the public key, an error is returned.
//
// Parameters:
// - sk: The private key string in hexadecimal format
//
// Returns:
// - The base32 encoded public key as a string
// - Any error that occurred during the process
func GetPublicKeyBase32(sk string) (string, error) {
b, err := hex.DecodeString(sk)
if err != nil {
return "", err
}
_, pk := btcec.PrivKeyFromBytes(b)
return base32.HexEncoding.WithPadding(base32.NoPadding).EncodeToString(schnorr.SerializePubKey(pk)), nil
}
// setSubscriptions sets up subscriptions for the Exit node to receive incoming events from the specified relays. // setSubscriptions sets up subscriptions for the Exit node to receive incoming events from the specified relays.
// It first obtains the public key using the configured Nostr private key. // It first obtains the public key using the configured Nostr private key.
// Then it calls the `handleSubscription` method to open a subscription to the relays with the specified filters. // Then it calls the `handleSubscription` method to open a subscription to the relays with the specified filters.
@ -140,7 +194,8 @@ func (e *Exit) handleSubscription(ctx context.Context, pubKey string, since nost
Since: &since, Since: &since,
Tags: nostr.TagMap{ Tags: nostr.TagMap{
"p": []string{pubKey}, "p": []string{pubKey},
}}, },
},
}) })
e.incomingChannel = incomingEventChannel e.incomingChannel = incomingEventChannel
return nil return nil
@ -182,9 +237,9 @@ func (e *Exit) processMessage(ctx context.Context, msg nostr.IncomingEvent) {
} }
switch protocolMessage.Type { switch protocolMessage.Type {
case protocol.MessageConnect: case protocol.MessageConnect:
e.handleConnect(ctx, msg, protocolMessage, false) e.handleConnect(ctx, msg, protocolMessage)
case protocol.MessageConnectReverse: case protocol.MessageConnectReverse:
e.handleConnectReverse(ctx, protocolMessage, false) e.handleConnectReverse(protocolMessage)
case protocol.MessageTypeSocks5: case protocol.MessageTypeSocks5:
e.handleSocks5ProxyMessage(msg, protocolMessage) e.handleSocks5ProxyMessage(msg, protocolMessage)
} }
@ -197,7 +252,10 @@ func (e *Exit) processMessage(ctx context.Context, msg nostr.IncomingEvent) {
// If the connection cannot be established, it logs an error and returns. // If the connection cannot be established, it logs an error and returns.
// It then stores the connection in the nostrConnectionMap and creates two goroutines // It then stores the connection in the nostrConnectionMap and creates two goroutines
// to proxy the data between the connection and the backend. // to proxy the data between the connection and the backend.
func (e *Exit) handleConnect(ctx context.Context, msg nostr.IncomingEvent, protocolMessage *protocol.Message, isTLS bool) { func (e *Exit) handleConnect(
ctx context.Context,
msg nostr.IncomingEvent,
protocolMessage *protocol.Message) {
e.mutexMap.Lock(protocolMessage.Key.String()) e.mutexMap.Lock(protocolMessage.Key.String())
defer e.mutexMap.Unlock(protocolMessage.Key.String()) defer e.mutexMap.Unlock(protocolMessage.Key.String())
receiver, err := nip19.EncodeProfile(msg.PubKey, []string{msg.Relay.String()}) receiver, err := nip19.EncodeProfile(msg.PubKey, []string{msg.Relay.String()})
@ -211,12 +269,7 @@ func (e *Exit) handleConnect(ctx context.Context, msg nostr.IncomingEvent, proto
netstr.WithUUID(protocolMessage.Key), netstr.WithUUID(protocolMessage.Key),
) )
var dst net.Conn var dst net.Conn
if isTLS { dst, err = net.Dial("tcp", e.config.BackendHost)
conf := tls.Config{InsecureSkipVerify: true}
dst, err = tls.Dial("tcp", e.config.BackendHost, &conf)
} else {
dst, err = net.Dial("tcp", e.config.BackendHost)
}
if err != nil { if err != nil {
slog.Error("could not connect to backend", "error", err) slog.Error("could not connect to backend", "error", err)
return return
@ -228,7 +281,7 @@ func (e *Exit) handleConnect(ctx context.Context, msg nostr.IncomingEvent, proto
go socks5.Proxy(connection, dst, nil) go socks5.Proxy(connection, dst, nil)
} }
func (e *Exit) handleConnectReverse(ctx context.Context, protocolMessage *protocol.Message, isTLS bool) { func (e *Exit) handleConnectReverse(protocolMessage *protocol.Message) {
e.mutexMap.Lock(protocolMessage.Key.String()) e.mutexMap.Lock(protocolMessage.Key.String())
defer e.mutexMap.Unlock(protocolMessage.Key.String()) defer e.mutexMap.Unlock(protocolMessage.Key.String())
connection, err := net.Dial("tcp", protocolMessage.Destination) connection, err := net.Dial("tcp", protocolMessage.Destination)
@ -236,12 +289,7 @@ func (e *Exit) handleConnectReverse(ctx context.Context, protocolMessage *protoc
return return
} }
var dst net.Conn var dst net.Conn
if isTLS { dst, err = net.Dial("tcp", e.config.BackendHost)
conf := tls.Config{InsecureSkipVerify: true}
dst, err = tls.Dial("tcp", e.config.BackendHost, &conf)
} else {
dst, err = net.Dial("tcp", e.config.BackendHost)
}
if err != nil { if err != nil {
slog.Error("could not connect to backend", "error", err) slog.Error("could not connect to backend", "error", err)
return return

View File

@ -3,9 +3,12 @@ package netstr
import ( import (
"bytes" "bytes"
"context" "context"
"encoding/base32"
"encoding/base64" "encoding/base64"
"encoding/hex"
"fmt" "fmt"
"github.com/asmogo/nws/protocol" "github.com/asmogo/nws/protocol"
"github.com/btcsuite/btcd/btcec/v2/schnorr"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/nbd-wtf/go-nostr" "github.com/nbd-wtf/go-nostr"
"github.com/nbd-wtf/go-nostr/nip04" "github.com/nbd-wtf/go-nostr/nip04"
@ -13,6 +16,7 @@ import (
"github.com/samber/lo" "github.com/samber/lo"
"log/slog" "log/slog"
"net" "net"
"strings"
"time" "time"
) )
@ -216,6 +220,10 @@ func (nc *NostrConnection) handleNostrWrite(b []byte, err error) (int, error) {
// If the prefix is "nprofile", the public key and relays are extracted. // If the prefix is "nprofile", the public key and relays are extracted.
// Returns the public key, relays (if any), and any error encountered. // Returns the public key, relays (if any), and any error encountered.
func ParseDestination(destination string) (string, []string, error) { func ParseDestination(destination string) (string, []string, error) {
// check if destination ends with .nostr
if strings.HasSuffix(destination, ".nostr") {
return ParseDestinationDomain(destination)
}
// destination can be npub or nprofile // destination can be npub or nprofile
prefix, pubKey, err := nip19.Decode(destination) prefix, pubKey, err := nip19.Decode(destination)
@ -237,6 +245,38 @@ func ParseDestination(destination string) (string, []string, error) {
return publicKey, relays, nil return publicKey, relays, nil
} }
func ParseDestinationDomain(destination string) (string, []string, error) {
url, err := protocol.Parse(destination)
if err != nil {
return "", nil, err
}
if !url.IsDomain {
// return "", nil, fmt.Errorf("destination is not a domain")
}
var subdomains []string
split := strings.Split(url.SubName, ".")
for _, subdomain := range split {
decodedSubDomain, err := base32.HexEncoding.WithPadding(base32.NoPadding).DecodeString(strings.ToUpper(subdomain))
if err != nil {
continue
}
subdomains = append(subdomains, string(decodedSubDomain))
}
// base32 decode the subdomain
decodedPubKey, err := base32.HexEncoding.WithPadding(base32.NoPadding).DecodeString(strings.ToUpper(url.Name))
if err != nil {
return "", nil, err
}
pk, err := schnorr.ParsePubKey(decodedPubKey)
if err != nil {
return "", nil, err
}
// todo -- check if this is correct
return hex.EncodeToString(pk.SerializeCompressed())[2:], subdomains, nil
}
func (nc *NostrConnection) Close() error { func (nc *NostrConnection) Close() error {
nc.cancel() nc.cancel()
return nil return nil

View File

@ -8,7 +8,6 @@ import (
"github.com/nbd-wtf/go-nostr" "github.com/nbd-wtf/go-nostr"
"log/slog" "log/slog"
"net" "net"
"strings"
) )
type DialOptions struct { type DialOptions struct {
@ -26,7 +25,6 @@ type DialOptions struct {
// Finally, it returns the Connection and nil error. If there are any errors, nil connection and the error are returned. // Finally, it returns the Connection and nil error. If there are any errors, nil connection and the error are returned.
func DialSocks(options DialOptions) func(ctx context.Context, net_, addr string) (net.Conn, error) { func DialSocks(options DialOptions) func(ctx context.Context, net_, addr string) (net.Conn, error) {
return func(ctx context.Context, net_, addr string) (net.Conn, error) { return func(ctx context.Context, net_, addr string) (net.Conn, error) {
addr = strings.ReplaceAll(addr, ".", "")
key := nostr.GeneratePrivateKey() key := nostr.GeneratePrivateKey()
connection := NewConnection(ctx, connection := NewConnection(ctx,
WithPrivateKey(key), WithPrivateKey(key),

380
protocol/domain.go Normal file
View File

@ -0,0 +1,380 @@
package protocol
import (
"errors"
"fmt"
"net"
"net/url"
"strings"
"golang.org/x/net/idna"
"golang.org/x/net/publicsuffix"
)
const (
ipV6URINotationPrefix = "["
ipV6URINotationSuffix = "]"
)
var ErrEmptyURL = errors.New("url to be parsed is empty")
// URL represents a URL with additional fields and methods.
type URL struct {
SubName, Name, TLD, Port string
IsDomain bool
*url.URL
}
// String returns the string representation of the URL.
// It includes the scheme if `includeScheme` is true.
func (url URL) String(includeScheme bool) string {
s := url.URL.String()
if !includeScheme {
s = RemoveScheme(s)
}
return s
}
// Domain returns the domain name of the URL. If includeSub is true and there is a subdomain, it includes the subdomain
// in the returned string. Otherwise, it only includes the domain.
func (url URL) Domain(includeSub bool) string {
if includeSub && url.SubName != "" {
return fmt.Sprintf("%s.%s.%s", url.SubName, url.Name, url.TLD)
}
return fmt.Sprintf("%s.%s", url.Name, url.TLD)
}
// NoWWW returns the domain name without the "www" subdomain.
// If the subdomain is not "www" or is empty, it returns the domain name as is.
// The returned domain name is a string in the format "subname.name.tld".
func (url URL) NoWWW() string {
if url.SubName != "www" && url.SubName != "" {
return fmt.Sprintf("%s.%s.%s", url.SubName, url.Name, url.TLD)
}
return fmt.Sprintf("%s.%s", url.Name, url.TLD)
}
// WWW returns the domain name with the "www" subdomain.
// If the subdomain is not "www", it returns the domain name as is.
// The returned domain name is a string in the format "subname.name.tld".
func (url URL) WWW() string {
if url.SubName != "" {
return fmt.Sprintf("%s.%s.%s", url.SubName, url.Name, url.TLD)
}
return fmt.Sprintf("%s.%s.%s", "www", url.Name, url.TLD)
}
// HTTPS returns the URL with HTTPS Scheme but leaves the URL itself untouched.
func (url URL) HTTPS() string {
rememberScheme := url.Scheme
url.Scheme = "https"
httpsURL := url.String(true)
url.Scheme = rememberScheme
return httpsURL
}
// StripWWW returns the URL without "www" subdomain, but leaves the URL itself untouched.
// This function returns the whole URL with its path, in contrast to NoWWW().
func (url URL) StripWWW(includeScheme bool) string {
if url.SubName == "www" {
return strings.Replace(url.String(includeScheme), "www.", "", 1)
}
return url.String(includeScheme)
}
// StripQueryParams removes query parameters and fragments from the URL and returns
// the URL as a string. If includeScheme is true, it includes the scheme in the returned URL.
func (url URL) StripQueryParams(includeScheme bool) string {
// Remember the original values of query parameters and fragments
rememberRawQuery := url.RawQuery
rememberFragment := url.Fragment
rememberRawFragment := url.RawFragment
// Clear the query parameters and fragments
url.RawQuery = ""
url.RawFragment = ""
url.Fragment = ""
// Get the URL without query parameters
urlWithoutQuery := url.String(includeScheme)
// Restore the original values of query parameters and fragments
url.RawQuery = rememberRawQuery
url.RawFragment = rememberRawFragment
url.Fragment = rememberFragment
return urlWithoutQuery
}
// IsLocal checks if the URL is a local address.
// It returns true if the URL's top-level domain (TLD) is "localhost" or if the URL's
// hostname resolves to a loopback IP address.
func (url URL) IsLocal() bool {
ip := net.ParseIP(strings.TrimPrefix(strings.TrimSuffix(url.Name, ipV6URINotationSuffix), ipV6URINotationPrefix))
return url.TLD == "localhost" || (ip != nil && ip.IsLoopback())
}
// Parse parses a string representation of a URL and returns a *URL and error.
// It mirrors the net/url.Parse function but returns a tld.URL, which contains extra fields.
func Parse(urlString string) (*URL, error) {
urlString = strings.TrimSpace(urlString)
// if the url to be parsed is empty after trimming, we return an error
if len(urlString) == 0 {
return nil, ErrEmptyURL
}
urlString = AddDefaultScheme(urlString)
parsedURL, err := url.Parse(urlString)
if err != nil {
return nil, fmt.Errorf("could not parse url: %w", err)
}
// always lowercase subdomain.domain.tld (host property)
parsedURL.Host = strings.ToLower(parsedURL.Host)
if parsedURL.Host == "" {
return &URL{URL: parsedURL}, nil
}
dom, port := domainPort(parsedURL.Host)
var domName, tld, sub string
ip := net.ParseIP(strings.TrimPrefix(strings.TrimSuffix(dom, ipV6URINotationSuffix), ipV6URINotationPrefix))
switch {
case ip != nil:
domName = dom
case dom == "localhost":
tld = dom
default:
etld1, err := publicsuffix.EffectiveTLDPlusOne(dom)
if err != nil {
return nil, fmt.Errorf("failed to extract eTLD+1: %w", err)
}
i := strings.Index(etld1, ".")
domName = etld1[0:i]
tld = etld1[i+1:]
sub = ""
if rest := strings.TrimSuffix(dom, "."+etld1); rest != dom {
sub = rest
}
}
urlString, err = idna.ToASCII(dom)
if err != nil {
return nil, fmt.Errorf("failed to convert domain to ASCII: %w", err)
}
return &URL{
SubName: sub,
Name: domName,
TLD: tld,
Port: port,
URL: parsedURL,
IsDomain: IsDomainName(urlString),
}, nil
}
// FromParsed mirrors the net/url.Parse function,
// but instead of returning a *url.URL, it returns a *URL,
// which is a struct that contains additional fields.
//
// The function first checks if the parsedUrl.Host field is empty.
// If it is empty, it returns a *URL with the URL field set to parsedUrl
// and all other fields set to their zero values.
//
// If the parsedUrl.Host field is not empty, it extracts the domain and port
// using the domainPort function.
//
// It then calculates the effective top-level domain plus one (etld+1)
// using the publicsuffix.EffectiveTLDPlusOne function.
//
// The etld+1 is then split into the domain name (domName) and the top-level domain (tld).
//
// It further determines the subdomain (sub) by checking if the domain is a subdomain of the etld+1.
//
// The domain name (domName) is then converted to ASCII using the idna.ToASCII function.
//
// Finally, it returns a *URL with the extracted values and the URL field set to parsedUrl.
// The IsDomain field is set to the result of the IsDomainName function called with the ASCII domain name.
// The SubName field is set to sub, the Name field is set to domName, and the T.
func FromParsed(parsedURL *url.URL) (*URL, error) {
if parsedURL.Host == "" {
return &URL{URL: parsedURL}, nil
}
dom, port := domainPort(parsedURL.Host)
// etld+1
etld1, err := publicsuffix.EffectiveTLDPlusOne(dom)
if err != nil {
return nil, fmt.Errorf("failed to extract eTLD+1: %w", err)
}
// convert to domain name, and tld
i := strings.Index(etld1, ".")
domName := etld1[0:i]
tld := etld1[i+1:]
// and subdomain
sub := ""
if rest := strings.TrimSuffix(dom, "."+etld1); rest != dom {
sub = rest
}
asciiDom, err := idna.ToASCII(dom)
if err != nil {
return nil, fmt.Errorf("failed to convert domain to ASCII: %w", err)
}
return &URL{
SubName: sub,
Name: domName,
TLD: tld,
Port: port,
URL: parsedURL,
IsDomain: IsDomainName(asciiDom),
}, nil
}
// domainPort extracts the domain and port from the host part of a URL.
// If the host contains a port, it returns the domain without the port and the port as strings.
// If the host does not contain a port, it returns the domain and an empty string for the port.
// If the host is all numeric characters, it returns the host itself and an empty string for the port.
// Note that the net/url package should prevent the string from being all numeric characters.
func domainPort(host string) (string, string) {
for i := len(host) - 1; i >= 0; i-- {
if host[i] == ':' {
return host[:i], host[i+1:]
} else if host[i] < '0' || host[i] > '9' {
return host, ""
}
}
// will only land here if the string is all digits,
// net/url should prevent that from happening
return host, ""
}
// IsDomainName checks if a string represents a valid domain name.
//
// It follows the rules specified in RFC 1035 and RFC 3696 for domain name validation.
//
// The input string is first processed with the RemoveScheme function to remove any scheme prefix.
// The domain name is then split into labels using the dot separator.
// The function checks that the number of labels is at least 2 and that the total length of the string is between 1 and
// 254 characters.
//
// The function iterates over the characters of the string and performs checks based on the character type.
// Valid characters include letters (a-zA-Z), digits (0-9), underscore (_), and hyphen (-).
// Each label can contain up to 63 characters and the last label cannot end with a hyphen.
// The function also checks that the byte before a dot or a hyphen is not a dot or a hyphen, respectively.
// Non-numeric characters are tracked to ensure the presence of at least one non-numeric character in the domain name.
//
// If any of the checks fail, the function returns false. Otherwise, it returns true.
//
// Example usage:
// s := "mail.google.com"
// isValid := IsDomainName(s).
func IsDomainName(name string) bool { //nolint:cyclop
name = RemoveScheme(name)
// See RFC 1035, RFC 3696.
// Presentation format has dots before every label except the first, and the
// terminal empty label is optional here because we assume fully-qualified
// (absolute) input. We must therefore reserve space for the first and last
// labels' length octets in wire format, where they are necessary and the
// maximum total length is 255.
// So our _effective_ maximum is 253, but 254 is not rejected if the last
// character is a dot.
split := strings.Split(name, ".")
// Need a TLD and a domain.
if len(split) < 2 { //nolint:gomnd
return false
}
l := len(name)
if l == 0 || l > 254 || l == 254 && name[l-1] != '.' {
return false
}
last := byte('.')
nonNumeric := false // true once we've seen a letter or hyphen
partlen := 0
for i := 0; i < len(name); i++ {
char := name[i]
switch {
default:
return false
case 'a' <= char && char <= 'z' || 'A' <= char && char <= 'Z' || char == '_':
nonNumeric = true
partlen++
case '0' <= char && char <= '9':
// fine
partlen++
case char == '-':
// Byte before dash cannot be dot.
if last == '.' {
return false
}
partlen++
nonNumeric = true
case char == '.':
// Byte before dot cannot be dot, dash.
if last == '.' || last == '-' {
return false
}
if partlen > 63 || partlen == 0 {
return false
}
partlen = 0
}
last = char
}
if last == '-' || partlen > 63 {
return false
}
return nonNumeric
}
// RemoveScheme removes the scheme from a URL string.
// If the URL string includes a scheme (e.g., "http://"), the scheme will be removed and the remaining string will be returned.
// If the URL string includes a default scheme (e.g., "//"), the default scheme will be removed and the remaining string will be returned.
// If the URL string does not include a scheme, the original string will be returned unchanged.
func RemoveScheme(s string) string {
if strings.Contains(s, "://") {
return removeScheme(s)
}
if strings.Contains(s, "//") {
return removeDefaultScheme(s)
}
return s
}
// add default scheme if string does not include a scheme.
func AddDefaultScheme(s string) string {
if !strings.Contains(s, "//") ||
(!strings.Contains(s, "//") && !strings.Contains(s, ":") && !strings.Contains(s, "@")) {
return addDefaultScheme(s)
}
return s
}
func AddScheme(s, scheme string) string {
if scheme == "" {
return AddDefaultScheme(s)
}
if strings.Index(s, "//") == -1 {
return fmt.Sprintf("%s://%s", scheme, s)
}
return s
}
// addDefaultScheme returns a new string with a default scheme added.
// The default scheme format is "//<original_string>".
func addDefaultScheme(s string) string {
return fmt.Sprintf("//%s", s)
}
// removeDefaultScheme removes the default scheme from a string.
func removeDefaultScheme(s string) string {
return s[index(s, "//"):]
}
func removeScheme(s string) string {
return s[index(s, "://"):]
}
// index returns the starting index of the first occurrence of the specified scheme in the given string.
// If the scheme is not found, it returns -1.
// The returned index is incremented by the length of the scheme to obtain the starting position of the remaining string.
func index(s, scheme string) int {
return strings.Index(s, scheme) + len(scheme)
}

45
protocol/domain_test.go Normal file
View File

@ -0,0 +1,45 @@
package protocol
import (
"net/url"
"reflect"
"testing"
)
type args struct {
s string
}
type parseTest struct {
name string
args args
want *URL
wantErr bool
}
func TestParse(t *testing.T) {
t.Parallel()
for _, test := range createParseTests() {
testCopy := test
t.Run(testCopy.name, func(t *testing.T) {
t.Parallel()
got, err := Parse(testCopy.args.s)
if (err != nil) != testCopy.wantErr {
t.Errorf("Parse() error = %v, wantErr %v", err, testCopy.wantErr)
return
}
if !reflect.DeepEqual(got, testCopy.want) {
t.Errorf("Parse() got = %v, want %v", got, testCopy.want)
}
})
}
}
func createParseTests() []parseTest {
return []parseTest{
{name: "1", args: args{s: "http://D1Q78S3J78NIURJFEDQ74BJQCLH6AP35CKN66R3FELI0.9B7NTQSU4PBM2JJQJ0CMGHUENQON4GB28RLGQCH3D3NK2AQVFE70.nostr"}, want: &URL{IsDomain: true, TLD: "nostr", Name: "9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70", SubName: "d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0", URL: &url.URL{Host: "d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0.9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70.nostr", Scheme: "http"}}, wantErr: false}, //nolint:lll
{name: "1", args: args{s: "http://d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0.9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70.nostr"}, want: &URL{IsDomain: true, TLD: "nostr", Name: "9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70", SubName: "d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0", URL: &url.URL{Host: "d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0.9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70.nostr", Scheme: "http"}}, wantErr: false}, //nolint:lll
{name: "1", args: args{s: "https://d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0.9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70.nostr"}, want: &URL{IsDomain: true, TLD: "nostr", Name: "9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70", SubName: "d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0", URL: &url.URL{Host: "d1q78s3j78niurjfedq74bjqclh6ap35ckn66r3feli0.9b7ntqsu4pbm2jjqj0cmghuenqon4gb28rlgqch3d3nk2aqvfe70.nostr", Scheme: "https"}}, wantErr: false}, //nolint:lll
}
}