Update README.md

README.md

@@ -1,17 +1,19 @@
 # Strfry Nostr Spam Filter
 
+A sophisticated spam filter for Strfry Nostr relays with reputation tracking, burst handling, and adaptive rate limiting.
 
 ## Features
 
-- Reputation-based filtering system
-- Per-user rate limiting
+- Reputation-based filtering system with detailed logging
+- Intelligent burst handling for follow lists
+- Per-user and kind-specific rate limiting
 - Content similarity detection
 - New account protection
 - Bot behavior detection
 - Progressive penalty system
 - Automatic recovery mechanism
-- Kind-specific rate limits
 - Configurable thresholds
+- Memory-efficient cleanup system
 
 ## Installation
 
@@ -26,7 +28,7 @@ chmod +x relay-spam-filter.js
 ```
 
 3. Configure your strfry relay to use the filter:
-```bash
+```yaml
 # Add to your strfry configuration
 plugins = [
   {
@@ -37,126 +39,139 @@ plugins = [
 
 ## Configuration
 
-The filter includes many configurable options. Here are the key settings:
+### Rate Limits and Burst Handling
 
-### Rate Limits
 ```javascript
-maxRepliesPerMinute: 50,        // Maximum replies per minute per user
-maxRepliesPerHour: 400,         // Maximum replies per hour per user
-```
+// General rate limits
+maxRepliesPerMinute: 50,
+maxRepliesPerHour: 400,
 
-### Event-Type Specific Limits
-```javascript
+// Kind-specific limits with burst handling
 kindSpecificLimits: {
-    0: { maxPerHour: 10 },      // Profile updates
-    3: { maxPerHour: 5 },       // Contact list updates
-    1: {                        // Regular posts
+    0: { // Profile updates
+        maxPerHour: 10,
+        minTimeBetween: 300
+    },
+    1: { // Regular posts
         maxPerMinute: 30,
         maxPerHour: 200
+    },
+    3: { // Follow lists
+        maxPerMinute: 120,
+        maxPerHour: 360,
+        minTimeBetween: 2,
+        ignoreSimilarity: true,
+        burstAllowance: {
+            maxBurst: 500,       // Maximum events in burst
+            burstWindow: 300,    // Window for burst (5 minutes)
+            cooldownPeriod: 3600 // Cooldown after burst (1 hour)
+        }
+    },
+    7: { // Reactions
+        maxPerMinute: 60,
+        maxPerHour: 300,
+        ignoreSimilarity: true
     }
 }
 ```
 
 ### Reputation System
+
 ```javascript
 reputationConfig: {
-    initialScore: 100,          // Starting reputation for new users
-    goodEventBonus: 1,          // Points gained for good events
-    spamPenalty: -15,           // Base penalty for spam
-    recoveryRate: 3,            // Points recovered per hour
-    minScore: -100,             // Minimum possible reputation
-    maxScore: 1000,             // Maximum possible reputation
-    blockThreshold: -50,        // Users blocked at this score
-    blockRecoveryThreshold: -25 // Must recover to this to post again
+    initialScore: 100,          // Starting reputation
+    goodEventBonus: 1,          // Points for good events
+    spamPenalty: -15,          // Base spam penalty
+    recoveryRate: 3,           // Points/hour recovery
+    minScore: -100,           // Minimum reputation
+    maxScore: 1000,           // Maximum reputation
+    blockThreshold: -50,      // Blocking threshold
+    blockRecoveryThreshold: -25, // Recovery threshold
+    
+    // Progressive penalties
+    penaltyScaling: {
+        0: -15,    // Above 0 reputation
+        "-25": -20, // Below -25
+        "-50": -25  // Below -50
+    },
+    
+    // Variable recovery rates
+    recoveryScaling: {
+        0: 3,      // Above 0
+        "-25": 2,  // Below -25
+        "-50": 1   // Below -50
+    }
 }
 ```
 
 ### New User Protection
-```javascript
-newPubkeyReplyThreshold: 60,    // Seconds new users must wait to reply
-newPubkeyMaxPostsIn5Min: 10     // Maximum posts in first 5 minutes
-```
 
-### Content Analysis
 ```javascript
-contentSimilarityThreshold: 0.8, // 80% similarity triggers spam detection
-fastReplyThreshold: 30,          // Minimum seconds between reply and original post
+newPubkeyReplyThreshold: 60,    // Seconds before first reply
+newPubkeyMaxPostsIn5Min: 10     // Initial rate limit
 ```
 
 ## How It Works
 
-### Reputation System
+### Enhanced Reputation System
 
-The filter maintains a reputation score for each user:
+The filter maintains a detailed reputation score for each user:
 
-1. New users start at 100 points
-2. Good behavior slowly increases reputation
-3. Spam behavior results in penalties:
-   - -15 points above 0 reputation
-   - -20 points below -25 reputation
-   - -25 points below -50 reputation
-4. Recovery rates:
-   - 3 points/hour above 0
-   - 2 points/hour between -25 and 0
-   - 1 point/hour below -50
-5. Users are blocked at -50 until they recover to -25
+- New users start at 100 points
+- Good behavior increases reputation gradually
+- Spam behavior triggers scaled penalties based on current reputation
+- Recovery rates vary by reputation level
+- Detailed logging of all reputation changes
+- Special handling for follow list operations
+
+### Burst Handling
+
+Special consideration for bulk operations:
+
+- Follow list updates (kind 3) can operate in burst mode
+- Up to 500 follows in a 5-minute window
+- Cooldown period after burst completion
+- Normal rate limits apply during cooldown
+- Reputation affects burst allowances
 
 ### Spam Detection
 
-The filter checks for several types of spam behavior:
+Multiple detection mechanisms:
 
-1. Rate Limiting
-   - Monitors post frequency per user
-   - Different limits for different event types
-   - Limits scale with reputation
+- Adaptive Rate Limiting
+  - Per-user and per-kind limits
+  - Reputation-based scaling
+  - Burst mode for specific operations
 
-2. Content Analysis
-   - Checks for duplicate content across users
-   - Detects suspiciously fast replies
-   - Identifies bot-like behavior patterns
+- Content Analysis
+  - Cross-user content similarity
+  - Fast reply detection
+  - Bot pattern recognition
 
-3. New Account Protection
-   - Stricter limits for new accounts
-   - Longer waiting periods for replies
-   - Limited initial posting rate
+- New Account Protection
+  - Progressive rate limits
+  - Reply waiting periods
+  - Initial posting restrictions
 
-4. Bot Detection
-   - Identifies automated posting patterns
-   - Checks for relay URL spam
-   - Monitors reply timing patterns
+### Logging and Monitoring
 
-### Progressive Enforcement
+Comprehensive logging system:
 
-The system becomes progressively stricter with problematic behavior:
-
-1. Initial warnings and small penalties
-2. Increased penalties for continued violations
-3. Stricter rate limits for low-reputation users
-4. Eventual blocking for persistent offenders
-
-## Bypass Rules
-
-Certain event kinds can bypass specific checks:
-
-```javascript
-allowedKinds: [3, 5, 10001, 10002, 30311],  // Bypass content checks
-bypassAllChecksKinds: [38383]                // Bypass all checks
 ```
-
-## Monitoring
-
-The filter provides detailed logging:
-
-```javascript
 [2024-12-19T10:15:30.123Z] Event abc123 from pubkey xyz789 (reputation: 85.50)
-[2024-12-19T10:15:30.124Z] Rate limit exceeded for pubkey xyz789
+[2024-12-19T10:15:30.124Z] Reputation update: 85.50 -> 86.50 (change: +1.00)
+[2024-12-19T10:15:30.125Z] Burst tracking - count: 42/500 (window: 300s remaining)
 ```
 
-## Performance Considerations
+### Memory Management
 
-- Memory usage is managed through periodic cleanup
-- Old events and stats are automatically purged
-- Reputation data persists for 24 hours of inactivity
+Efficient resource usage:
+
+- Periodic cleanup of old data
+- Configurable retention periods
+- Automatic purging of inactive users
+- Memory-efficient data structures
+- Burst tracking cleanup
 
 ## Contributing