Initial working version

docs/queue.md

@@ -0,0 +1,94 @@
+# Twitch Queue System
+
+This README explains how to use the Huey-based queue system with your Twitch bot.
+
+## Overview
+
+The queue system allows your Twitch bot to process commands and messages asynchronously. This is useful for:
+
+- Handling high-volume chat without performance issues
+- Processing commands that take time to complete
+- Distributing processing across multiple processes or machines
+- Maintaining history of commands for analytics
+
+## Components
+
+The queue system consists of three main components:
+
+1. **Queue Server**: Defined in `src/queue/server.py`, this establishes the Huey queue with SQLite backend
+2. **Consumer**: Processes tasks from the queue (can run in a separate process)
+3. **Producer**: Your Twitch bot that sends tasks to the queue
+
+## Setup
+
+1. Make sure you have Huey installed:
+   ```bash
+   pip install huey
+   ```
+
+2. Enable the queue in your bot by using the `--use-queue` flag:
+   ```bash
+   python main.py --use-queue
+   ```
+
+3. Start a consumer to process tasks (in a separate terminal):
+   ```bash
+   # Option 1: Use the Huey consumer CLI
+   python -m huey.bin.huey_consumer src.queue.server.huey
+
+   # Option 2: Use our simple wrapper
+   python scripts/run_consumer.py
+   
+   # Option 3: Use the bot with built-in consumer (not recommended for production)
+   python main.py --use-queue --start-consumer
+   ```
+
+## How It Works
+
+1. When `--use-queue` is enabled, the bot sends all commands to the queue instead of processing them directly.
+2. Each command becomes a task in the SQLite database.
+3. The consumer process picks up tasks and executes them in the background.
+4. Results can be stored and retrieved later.
+
+## Commands
+
+The queue system adds the following command to the bot:
+
+- `!qstats` - Display statistics about the queue (processed tasks, pending tasks, etc.)
+
+## Scaling
+
+For high-volume applications:
+
+1. Run multiple consumers to process tasks in parallel:
+   ```bash
+   python -m huey.bin.huey_consumer src.queue.server.huey -w 4  # 4 worker processes
+   ```
+
+2. Consider using Redis instead of SQLite for better performance:
+   (Requires modifying `src/queue/server.py` to use RedisHuey instead of SqliteHuey)
+
+## Architecture Diagram
+
+```
+┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
+│                 │     │                 │     │                 │
+│   Twitch Chat   │────▶│    Twitch Bot   │────▶│  Queue (SQLite) │
+│                 │     │    (Producer)   │     │                 │
+└─────────────────┘     └─────────────────┘     └────────┬────────┘
+                                                          │
+                                                          ▼
+                                               ┌─────────────────────┐
+                                               │                     │
+                                               │  Huey Consumer(s)   │
+                                               │                     │
+                                               └─────────────────────┘
+```
+
+## Customization
+
+You can customize the queue system by:
+
+1. Adding more task types in `src/queue/server.py`
+2. Creating custom handlers for processed tasks
+3. Implementing result handling in the bot