Initial working version
This commit is contained in:
Joey Yakimowich-Payne
commit
ec1c5958ce
33 changed files with 4547 additions and 0 deletions
94
docs/queue.md
Normal file
94
docs/queue.md
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
# Twitch Queue System
|
||||
|
||||
This README explains how to use the Huey-based queue system with your Twitch bot.
|
||||
|
||||
## Overview
|
||||
|
||||
The queue system allows your Twitch bot to process commands and messages asynchronously. This is useful for:
|
||||
|
||||
- Handling high-volume chat without performance issues
|
||||
- Processing commands that take time to complete
|
||||
- Distributing processing across multiple processes or machines
|
||||
- Maintaining history of commands for analytics
|
||||
|
||||
## Components
|
||||
|
||||
The queue system consists of three main components:
|
||||
|
||||
1. **Queue Server**: Defined in `src/queue/server.py`, this establishes the Huey queue with SQLite backend
|
||||
2. **Consumer**: Processes tasks from the queue (can run in a separate process)
|
||||
3. **Producer**: Your Twitch bot that sends tasks to the queue
|
||||
|
||||
## Setup
|
||||
|
||||
1. Make sure you have Huey installed:
|
||||
```bash
|
||||
pip install huey
|
||||
```
|
||||
|
||||
2. Enable the queue in your bot by using the `--use-queue` flag:
|
||||
```bash
|
||||
python main.py --use-queue
|
||||
```
|
||||
|
||||
3. Start a consumer to process tasks (in a separate terminal):
|
||||
```bash
|
||||
# Option 1: Use the Huey consumer CLI
|
||||
python -m huey.bin.huey_consumer src.queue.server.huey
|
||||
|
||||
# Option 2: Use our simple wrapper
|
||||
python scripts/run_consumer.py
|
||||
|
||||
# Option 3: Use the bot with built-in consumer (not recommended for production)
|
||||
python main.py --use-queue --start-consumer
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. When `--use-queue` is enabled, the bot sends all commands to the queue instead of processing them directly.
|
||||
2. Each command becomes a task in the SQLite database.
|
||||
3. The consumer process picks up tasks and executes them in the background.
|
||||
4. Results can be stored and retrieved later.
|
||||
|
||||
## Commands
|
||||
|
||||
The queue system adds the following command to the bot:
|
||||
|
||||
- `!qstats` - Display statistics about the queue (processed tasks, pending tasks, etc.)
|
||||
|
||||
## Scaling
|
||||
|
||||
For high-volume applications:
|
||||
|
||||
1. Run multiple consumers to process tasks in parallel:
|
||||
```bash
|
||||
python -m huey.bin.huey_consumer src.queue.server.huey -w 4 # 4 worker processes
|
||||
```
|
||||
|
||||
2. Consider using Redis instead of SQLite for better performance:
|
||||
(Requires modifying `src/queue/server.py` to use RedisHuey instead of SqliteHuey)
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ │ │ │ │ │
|
||||
│ Twitch Chat │────▶│ Twitch Bot │────▶│ Queue (SQLite) │
|
||||
│ │ │ (Producer) │ │ │
|
||||
└─────────────────┘ └─────────────────┘ └────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ │
|
||||
│ Huey Consumer(s) │
|
||||
│ │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
## Customization
|
||||
|
||||
You can customize the queue system by:
|
||||
|
||||
1. Adding more task types in `src/queue/server.py`
|
||||
2. Creating custom handlers for processed tasks
|
||||
3. Implementing result handling in the bot
|
||||
118
docs/twitch_auth_setup.md
Normal file
118
docs/twitch_auth_setup.md
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
# Twitch Authentication Setup
|
||||
|
||||
This document explains how to set up Twitch authentication for the application.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. You need a Twitch account
|
||||
2. You need to register a Twitch application at [Twitch Developer Console](https://dev.twitch.tv/console/apps)
|
||||
3. Required Python packages:
|
||||
- `cryptography` - For generating SSL certificates
|
||||
- `requests` - For making HTTP requests to the Twitch API
|
||||
- `python-dotenv` - For managing environment variables
|
||||
|
||||
You can install these dependencies using the provided script:
|
||||
|
||||
```bash
|
||||
python scripts/install_auth_deps.py
|
||||
```
|
||||
|
||||
## Registering a Twitch Application
|
||||
|
||||
1. Go to [Twitch Developer Console](https://dev.twitch.tv/console/apps)
|
||||
2. Click "Register Your Application"
|
||||
3. Fill in the details:
|
||||
- **Name**: Choose a name for your application (e.g., "MyStreamInteract")
|
||||
- **OAuth Redirect URLs**: Set to `https://localhost:3000` (note: HTTPS is required)
|
||||
- **Category**: Choose "Chat Bot"
|
||||
4. Click "Create"
|
||||
5. Once created, click "Manage" for your application
|
||||
6. Make note of your **Client ID**
|
||||
7. Generate a **Client Secret** and store it securely
|
||||
|
||||
## Setting Up Authentication
|
||||
|
||||
There are two ways to authenticate:
|
||||
|
||||
### Method 1: Using the Setup Script (Recommended)
|
||||
|
||||
We provide a setup script that walks you through the authentication process:
|
||||
|
||||
```bash
|
||||
python scripts/setup_twitch_auth.py
|
||||
```
|
||||
|
||||
The script will:
|
||||
1. Ask for your Client ID and Client Secret
|
||||
2. Generate a self-signed SSL certificate for HTTPS (required by Twitch)
|
||||
3. Open a browser window for authentication
|
||||
4. Store the tokens securely
|
||||
5. Update your .env file with the credentials
|
||||
|
||||
> **Note:** When the browser opens, you will see a security warning about the self-signed certificate.
|
||||
> This is normal for local development. You need to proceed past this warning to continue the authentication process.
|
||||
> In Chrome, click "Advanced" and then "Proceed to localhost (unsafe)".
|
||||
|
||||
If you want to force the generation of a new token (ignoring any cached ones):
|
||||
|
||||
```bash
|
||||
python scripts/setup_twitch_auth.py --force-new-token
|
||||
```
|
||||
|
||||
### Method 2: Manual Setup
|
||||
|
||||
1. Create or edit a `.env` file in the root directory of the project
|
||||
2. Add the following information:
|
||||
|
||||
```
|
||||
TWITCH_CLIENT_ID=your_client_id_here
|
||||
TWITCH_CLIENT_SECRET=your_client_secret_here
|
||||
TWITCH_USERNAME=your_twitch_username
|
||||
TWITCH_CHANNEL=channel_to_connect_to
|
||||
```
|
||||
|
||||
3. Run the authentication script:
|
||||
|
||||
```bash
|
||||
python scripts/setup_twitch_auth.py --manual
|
||||
```
|
||||
|
||||
4. Follow the instructions to complete authentication
|
||||
|
||||
## Windows-Specific Information
|
||||
|
||||
On Windows, the application uses the Python `cryptography` library to generate self-signed certificates instead of relying on the OpenSSL command-line tool. This ensures better compatibility with Windows systems.
|
||||
|
||||
If you encounter any issues with certificate generation on Windows:
|
||||
|
||||
1. Make sure you have the latest version of pip: `python -m pip install --upgrade pip`
|
||||
2. Try reinstalling the cryptography package: `pip install --force-reinstall cryptography`
|
||||
3. Visual C++ Build tools might be required. If you get build errors, you may need to install the [Microsoft C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
|
||||
|
||||
## Important Notes
|
||||
|
||||
- The application uses user access tokens with `chat:read` and `chat:edit` scopes
|
||||
- IRC chat access requires a user access token (client credentials are not sufficient)
|
||||
- Twitch requires HTTPS for OAuth redirects, so we use a self-signed certificate for local development
|
||||
- The tokens are cached in `data/cache/token_cache.json` and refreshed automatically when needed
|
||||
- Never share your Client Secret or access tokens
|
||||
|
||||
## Self-Signed Certificates
|
||||
|
||||
The application automatically generates a self-signed certificate for HTTPS, which is stored in `data/ssl/`. This certificate is used for handling OAuth redirects from Twitch.
|
||||
|
||||
If you encounter certificate issues, you can delete the files in `data/ssl/` to generate a new certificate:
|
||||
- `data/ssl/localhost.crt`
|
||||
- `data/ssl/localhost.key`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you encounter authentication issues:
|
||||
|
||||
1. Make sure your Client ID and Client Secret are correct
|
||||
2. Check that your OAuth redirect URL in the Twitch Developer Console is exactly `https://localhost:3000` (with HTTPS)
|
||||
3. When prompted with certificate warnings in your browser, make sure to proceed past these warnings
|
||||
4. Verify that your Twitch account has the necessary permissions
|
||||
5. Delete the token cache file (`data/cache/token_cache.json`) and try again
|
||||
6. Check the application logs for detailed error messages
|
||||
7. If the browser doesn't open automatically, you can copy and paste the provided URL
|
||||
Loading…
Add table
Add a link
Reference in a new issue