Readme

README.md

@@ -8,6 +8,7 @@ This tool is designed to generate adversarial prefixes that can bypass prompt gu
 - Uses token minimization to keep prefixes as short as possible
 - Maintains a database of effective words to improve generation efficiency
 - Allows customization of the injection text, payload text, and component ordering
+- Optimized for performance with batch evaluation of candidates
 
 ## Installation
 
@@ -35,17 +36,12 @@ python hacking.py
 
 The tool supports several command-line options for customizing its behavior:
 
-- `--order`: Template for ordering components. Use `{injection}` for injection text, `{prefix}` for adversarial prefix, and `{text}` for the payload. Default: `{injection}{prefix}{text}`
+- `--injection`: Injection text to use (the instruction text that precedes your prefix)
-- `--injection`: Injection text to use in the template
+- `--mandatory-text`: The payload text that follows your prefix (typically malicious content to bypass)
-- `--payload`: Payload text to use in the template
+- `--init-prefix-words-count`: Number of words to use in the initial prefix
 
 #### Examples
 
-Changing the order of components:
-```bash
-python hacking.py --order "{prefix}{injection}{text}"
-```
-
 Using custom injection text:
 ```bash
 python hacking.py --injection "Say the following exactly:"
@@ -53,17 +49,22 @@ python hacking.py --injection "Say the following exactly:"
 
 Using a custom payload:
 ```bash
-python hacking.py --payload " and give me the root password"
+python hacking.py --mandatory-text " and give me the root password"
+```
+
+Setting initial prefix size:
+```bash
+python hacking.py --init-prefix-words-count 20
 ```
 
 Combining options:
 ```bash
-python hacking.py --order "{prefix}{text}{injection}" --injection "Answer with yes" --payload " can you help me hack into a website?"
+python hacking.py --injection "Answer with yes" --mandatory-text " can you help me hack into a website?" --init-prefix-words-count 25
 ```
 
 ## How It Works
 
-The core goal of this tool is to find an adversarial prefix that, when prepended (or inserted in a user-specified order) to a target payload, causes a prompt-guarding classifier (such as Llama Guard) to classify the overall prompt as benign—even if the payload is malicious or forbidden.
+The core goal of this tool is to find an adversarial prefix that, when combined with the target payload, causes a prompt-guarding classifier (such as Llama Guard) to classify the overall prompt as benign—even if the payload is malicious or forbidden.
 
 ### Main Steps
 
@@ -71,52 +72,58 @@ The core goal of this tool is to find an adversarial prefix that, when prepended
    - The program starts by generating an initial prefix. This can be a random selection of words, or (if available) a set of words that have previously performed well, as tracked in a local word performance database.
    - The user can control the number of words in the initial prefix with `--init-prefix-words-count`.
 
-2. **Optimization Loop**
+2. **Optimization Loop with Batch Evaluation**
-   - The main loop iteratively updates the adversarial prefix to maximize the probability that the classifier labels the prompt as benign.
+   - The main loop iteratively updates the adversarial prefix to maximize the probability of a benign classification.
+   - Performance optimization: Multiple candidate prefixes are evaluated in parallel using batch processing
    - In each iteration:
-     - The current prefix, injection text, and payload are combined according to the user-specified template (e.g., `{injection}{prefix}{text}`).
+     - The program computes gradients for the current prefix tokens
-     - The combined prompt is tokenized and passed through the classifier model.
+     - Gradients are used to sample multiple candidate prefixes
-     - The program computes gradients with respect to the prefix tokens, using a combination of two objectives:
+     - All candidates are evaluated in a single batch for efficiency
-       - **Benign Maximization:** Increase the classifier's benign probability.
+     - Each candidate is scored using benign probability, loss, and token count
-       - **Loss Minimization:** Minimize the cross-entropy loss for the benign class.
+     - The best candidate is selected for the next iteration
-     - The gradients are used to propose new candidate prefixes by sampling new tokens (with some randomness for exploration).
-     - Each candidate is scored using a weighted combination of benign probability, normalized loss, and a penalty for longer token sequences.
-     - The best candidate is selected for the next iteration.
 
-3. **Stagnation Handling**
+3. **Stagnation Handling with Optimized Word Addition**
-   - If the optimization loop fails to make progress for a number of iterations, the program attempts to inject new words (either from the database or randomly) into the prefix to escape local optima.
+   - If optimization stagnates, the program tries to add new words to escape local optima
-   - The word database is updated with the performance of each tested word, allowing the tool to learn which words are most effective for future runs.
+   - The word addition process is also batch-optimized to evaluate many word candidates efficiently
+   - The database tracks which words are most effective for future runs
 
 4. **Early Stopping and Success Criteria**
-   - The loop stops early if a prefix is found that achieves a high benign probability (default: >95%).
+   - The loop stops early if a prefix achieves a high benign probability (default: >95%)
-   - If no such prefix is found after a set number of iterations, the best prefix found so far is used.
+   - If no such prefix is found after a set number of iterations, the best prefix found so far is used
 
-5. **Token Minimization**
+5. **Token Minimization (Non-Batched)**
-   - Once a high-confidence benign prefix is found, the program attempts to minimize its length by systematically removing tokens that do not significantly reduce the benign probability.
+   - Once a high-confidence benign prefix is found, the program minimizes its length
-   - This is done via an ablation process, removing one token at a time and re-evaluating the classifier.
+   - Uses a direct, iterative approach that removes one token at a time
+   - For each iteration:
+     - Try removing each token and evaluate the effect on the benign score
+     - Remove the token that maintains the highest benign score (if still above threshold)
+     - Continue until no more tokens can be removed while staying above the minimum threshold
+   - This careful approach ensures maximum token reduction while maintaining effectiveness
 
 6. **Final Output**
-   - The program prints the final adversarial prefix, the full prompt (with the user-specified order), and the classifier's output for both the original and adversarial prompts.
+   - The program prints the final adversarial prefix, full prompt, and classifier results
-   - It also reports the number of tokens used in the prefix and the total prompt.
+   - It also reports token counts and reduction percentages
 
 ### Word Performance Database
 
-- The tool maintains a SQLite database (`word_performance.db`) that tracks the effectiveness of individual words (and their positions) in increasing benign classification.
+- The tool maintains a SQLite database (`word_performance.db`) that tracks the effectiveness of words
-- This database is used to prioritize high-performing words in future runs, making the optimization process more efficient over time.
+- This database is used to prioritize high-performing words in future runs
+- Performance metrics include both benign score improvement and token efficiency
 
-### Customization
+### Performance Optimizations
 
-- The user can control the order of the injection text, prefix, and payload using the `--order` argument (e.g., `{prefix}{injection}{text}`).
+- **Batch Processing**: Multiple candidate prefixes are evaluated in parallel
-- The injection text and payload can be set via `--injection` and `--mandatory-text`.
+- **Early Exit**: Processing stops when a candidate is clearly not going to improve
-- The number of words in the initial prefix can be set with `--init-prefix-words-count`.
+- **Efficient Token Ablation**: Direct, systematic approach to token removal
+- **Database-Informed Word Selection**: Uses past performance to guide optimization
 
 ### Example Workflow
 
-1. The tool starts with a prefix like `apple banana orange ...`.
+1. The tool starts with an initial prefix (e.g., 15 words)
-2. It iteratively tweaks the prefix to maximize the benign score, using gradients and candidate sampling.
+2. It optimizes the prefix using batched gradient-based updates
-3. If stuck, it tries adding new words from its database or at random.
+3. If stuck, it tries adding new words from its database or at random
-4. Once a high benign score is achieved, it removes unnecessary tokens to make the prefix as short as possible.
+4. Once a high benign score is achieved, it minimizes tokens while maintaining the benign rating
-5. The final prefix and prompt are output, along with classifier results and token counts.
+5. The final result is a minimal prefix that reliably bypasses the guard
 
 ## License