YARA vs YARA-X Benchmarking Suite

📁 Structured Corpus Architecture

Targets are structured modularly inside the corpus/ folder:

• Subdirectories (e.g., corpus/local_clean/): The folder name is the name of the corpus. It contains compressed .zip files representing the scan targets (malware simulation zips use the standard password infected).
• Hash List Files (e.g., corpus/pe_files.txt): The file name (without extension) is the name of the corpus. It contains a list of SHA-256 hashes to fetch from VirusTotal.

yara-x-benchmarks/
├── corpus/                    # Structured input corpora
│   ├── local_clean/           # Clean corpus folder
│   │   └── clean.zip          # Compressed clean targets (~50MB)
│   └── pe_files.txt          # Hash list corpus (corpus name: 'vt_hashes')
├── targets/                   # Extracted scan targets directory (git-ignored)
│   ├── local_clean/           # Automatically unpacked clean files
│   └── pe_files/              # Automatically unpacked simulated malware files
├── rules/                     # Hand-curated rule sets for performance comparisons
│   ├── aho_corasic.yar        # Standard strings, hex sequences, and baseline indicators
│   ├── teddy.yar              # Simple rule with very few patterns, where YARA-X uses Teddy instead of Aho-Corasick.
│   └── pe.yar                 # Rules using the PE module.
├── manage_targets.py          # Structured target extractor, packager, and VT downloader
├── run_benchmarks.py          # Dynamic orchestrator running hyperfine and rendering reports

---

Tip

Custom Targets Benchmarking: You do not have to define compressed archives or VT hash lists in the corpus/ folder to run custom benchmarks. You can simply create your own subdirectory directly inside the targets/ folder (e.g. targets/my_custom_corpus/) and place any set of files you wish to scan inside it. The orchestrator will automatically discover it and include it in your benchmark report the next time you execute python3 run_benchmarks.py.

🚀 Quick Start

1. Setup & Install Dependencies

# Install hyperfine
brew install hyperfine

# Install python dependencies
pip install -r requirements.txt

2. Auto-Prepare target Corpora

Prepare all structured directories inside corpus/ at once:

python3 manage_targets.py --prepare-all

This extracts compressed .zip files under corpus/ to their corresponding subdirectory inside targets/.

3. Run Benchmarks

Execute the orchestrator script to auto-discover prepared targets, compile rules, and run the benchmarks:

python3 run_benchmarks.py

4. Cleanup Target Files

Safely remove all unpacked test files from your workspace:

python3 manage_targets.py --clean

---

🛡️ VirusTotal Integration

To configure a VirusTotal-driven hashlist corpus:

1. Set your VirusTotal API key:

   export VT_API_KEY="your_virustotal_api_key"

2. Run the prepare script for this specific corpus:

   python3 manage_targets.py --prepare pe_files

The downloaded targets are automatically saved to targets/pe_files/ and will be dynamically discovered and scanned during the next benchmark run.