rdkit-fp

Parallel RDKit fingerprints with a beginner one-liner and an advanced staged pipeline.

Install

pip install rdfp

Quickstart (10 seconds)

rdfp examples/chembl_1k.smi outputs/fps/

With explicit preset:

rdfp examples/chembl_1k.smi outputs/fps/ --preset ecfp4

This default command is equivalent to rdfp fps INPUT OUTPUT.

Common CLI usage

Simple run with explicit workers/chunk size:

rdfp examples/chembl_1k.smi outputs/fps/ --workers -1 --chunk 100000

Use IDs from a column and keep stable row mapping metadata:

rdfp data/compounds.smi outputs/fps/ --smiles-col 0 --id-col 1

Write gzip-compressed NumPy chunks:

rdfp examples/chembl_1k.smi outputs/fps_gz/ --format numpy --gzip

Resume a stopped large run:

rdfp data/compounds.smi outputs/fps/ --resume

Run built-in demo data:

rdfp demo

Presets

Preset	Meaning
ecfp4 (default)	Morgan radius 2, 2048 bits
ecfp6	Morgan radius 3, 2048 bits
rdkit	RDKit topological, 2048 bits
ap	Atom-pair, 2048 bits
tt	Topological torsion, 2048 bits
pattern	Pattern fingerprint, 2048 bits

You can override preset values with explicit flags (--fp-type, --fp-size, --radius, --include-chirality).

Advanced lane

Stage 1 only (SMILES -> mols):

rdfp mols examples/chembl_1k.smi outputs/mols/

Stage 2 only (mols -> fingerprints):

rdfp fps-from-mols outputs/mols/ outputs/fps_from_mols/ --resume

Both stages with mol persistence:

rdfp fps examples/chembl_1k.smi outputs/fps/ --save-mols outputs/mols/

Helpful flags

• --workers (alias of --n-jobs)
• --chunk (alias of --chunk-size)
• --preset ecfp4|ecfp6|rdkit|ap|tt|pattern
• --input-smiles-col (alias of --smiles-col)
• --id-col N parse an input ID column if present
• --resume skip existing chunk outputs
• --include-row-mapping include compact invalid-row metadata in each chunk
• --format numpy|packed|pickle (packed default)
• --gzip gzip-compress chunk outputs and JSON metadata

Python API

from pathlib import Path
import rdfp

smiles_path = Path("examples/chembl_1k.smi")
rdfp.smiles_to_fps_chunked(
    rdfp.iter_smiles_records(smiles_path),
    output_dir="outputs/fps_api/",
    fp_type="morgan",
    fp_size=2048,
    radius=2,
    fmt="packed",
    n_jobs=-1,
    chunk_size=100_000,
    include_row_mapping=False,
    resume=True,
)

Output layout

packed output (default):

outputs/fps/
  fps_0000.npy
  fps_0000.json
  fps_0001.npy
  fps_0001.json
  metadata.json

Each chunk stores only valid fingerprints in fps, and metadata.json summarizes totals for the whole run.

If you enable --include-row-mapping, each chunk stores compact invalid-row metadata (invalid_indices and invalid_row_indices) alongside the chunk instead of writing a separate .index.json file.

Compressed output with --gzip:

outputs/fps_gz/
  fps_0000.npy.gz
  fps_0000.json.gz
  fps_0001.npy.gz
  fps_0001.json.gz
  metadata.json.gz

For numpy and packed, each chunk keeps its compact JSON sidecar (fps_0000.json or fps_0000.json.gz).

Storage Benchmark

Reproducible benchmark script:

python scripts/benchmark_storage_formats.py --repeats 25 --warmups 2

This benchmark uses examples/chembl_1k.smi, computes ECFP4 once up front, and reports storage-only overhead for writing the output files.

Format	Output size	Processing time
numpy	200.6 KiB	0.2 ms
numpy + gzip	6.6 KiB	16.9 ms
packed	25.6 KiB	0.2 ms
packed + gzip	5.6 KiB	16.7 ms
pickle	200.6 KiB	0.1 ms
pickle + gzip	6.6 KiB	17.6 ms

On this benchmark, packed is the smallest uncompressed format and packed + gzip is the smallest overall. numpy and pickle are effectively tied for uncompressed bit fingerprints because both store the full unpacked uint8 matrix.

Compatibility and packaging

• Repo name stays rdkit-fp.
• Install/import/CLI are centered on rdfp.

Release flow

• CI runs on pushes/PRs to main.
• PyPI publish runs from tags matching v* (for example v0.1.1).