MANTIS

Multimodal Alignment of Normalized Tissue Sections

Python tools to align H&E and IHC whole-slide images (WSIs) via comparable thumbnails, tissue masks, and 2D mask cross-correlation. A second CLI command correlates MIL attention scores with Ki67 intensity on the mapped WSI.

Quick start (Poetry)

cd MANTIS
poetry env use /path/to/python3.10+   # e.g. micromamba env
poetry install

CLI

mantis exposes two subcommands:

Subcommand	Purpose
pipeline	Full H&E ↔ Ki67 alignment → {output_dir}/{slide_id}.tif
attention-corr	Pearson r between MIL attention and mapped Ki67 grayscale per patch

(pipeline is optional: mantis --he-slide ... still works as shorthand.)

pipeline — alignment

poetry run mantis pipeline \
  --he-slide /path/to/365940he.tif \
  --ki67-slide /path/to/365940ki67.tif \
  -o ./mantis_out
# → ./mantis_out/365940he.tif
# → ./mantis_out_work/365940he/  (thumbnails, masks, …)

• Output dir (-o): final mapped WSI only, named {slide_id}.tif (default slide_id = H&E slide stem, e.g. 293892he.tif).
• Work dir (--work-dir = root, default {output-dir}_work): per-slide folder {work-dir}/{slide_id}/ with thumbnails, masks, labels, memmaps, JSON/CSV (safe for parallel jobs).

attention-corr — attention vs mapped Ki67

After pipeline (or an existing mapped WSI), compute the correlation used in notebooks/analysis.ipynb: mean grayscale per patch (ITU-R luma) vs MIL attention, with both variables z-scored on valid patches before Pearson correlation.

Stdout is only the correlation float (no labels), suitable for scripting:

CORR=$(poetry run mantis attention-corr \
  --slide-id 293892he \
  -o ./mantis_out \
  --processed-data-root /mnt/beegfs/home/g.cabas/hrr_scanner/processed_data \
  --feature-extractor uni_v2 \
  --mil abmil \
  --patch-size 256)
echo "$CORR"
# e.g. 0.140826

Optional side outputs (silent on stdout):

poetry run mantis attention-corr \
  --slide-id 293892he \
  --plot-out ./mantis_out/attention_corr_grids.png \
  --json-out ./mantis_out/attention_corr.json

Auto-resolved HDF5 paths (under --processed-data-root):

• Coords: trident/20x_256px_0px_overlap/features_{feature_extractor}/{slide_id}.h5
• Attention: results/ki/multi_instance/heatmaps/{feature_extractor}.{mil}/attention_scores/{slide_id}.h5

Override with --coords-h5 and/or --attention-h5 if needed.

Pipeline steps

1. Thumbnails — thumb_he.jpg / thumb_ki67.jpg at shared µm/px (thumbnail_meta.json).
2. Segmentation — Otsu masks (mask_he.png, mask_ki67.png).
3. Global align — Coarse + fine cross-correlation (correlation_table.csv).
4. Clusters + refine — DBSCAN, overlap/elbow filters, per-cluster rigid align, WSI export.

Requirements

Managed by Poetry: OpenCV, NumPy, SciPy, tifffile, imagecodecs, h5py.

CLI options (pipeline)

Option	Default	Description
--he-slide	(required)	H&E pyramid TIFF
--ki67-slide	(required)	Ki67 pyramid TIFF
-o / --output-dir	mantis_out	Final mapped WSI directory
--work-dir	{output-dir}_work	Work root; files go under {work-dir}/{slide-id}/
--slide-id	H&E slide stem	Final TIFF name {slide_id}.tif
--mag-he, --mag-ki67	20	Magnification if mpp not in TIFF
--max-long-edge	2000	Thumbnail long edge (px)
--coarse-step	10	Coarse alignment step
--refine-size	20	Fine search window
--eps, --min-samples	1.5, 10	DBSCAN clustering
--min-overlap-ratio	0.5	HE∩Ki67 overlap filter
--angle-max, --trans-range	15, 10	Per-cluster search
--export-viz	off	Extra aligned label PNGs

CLI options (attention-corr)

Option	Default	Description
--slide-id	(required)	Slide stem for {slide_id}.h5 files
-o / --output-dir	mantis_out	Directory with mapped WSI
--mapped-wsi	{output-dir}/{slide-id}.tif	Override mapped WSI path
--processed-data-root	/mnt/beegfs/home/g.cabas/hrr_scanner/processed_data	Root for Trident + heatmap HDF5
--feature-extractor	uni_v2	Trident features_{name}/ folder
--mil	abmil	Heatmap folder {extractor}.{mil}
--patch-size	256	Patch edge on mapped WSI (px)
--coords-h5, --attention-h5	—	Override auto-resolved HDF5 paths
--plot-out	—	Save z-scored attention/grayscale grid PNG (no stdout message)
--json-out	—	Save summary JSON with correlation, n_patches, n_valid

Exit code 0 on success; errors go to stderr.

Attention correlation (Python API)

from mantis import MappedSlideAttentionCorrelation

analysis = MappedSlideAttentionCorrelation.from_h5(
    mapped_wsi_path="mantis_out/293892he.tif",
    temp_dir="mantis_out_work/293892he",
    attention_h5_path=".../attention_scores/293892he.h5",
    coords_h5_path=".../features_uni_v2/293892he.h5",
    patch_size=256,
)
result = analysis.run()
result.correlation       # Pearson r (z-scored gray vs attention)
result.patch_intensity   # per-patch mean grayscale
result.features_table()  # pandas DataFrame (optional)

Coordinate mapping (Python API)

from mantis.coord_map import Ki67ToHEMapper

mapper = Ki67ToHEMapper.from_mantis_dir("mantis_out", slide_id="293892he")
# reads mantis_out_work/293892he/

# Thumbnail pixels (thumb_ki67 → thumb_he)
he_y, he_x, cluster_id, method = mapper.ki67_to_he(ki67_y, ki67_x)

# WSI pyramid level 0
he_y_wsi, he_x_wsi, ki_y_wsi, ki_x_wsi, cid, method = mapper.ki67_to_he_wsi(ki67_y, ki67_x)

Scale: y_wsi = y_thumb * scale_y (per slide in thumbnail_meta.json).

Tests

poetry install --with dev
poetry run pytest

CI runs on push/PR to main or master (see .github/workflows/tests.yml).