HistologyEvaluateMIL

Nextflow pipeline to evaluate Multi-Instance Learning (MIL) approaches on digital histology slides, using immunohistochemistry (IHC) as ground truth.

The pipeline aligns H&E slides with their corresponding IHC slides (e.g. Ki67), computes the correlation between MIL attention maps and IHC signal, and aggregates the results into a comparative table.

Overview

flowchart LR
    subgraph inputs [Inputs]
        DATA[data_csv]
        FE[feature_extractors_csv]
        ARCH[architectures_csv]
        FR[features_root]
        MR[mil_results_root]
    end

    MATCH[MATCH]
    CORR[CORR]
    CONCAT[CONCAT_FILES]

    DATA --> MATCH
    MATCH --> CORR
    FE --> CORR
    ARCH --> CORR
    FR --> CORR
    MR --> CORR
    CORR --> CONCAT
    CONCAT --> OUT[corr.txt]

Loading

1. MATCH — Aligns each H&E / IHC pair with mantis pipeline.
2. CORR — For every slide × feature extractor × MIL architecture combination, computes the attention–IHC correlation with mantis attention-corr.
3. CONCAT_FILES — Concatenates all per-combination results into corr.txt.

Requirements

• Nextflow ≥ 22.x
• MANTIS — Multimodal Alignment of Normalized Tissue Sections
• On the Kutral cluster: Singularity and Slurm access

Installing MANTIS

MANTIS must be installed before running the pipeline. It provides the mantis CLI used by the MATCH and CORR processes.

pip install git+https://github.com/digenoma-lab/MANTIS.git

Verify the installation:

mantis --help

MANTIS exposes two subcommands used by this pipeline:

Subcommand	Purpose
pipeline	Full H&E ↔ Ki67 alignment → {output_dir}/{slide_id}.tif
attention-corr	Pearson r between MIL attention and mapped Ki67 grayscale per patch

See the MANTIS repository for full CLI options, Python API, and pipeline internals.

Processes in this pipeline do not define their own container; mantis must be available on the PATH in the execution environment (e.g. a pre-configured Singularity image or cluster module).

Repository structure

HistologyEvaluateMIL/
├── main.nf                  # Workflow entry point
├── nextflow.config          # Resources, profiles, and reports
├── modules/
│   ├── match.nf             # MATCH process
│   └── corr.nf              # CORR and CONCAT_FILES processes
├── params/
│   ├── params_ki.yml        # Parameters for real Ki67 data
│   └── params_stub.yml      # Minimal parameters for testing
└── data/
    ├── data.csv             # Example slides (stub)
    ├── data_ki.csv          # Ki67 slides from the HRR project
    ├── feature_extractors.csv
    └── architectures.csv

Parameters

Parameter	Description
data_csv	CSV with H&E / IHC pairs per slide
feature_extractors_csv	CSV with patch feature extractor configurations
architectures_csv	CSV with MIL architectures to evaluate
features_root	Root directory with extracted features (e.g. Trident output)
mil_results_root	Root directory with MIL training/inference results
outdir	Output directory (default: ./results/)

data_csv format

slide_id,he,ihc
352811he,/path/to/352811he.tif,/path/to/352811ki67.tif
352825he,/path/to/352825he.tif,/path/to/352825ki67.tif

• slide_id: unique slide identifier (must match names used in features_root and mil_results_root).
• he: absolute path to the H&E image.
• ihc: absolute path to the reference IHC image.

feature_extractors.csv format

patch_encoder,patch_size,mag,overlap
uni_v2,256,20,0
virchow2,224,20,0

Column	Description
patch_encoder	Feature extraction model (e.g. uni_v2, virchow2)
patch_size	Patch size in pixels
mag	Magnification (e.g. 20 for 20×)
overlap	Overlap between patches

architectures.csv format

architecture
abmil
dsmil
dftd
clam
transformer
wikg

Each row defines a MIL architecture whose attention maps will be compared against IHC.

Usage

Local test (stub)

Does not run mantis; generates placeholder files to validate the pipeline graph:

nextflow run main.nf \
  -params-file params/params_stub.yml \
  -stub-run \
  -profile stub

Local execution with Singularity

nextflow run main.nf \
  -params-file params/params_ki.yml \
  -profile local

Kutral cluster (Slurm + Singularity)

nextflow run main.nf \
  -params-file params/params_ki.yml \
  -profile kutral

The kutral profile configures:

• Slurm executor, queue ngen-ko
• Singularity with /mnt/beegfs/labs/ bind mount

Quick reference

nextflow run main.nf -params-file params/params_ki.yml -profile kutral

Processes

MATCH

Aligns the H&E slide with its corresponding IHC slide using MANTIS.

Input	(slide_id, he, ihc)
Output	(slide_id, matched_ihc/<slide_id>.tif, matched_ihc_work/)
Command	mantis pipeline --he-slide <he> --ki67-slide <ihc> -o matched_ihc
Resources	4 CPUs, 50 GB RAM

CORR

Computes the correlation between the MIL attention map and the aligned IHC, for a specific feature extractor and architecture combination.

Input	MATCH output + (feature_extractor, patch_size, mag, overlap, architecture, features_root, mil_results_root)
Output	corr_<slide_id>-<extractor>-<patch_size>-<mag>-<overlap>-<mil>.txt
Command	mantis attention-corr ...
Resources	1 CPU, 10 GB RAM

This process runs over the Cartesian product of slides × feature extractors × architectures.

CONCAT_FILES

Aggregates all corr_*.txt files into a single table.

Input	List of corr_*.txt files
Output	corr.txt (published to params.outdir)
Resources	1 CPU, 10 GB RAM

Outputs

corr.txt

TSV table with header:

slide_id    feature_extractor    patch_size    mag    overlap    mil    corr

Each row corresponds to a unique combination of slide, feature extractor, and MIL architecture, with the attention–IHC correlation value.

Nextflow reports

Written to results/pipeline_info/ (or params.outdir):

• execution_timeline_*.html — execution timeline
• execution_report_*.html — resource usage report
• execution_trace_*.txt — detailed trace
• pipeline_dag_*.html — pipeline DAG

CI

The GitHub Actions workflow (.github/workflows/test.yml) runs the pipeline in stub mode on every push to main.

Author

Gabriel Cabas — version 0.0.1