AI-Powered MRI Analysis for Brain Tumor Detection with Automated Clinical Reporting
- Overview
- System Architecture
- Model Performance
- Features
- Project Structure
- Installation
- Usage
- API Reference
- Output Artifacts
- Training Pipeline
- Evaluation Results
- Technologies Used
- Contributing
- License
- Citation
This project presents a production-ready medical AI system for brain tumor detection from MRI scans. Built on Ultralytics YOLOv8m (medium variant), it detects and classifies brain tumors into four categories:
| Class | Description |
|---|---|
| Glioma | Tumor originating from glial cells; varies in aggressiveness |
| Meningioma | Typically slow-growing tumor of the meninges |
| Pituitary | Adenoma of the pituitary gland; usually benign |
| No Tumor | Normal brain anatomy — no mass detected |
Each prediction generates:
- ✅ Annotated MRI — Bounding boxes with class labels & confidence scores
- ✅ Heatmap overlay — Visual activation map for interpretability
- ✅ Clinical PDF Report — Professional-grade, ready for medical records
The system is designed as a full-stack web application with a FastAPI backend and a responsive HTML/CSS/JS frontend, deployable on local infrastructure or cloud environments.
┌─────────────────────────────────────────────────────────┐
│ Client (Browser) │
│ ┌─────────────┐ ┌──────────────┐ ┌───────────────┐ │
│ │ Upload MRI │ │ View Results│ │ Download PDF │ │
│ └──────┬──────┘ └──────┬───────┘ └───────┬───────┘ │
└─────────┼────────────────┼──────────────────┼────────────┘
│ │ │
┌─────▼────────────────▼──────────────────▼────────────┐
│ FastAPI Web Server (port 8000) │
│ ┌──────────────────┐ ┌────────────────────────┐ │
│ │ /predict-all │ │ /download/pdf/{name} │ │
│ │ /model-info │ │ /images & /heatmaps │ │
│ └────────┬─────────┘ └────────────────────────┘ │
└───────────┼──────────────────────────────────────────┘
│
┌───────────▼──────────────────────────────────────────┐
│ Inference Engine │
│ ┌──────────────────────────────────────────────┐ │
│ │ YOLOv8m (best.pt) │ │
│ │ • ONNX Runtime • PyTorch • OpenCV │ │
│ └──────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────┘
│
┌───────────▼──────────────────────────────────────────┐
│ Output Generation Pipeline │
│ ┌───────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ Annotated │ │ Heatmap │ │ PDF Report │ │
│ │ Images │ │ Overlay │ │ (ReportLab) │ │
│ └───────────┘ └──────────┘ └──────────────────┘ │
└──────────────────────────────────────────────────────┘
│
┌───────────▼──────────────────────────────────────────┐
│ Persistence Layer │
│ ┌──────────────────────────────────────────────┐ │
│ │ SQLite (report_counter) — Sequential IDs │ │
│ │ Format: BTD-YYYY-XXXX │ │
│ └──────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────┘
The YOLOv8m model was trained on a curated brain tumor MRI dataset and evaluated on held-out validation and test splits.
| Metric | Value |
|---|---|
| mAP@0.5 | 97.2% |
| mAP@0.5:0.95 | 88.6% |
| Precision | 95.1% |
| Recall | 93.8% |
| Inference Time | ~45 ms (GPU) / ~180 ms (CPU) |
Full evaluation curves available under
eval_val/andeval_test/
Precision-Recall curve — strong performance across all classes with high AUC
Confusion matrix showing accurate classification with minimal misclassifications
Training metrics across epochs — loss convergence, precision, recall, and mAP progression
- Real-time Detection — Upload an MRI and get results in under 200ms
- 4-Class Classification — Glioma, Meningioma, Pituitary, No Tumor
- Configurable Threshold — Adjust confidence threshold via UI or API
- Annotated Visualizations — Bounding boxes with labels and confidence scores
- Activation Heatmaps — Class activation overlay for model interpretability
- Professional PDF Reports — Structured medical reports generated automatically via ReportLab
- Clinical Summaries — Disease-specific summaries with recommended follow-up actions
- Report Tracking — Sequential IDs (
BTD-YYYY-XXXX) for audit trails - Medical-Grade Styling — Clean, formal design suitable for clinical environments
- Responsive UI — Works on desktop and tablet
- Interactive Controls — Confidence threshold slider, tabbed navigation
- Instant Preview — Side-by-side original, annotated, and heatmap views
- One-Click Download — Save reports and images locally
├── main.py # FastAPI application — core server logic
├── index.html # Frontend web interface
├── requirements.txt # Python dependencies
├── .gitignore # Git exclusion rules
│
├── weights/ # Trained model weights
│ ├── best.pt # Best YOLOv8m checkpoint
│ ├── last.pt # Last training checkpoint
│ └── model.onnx # ONNX-exported model for deployment
│
├── training/ # Training artifacts & evaluation curves
│ ├── results.png # Training metrics plot
│ ├── results.csv # Raw training metrics data
│ ├── confusion_matrix.png # Confusion matrix
│ ├── confusion_matrix_normalized.png
│ ├── BoxPR_curve.png # Precision-Recall curve
│ ├── BoxP_curve.png # Precision curve
│ ├── BoxR_curve.png # Recall curve
│ ├── BoxF1_curve.png # F1 curve
│ ├── labels.jpg # Label distribution
│ └── train_batch*.jpg # Training batch samples
│
├── eval_val/ # Validation set evaluation
│ ├── confusion_matrix.png
│ ├── BoxPR_curve.png
│ ├── BoxP_curve.png
│ ├── BoxR_curve.png
│ ├── BoxF1_curve.png
│ └── val_batch*_*.jpg # Validation batch predictions
│
├── eval_test/ # Test set evaluation
│ ├── confusion_matrix.png
│ ├── BoxPR_curve.png
│ ├── BoxP_curve.png
│ ├── BoxR_curve.png
│ ├── BoxF1_curve.png
│ └── val_batch*_*.jpg # Test batch predictions
│
├── outputs/ # Generated inference outputs
│ ├── images/ # Annotated detection images
│ ├── heatmaps/ # Heatmap overlay images
│ └── reports_pdf/ # Generated PDF clinical reports
│
├── database/ # SQLite persistence
│ └── counter.db # Report ID counter
│
├── __pycache__/ # Python cache
│
├── Brain_Tumor_YOLOv8.ipynb # Training & evaluation notebook
├
├── dataset distribution.png # Dataset class distribution
└── metics.png # Additional metrics visualization
- Python 3.9+
- pip / conda
- (Optional) CUDA-capable GPU for accelerated inference
# 1. Clone the repository
git clone https://github.com/yourusername/brain-tumor-yolov8.git
cd brain-tumor-yolov8
# 2. Create virtual environment
python -m venv venv
source venv/bin/activate # Linux/macOS
# venv\Scripts\activate # Windows
# 3. Install dependencies
pip install -r requirements.txtuvicorn main:app --host 0.0.0.0 --port 8000 --reloadThe server starts at http://localhost:8000.
Open index.html in your browser (or serve it via any HTTP server):
# Option: Serve via Python's HTTP server
python -m http.server 5500
# Then navigate to http://localhost:5500curl http://localhost:8000/Response:
{ "status": "OK", "message": "Medical Brain Tumor Detection API is running!" }curl http://localhost:8000/model-infoResponse:
{
"model_path": "weights/best.pt",
"model_type": "YOLO (Ultralytics)",
"classes": { "0": "glioma", "1": "meningioma", "2": "no tumor", "3": "pituitary" },
"task": "Brain Tumor Object Detection",
"input_format": "MRI Images"
}curl -X POST http://localhost:8000/predict-all \
-F "file=@/path/to/mri.jpg" \
-G -d "conf=0.25"Parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
file |
UploadFile | — | MRI image file (JPEG/PNG) |
conf |
float | 0.25 |
Confidence threshold (0.0–1.0) |
Response includes:
- Summary metadata (Report ID, filename, timestamp, inference time)
- Detection list (class, confidence, bounding box coordinates)
- URLs for annotated image, heatmap, and downloadable PDF report
| Aspect | Details |
|---|---|
| Input Format | JPEG, PNG, BMP, TIFF |
| Input Resolution | Any (automatically resized by YOLO) |
| Output Format | JSON + JPEG images + PDF report |
| Report ID Format | BTD-{YYYY}-{XXXX} (e.g., BTD-2026-0042) |
| Report Content | Metadata table, clinical summary, annotated image, heatmap, detection table, disclaimer |
- Header — System name and report title
- Metadata Table — Report ID, filename, date, confidence threshold, inference time, detections count
- Clinical Summary — Status indicator (Finding Detected / No Tumor), diagnosis name, clinical description
- Imaging Results — Annotated detection image and heatmap overlay
- Detection Details — Tabular listing of all detections with coordinates and confidence
- Medical Disclaimer — Required legal notice for AI-assisted diagnostics
Bounding boxes with class labels and confidence scores overlaid on original MRI
Clear activation map highlighting regions of interest detected by the model, generated via OpenCV colormap blending with the original scan.
A professional multi-section PDF document containing all diagnostic information, formatted for clinical use and patient records.
The model was trained using Brain_Tumor_YOLOv8.ipynb with the following configuration:
| Hyperparameter | Value |
|---|---|
| Model | YOLOv8m |
| Image Size | 640 × 640 |
| Batch Size | 16 |
| Epochs | 100 |
| Optimizer | AdamW |
| Learning Rate | 0.001 (cosine decay) |
| Data Augmentation | Mosaic, MixUp, HSV jitter, flip, rotation |
| Early Stopping | Patience = 15 epochs |
Class distribution across training, validation, and test splits
Precision curve over training epochs
Recall curve showing consistent improvement
Normalized confusion matrix on validation set — high diagonal values confirm accurate multi-class discrimination
| Class | Precision | Recall | mAP@0.5 | mAP@0.5:0.95 |
|---|---|---|---|---|
| glioma | 94.8% | 93.2% | 97.5% | 89.1% |
| meningioma | 95.3% | 94.1% | 97.8% | 88.4% |
| pituitary | 96.2% | 94.9% | 98.1% | 90.2% |
| no tumor | 94.1% | 93.0% | 95.4% | 86.7% |
Precision-Recall curve on held-out test set confirming generalization
| Technology | Purpose |
|---|---|
| Ultralytics YOLOv8 | State-of-the-art object detection model |
| FastAPI | High-performance async web framework |
| OpenCV | Image processing, heatmap generation |
| ReportLab | PDF document generation |
| NumPy | Numerical computing |
| SQLite | Lightweight local database |
| Python-Multipart | File upload handling |
| HTML5 / CSS3 / JavaScript | Frontend web interface |
Contributions are welcome! Please follow these steps:
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
Please ensure your code follows the existing style and includes appropriate documentation.
This project is licensed under the MIT License — see the LICENSE file for details.
If you use this project in your research or clinical work, please cite:
@software{brain_tumor_yolov8,
title = {Brain Tumor Detection System using YOLOv8m},
author = {Your Name},
year = {2026},
url = {https://github.com/yourusername/brain-tumor-yolov8}
}