v0.16.1

Bug Fixes

• from ludwig.api import LudwigModel fails on Python 3.12 — When torchao and PyTorch are version-mismatched (torchao calls torch.utils._pytree.register_constant, added in PyTorch 2.5+), transformers's lazy loader raises ModuleNotFoundError for any class defined in modeling_utils.py, including PreTrainedModel. llm_utils.py and text_feature.py both imported these classes at module level; they are now deferred to TYPE_CHECKING. Follows the same fix applied to hf_utils.py in v0.15.1. (#4142)

• Replace assert with explicit exceptions; fix mutable default arg — Internal assert statements replaced with ValueError/RuntimeError so they aren't silently stripped with python -O. Fixed a mutable default argument that could cause cross-call state leakage. (#4152)

• Unified ruff toolchain — Replaced black + isort + flake8 with a single ruff invocation for linting and formatting. No behavior change for users.

v0.16.0

New Features

Timeseries Forecasting

• PatchTST & N-BEATS encoders — State-of-the-art patch-based and basis-expansion timeseries encoders. Both support multivariate and univariate forecasting with TimeseriesOutputFeature. (#4147)
• MASE & sMAPE metrics — Mean Absolute Scaled Error and symmetric Mean Absolute Percentage Error added for forecasting evaluation. (#4147)

Advanced PEFT Adapters

• New adapter types — TinyLoRA, C3A, OFT (Orthogonal Fine-Tuning), HRA (Householder Reflection Adaptation), WaveFT, LN-Tuning, VBLoRA. (#4146)
• New LoRA initializers — PiSSA (Principal Singular values and Singular vectors Adaptation), EVA (Explained Variance Adaptation), CorDA/LoftQ. (#4146)

Phase 6: Future Capabilities

• LLM config generation — ludwig generate_config "describe your task" uses an LLM to write the YAML config for you. (#4092)
• HyperNetwork combiner — Conditioning-based feature fusion where one feature generates weights for others. (#4092)
• Nash-MTL & Pareto-MTL — Game-theoretic and preference-based multi-task loss balancing strategies. (#4092)

New Examples

• VLM fine-tuning — LLaVA, Qwen2-VL, InternVL via is_multimodal: true. (#4140)
• Mamba-2 / Jamba encoders — State-space model encoders for sequence tasks. (#4140)
• Ray Serve & KServe deployment — Distributed and Kubernetes-native serving shims. (#4140)
• Multi-task & HyperNetwork examples (#4112)

Bug Fixes

• Python 3.12 import fix — Deferred PreTrainedModel import to TYPE_CHECKING to fix ImportError on Python 3.12 when HuggingFace transformers is not installed in all code paths.
• Dask image bytes UnicodeDecodeError — dask.config.set({"dataframe.convert-string": False}) is now applied at import time, preventing UnicodeDecodeError when image bytes pass through Dask string columns. (#4151)
• Dask shuffle partd race condition — Replaced file-based Dask shuffle (which hit a partd lock race under concurrent workers) with tasks-based shuffle. (#4150)
• Encoder input_shape contract — Fixed a contract violation where certain encoders did not correctly report or handle input_shape, causing shape mismatches during model construction. (#4148)
• Ray backend GPU underutilization — RayDatasetBatcher now runs to_tensors locally in the producer thread instead of spawning remote Ray tasks per block. Datasets are materialized before training to avoid Parquet re-reads on every epoch. (#4144)

v0.15.1

Bug fixes

• Ray training 3.7x slowdown eliminated — LudwigProgressBar was calling rt.report() on every training batch when running inside Ray workers (~1.9 s/call through the Ray GCS). With hundreds of batches this completely dominated wall-clock time. Per-batch progress reporting is now suppressed; training metrics continue to be reported correctly at eval/checkpoint time. Ray overhead is now ~1.7x vs local (fixed TorchTrainer setup cost), down from 3.7x. (#4144)

• GPU underutilization in Ray backend fixed — RayDatasetBatcher was running to_tensors via map_batches (spawning a Ray remote task per dataset block with scheduling overhead). It now runs locally in the producer thread. Also: datasets are now materialized before training to avoid re-reading Parquet from disk on every epoch. (#4144)

• Python 3.14 compatibility — LudwigBaseConfig subclasses crashed with PydanticUserError: Field requires a type annotation on Python 3.14 because annotations are now stored lazily via __annotate_func__. Fixed in _LudwigModelMeta.__new__. (#4144)

• ModernBERT tokenizer — Models containing "bert" in their name (e.g. answerdotai/ModernBERT-base) were incorrectly routed to BertTokenizer (WordPiece), causing Missing [UNK] token errors. ModernBERT now correctly uses HFTokenizer (AutoTokenizer / BPE). (#4144)

• Dask meta= parameter — Multiple feature types (binary, category, sequence, timeseries, text, vector) called .map() without a meta= argument, causing ValueError: Metadata inference failed in map when using the Dask engine (backend: {type: ray, processor: {type: dask}}). All bare .map() calls are now fixed. (#4144)

Ludwig 0.15.0

Ludwig 0.15.0

New Features

GRPO Alignment
Reward-model-free RLHF using Group Relative Policy Optimization. Set trainer.type: grpo to train LLMs directly from a reward signal without a separate reward model.

torchao Quantization + QAT
PyTorch-native quantization backend alongside bitsandbytes. Supports int4_weight_only, int8_weight_only, int8_dynamic, and float8 modes. Set qat: true to insert fake-quantization observers before training, recovering 1–2 perplexity points versus post-training quantization.

Multi-Adapter PEFT
Train and serve multiple named LoRA adapters on the same base model using adapters: (plural). Supports all PEFT weighted merge strategies: linear, SVD, TIES, DARE-linear, DARE-TIES, and magnitude pruning.

Native Optuna Hyperopt Executor
executor.type: optuna runs hyperparameter optimization without requiring Ray Tune. Supports Auto, GP, TPE, CMA-ES, and random samplers; median, Hyperband, SHA, and NOP pruners; and SQLite/PostgreSQL storage for resumable runs.

Timeseries Forecasting
First-class TimeseriesOutputFeature with a projector decoder that directly predicts all horizon steps in one forward pass. model.forecast(dataset, horizon=N) generates iterative multi-step predictions using O(window_size + horizon) incremental preprocessing.

Muon and ScheduleFreeAdamW Optimizers
Two new optimizers: muon for large-scale pretraining and schedule_free_adamw for fine-tuning without a learning rate schedule.

Image Segmentation Decoders
unet, segformer, and fpn decoders for semantic segmentation tasks on image output features.

Dependency Upgrades

Package	Previous	0.15.0
Python	3.11	3.12
PyTorch	2.5	2.7+
Transformers	4.x	5.x
Ray	2.x	2.54
Pydantic	1.x	2.x

Breaking Changes

• Horovod, Neuropod, and LightGBM backends removed
• training. hyperopt parameter prefix renamed to trainer. (e.g. training.learning_rate → trainer.learning_rate)
• ftrl optimizer removed; use adagrad or sgd with momentum
• model.train() returns a TrainingResults dataclass; tuple unpacking still works but is deprecated

Full Changelog

v0.14.1...v0.15.0

Ludwig 0.14.1

Patch release following v0.14.0.

Test suite

Fix the three (later four) row-ordering failures in `tests/integration_tests/test_ray.py::check_preprocessed_df_equal` that surfaced on `main` right after v0.14.0. Ray's compute path materializes the preprocessed dataframe through `Ray Dataset -> dataset.to_dask()`, which does not preserve the source row index — so the ray-produced and local-produced dataframes could list the same rows in different orders. The row-wise equality check then flagged visually-identical category / vector columns as unequal.

`check_preprocessed_df_equal` now sorts both sides by a per-row hash computed over every deterministic-content column, using `ndarray.tobytes()` for vector / sequence / timeseries / etc. cells so that tests with only a single scalar column (notably `test_ray_vector`) still produce a unique ordering. Binary / image / audio columns are excluded from the sort key because NaN-fill strategies can legitimately differ across backends; those columns were already compared with order-independent shape-only checks.

Fixes:

• `test_ray_tabular[dask]`
• `test_ray_tabular_save_inputs[parquet]`
• `test_ray_vector[parquet]`
• `test_ray_vector[csv]`

Documentation

README: drop the stale `ludwig export_torchscript` and Triton references (neither exists in the 0.14 codebase) and document the real 0.14 export surface — SafeTensors (default), `torch.export`, and ONNX via the dynamo-based exporter — with the actual `ludwig export_model` CLI flags (`--model_path`, `--output_path`, `--format`).

---

PyPI upload is handled automatically by `.github/workflows/upload-pypi.yml` on release publish.

Ludwig 0.14.0

Major feature release. See the full commit log and the updated
documentation for details.

Encoders

• Text: modernbert (Flash Attention 2, RoPE, 8192-token context). Removed deprecated transformer_xl, ctrl, and flaubert encoders.
• Image: clip, dinov2, siglip, convnextv2.
• Audio: wav2vec2, whisper, hubert.
• Sequence: mamba (linear-time state space model); RoPE support for the stacked Transformer encoder.
• Category: target (mean-target encoding with cross-fitting) and hash (feature hashing) for high-cardinality features.
• Number: bins (learned discretization).

Decoders

• Sequence: new transformer_generator decoder, teacher_forcing_decay scheduled sampling, and beam_width / beam_length_penalty beam search shared across generator variants.
• Category: new mlp_classifier decoder; calibration: temperature_scaling (Guo et al., ICML 2017); mc_dropout_samples for Monte Carlo dropout uncertainty (Gal & Ghahramani, ICML 2016).
• Image segmentation: configurable U-Net depth (num_stages), new segformer and fpn decoders.
• LLM: category_extractor now supports regex and json_schema match strategies plus constrain_to_vocabulary constrained decoding.

Losses

• focal_loss, dice_loss, lovasz_softmax_loss, nt_xent_loss, poly_loss.
• entmax_1.5_loss registered for category / text / sequence features.
• Open-set recognition: entropic_open_set and objectosphere (Dhamija et al., NeurIPS 2018).
• Anomaly detection: deep_svdd, deep_sad, drocc.

Trainer

• New optimizers: radam, adafactor, schedule_free_adamw, muon, soap.
• New LR schedulers: one_cycle, inverse_sqrt, polynomial, wsd (warmup-stable-decay).
• Preference-based LLM training: dpo, kto, orpo, grpo.
• Loss balancing: uncertainty, famo, gradnorm, log_transform.
• Quality presets (medium_quality / high_quality / best_quality), model soup, modality dropout.
• Removed: the ftrl optimizer.

Serving & export

• Auto-generated Pydantic request/response schemas from model config.
• Prometheus /metrics endpoint, structured logging, and request timeouts (HTTP 504).
• OpenAI-compatible vLLM server (PagedAttention, continuous batching) via ludwig.serve_vllm.run_vllm_server.
• Model export: SafeTensors (default), torch.export (.pt2), and the dynamo-based ONNX exporter (torch.onnx.export(dynamo=True)).

Infrastructure

• Python 3.12, PyTorch 2.7+, Pydantic 2, Transformers 5, NumPy 2.
• Ray 2.54 on the modern ray.data.Dataset pipeline, Dask 2026.1.2, MLflow 3.10.
• uv-based test runner and modernized GitHub Actions Docker workflows.
• Fix for distributed metric aggregation across Ray eval workers.
• Removed: Horovod, Neuropod, and the GBM / LightGBM backend.

---

PyPI upload is handled automatically by .github/workflows/upload-pypi.yml on release publish.

Ludwig 0.13.0

Ludwig 0.13.0

Alignment Training: KTO, ORPO, GRPO

• KTO (Kahneman-Tversky Optimization): human-feedback alignment without paired preference data
• ORPO (Odds Ratio Preference Optimization): reference-free preference learning
• GRPO (Group Relative Policy Optimization): DeepSeek-style RL with group-relative rewards
• Full DPO trainer with sequence packing support

Expanded PEFT & Quantization

• LoRA+, DoRA, LoftQ, VeRA, FourierFT adapters
• PEFT now available for ECD encoders (not just LLMs)
• torchao integration: int4/int8/float8 quantization
• trust_remote_code support fixed for custom-code models (#4094, #4095)

Encoder Modernization

• RoPE positional embeddings, ConvNeXtV2, TfIdf n-gram encoders
• Accelerate strategy replaces DDP/FSDP/DeepSpeed — simpler distributed training with full Accelerate ecosystem support
• DDP, FSDP, and DeepSpeed distributed strategy classes removed (use strategy: accelerate)

Model Export & Serving

• vLLM serving backend for optimized LLM inference
• MLflow 3.x integration
• Auto-generated model card and training report after every training run
• Ray Job Submission example for remote cluster training

Dependency Cleanup

• FT-Transformer is now the default combiner for 3+ features (stronger baseline)
• Ray Docker images updated to 2.54.0
• Removed internal DictWrapper class from LLM model
• Streamlined dependency groups in pyproject.toml

Ludwig 0.12.0

Ludwig 0.12.0

Modernized Build System

• Migrated from setup.py + 10 requirements files to a single pyproject.toml with hatchling
• Dynamic versioning from ludwig/globals.py
• SafeTensors for secure, zero-copy model weight serialization (ECD models + training checkpoints)
• Added torchcodec dependency (required by torchaudio 2.x)

Clean Config Layer

• Removed all marshmallow backward-compatibility layers (@ludwig_dataclass, _SchemaAdapter, .Schema().load/dump())
• Renamed BaseMarshmallowConfig -> LudwigBaseConfig, DictMarshmallowField -> NestedConfigField
• Strict validation by default — unknown config fields now warn and are stripped

4 New Combiners

• FTTransformerCombiner (type: ft_transformer): [CLS] token + Transformer self-attention (Gorishniy et al., NeurIPS 2021)
• CrossAttentionCombiner (type: cross_attention): Pairwise cross-attention between all feature pairs
• PerceiverCombiner (type: perceiver): Learnable latent bottleneck tokens (Jaegle et al., ICML 2022)
• GatedFusionCombiner (type: gated_fusion): Flamingo-inspired gated cross-modal fusion

Numerical Feature Tokenization

• PLEEncoder (type: ple): Piecewise Linear Encoding with quantile bin edges (Gorishniy et al., NeurIPS 2022)
• PeriodicEncoder (type: periodic): Learned sinusoidal features

Multi-Task Loss Balancing (trainer.loss_balancing)

• log_transform: log(1+loss) compression (DB-MTL)
• uncertainty: Homoscedastic uncertainty weighting (Kendall et al., CVPR 2018)
• famo: Fast Adaptive Multitask Optimization (Liu et al., NeurIPS 2023)
• gradnorm: Gradient normalization (Chen et al., ICML 2018)

Model Soup (trainer.model_soup)

Checkpoint weight averaging for better generalization at zero inference cost (Wortsman et al., ICML 2022)

Modality Dropout (trainer.modality_dropout)

Learnable missing-modality embeddings for robustness to missing inputs at inference

Quality Presets (preset: medium_quality|high_quality|best_quality)

AutoGluon-inspired one-line configuration for different quality/speed tradeoffs

Other

• Added california_housing dataset to ludwig.datasets
• Simplified torchaudio calls (removed legacy backend version checks)
• Fixed SafeTensors shared-memory tensor handling for tied weights

Benchmark Results

Model	Adult Census (AUC)	California Housing (RMSE)
ft_transformer	0.919	0.461
transformer	0.918	0.469
cross_attention	0.916	0.477
perceiver	0.916	0.477
concat (baseline)	0.911	0.491

FT-Transformer matches paper-reported results within 0.2%.

v0.11.4

Fixes

• Fix DDP checkpoint race condition: use os.makedirs(exist_ok=True) to prevent FileExistsError when multiple workers create the training checkpoints directory simultaneously
• Fix dask metadata mismatch in batch_predict: keep from_ray_dataset() inside tensor_extension_casting(False) context so partition dtypes match metadata during calibration
• Pin minimum dask version to 2026.1.2 in distributed requirements
• Disable tensor extension casting in batch_predict to fix dask metadata mismatch

v0.11.3

What's Changed

Features

• Upgrade to PyTorch 2.7.1 with bitsandbytes and CUDA compatibility fixes

Bug Fixes

• Fix dask metadata mismatch in batch_predict by disabling tensor extension casting
• Fix dropout removal in sequence encoder gradient test
• Fix StackedCNN gradient test to pass with sparse updates
• Pin torchvision and torchaudio versions in CI to match torch 2.6.0

Maintenance

• Pre-commit suggestions (#4075)