odh-v3.5-EA1

What's Changed

• [RHOAIENG-22171] Added option to build and push kserve images. by @andresllh in #609
• add checklist item for jira link by @VedantMahabaleshwarkar in #617
• 250514 sync master by @spolti in #614
• Allow OCI for multi-node/multi-gpu by @israel-hdez in #618
• Merge KServe upstream into ODH by @spolti in #621
• 250520 sync master by @spolti in #623
• Add pierDipi and bartoszmajsak to OWNERS by @pierDipi in #633
• RHOAIENG-26179: Deploy CMA / KEDA during E2E tests setup by @pierDipi in #631
• Add guide to create a new branch from an upstream release by @israel-hdez in #615
• Disallow Propagation of haproxy.router.openshift.io/timeout Annotation by @brettmthompson in #637
• RHOAIENG-25750: Split syncable and static params by @pierDipi in #640
• [master] Fix issue with OCI support for multi-node/multi-gpu by @israel-hdez in #628
• Revert "RHOAIENG-25750: Split syncable and static params" by @pierDipi in #642
• RHOAIENG-24134: KServe Support secure access to the metrics server with TriggerAuthentication for KEDA autoscaling by @pierDipi in #643
• Back-porting fixes for InferenceGraph by @israel-hdez in #647
• Adding fvt tests and setup for s3 storage with tls by @brettmthompson in #645
• Remove extra step that printed the old tag by @hdefazio in #654
• [Cherry pick] Python test fixes from upstream by @hdefazio in #659
• [Cherry pick] Stop and resume a model [Raw Deployment] (kserve#4455) by @hdefazio in #658
• [RHOAIENG-26152] created-fvt-test-for-knative-feature-schedulername by @andresllh in #657
• [RHOAIENG-27166] Fix flaky test failure with mc alias by @andresllh in #670
• [RHOAIENG-26313]-create-fvt-test-for-multi-container-probing by @andresllh in #665
• Resolve Resource Conflict In FVT Tests by @brettmthompson in #678
• [RHOAIENG-28024] update pip version by @Jooho in #684
• Docs on how to get a token for protected InferenceServices by @israel-hdez in #677
• Resolve inference endpoint using runtime protocol when applicable by @israel-hdez in #667
• Sync/250623 upstream master by @brettmthompson in #689
• Sync/250625 upstream master by @brettmthompson in #693
• [Cherry-pick] Stop and Resume a transformer by @hdefazio in #703
• fix keda raw e2e tests for odh by @VedantMahabaleshwarkar in #619
• [RHOAIENG-15553] Adding the kfp template to the odh/kfp docs. by @mholder6 in #650
• [Cherry pick] Stop and resume an explainer by @hdefazio in #705
• Restore zero initial scale behavior on master branch by @brettmthompson in #709
• Improve Model Load Status Check in s3-TLS FVT Tests by @brettmthompson in #714
• [master] Align support secure access to prometheus in Keda autoscaling to upstream API by @pierDipi in #740
• Updating s3 tls fvt tests to cover all new scenarios by @brettmthompson in #756
• 250717 sync master by @spolti in #752
• 250731 sync kserve/master into odh/master batch1 by @mholder6 in #805
• 250731 sync kserve/master into odh/master batch2 by @mholder6 in #810
• 250731 sync kserve/master into odh/master batch3 by @mholder6 in #820
• [RHOAIENG-31248] - KServe http: TLS handshake error from XXXXXX: remo… by @spolti in #816
• [Sync]2025-0815 from upstream kserve master to odh kserve master by @Jooho in #844
• Revert "[Sync]2025-0815 from upstream kserve master to odh kserve master" by @Jooho in #849
• [Sync]2025-0815 from upstream kserve master to odh kserve master by @Jooho in #851
• modify keda autoscaling e2e test to work with ODH by @VedantMahabaleshwarkar in #873
• Fix indentation issue on go unit test:check-coverage workflow by @hdefazio in #881
• [Sync]2025-0822 from upstream kserve master to odh kserve master by @vivekk16 in #866
• Support for Configurable oauth-proxy Timeout by @brettmthompson in #919
• Adding serverless to raw migration script by @andresllh in #896
• [RHOAIENG-33172] - ModelMesh to RawDeployment migration helper by @spolti in #893
• [RHOAIENG-32520] - Update kube-rbac-proxy image by @spolti in #901
• Revert and Resync [Sync]2025-0822 from upstream KServe master to ODH master (preserve history) by @vivekk16 in #924
• Add vivekk16 to OWNERS by @vivekk16 in #956
• Update README.md for Migration by @vivekk16 in #966
• Minor fixes by @spolti in #959
• [RHOAIENG-34931] - Storage Initializer can't handle versioned models storage by @spolti in #968
• Update minio image to use what upstream uses by @mholder6 in #975
• [RHOAIENG-31086] - IG test for raw deployment by @spolti in #973
• 20251118 sync upstream by @mholder6 in #976
• Revert "20251118 sync upstream" by @mholder6 in #991
• [RHOAIENG-36758] - Investigate and Optimize KServe Unit Test Executio… by @spolti in #992
• [RHOAIENG-36673] - Investigate and Refactor KServe E2E Test Suite for… by @spolti in #987
• Sync upstream kserve/kserve - Batch 1 (commits 1-13 + dependencies) by @andresllh in #1006
• [Sync Upstream] Batch 2: Autoscaling, LLMISVC improvements, and Time Series API by @andresllh in #1007
• Add self to OWNERS; remove hdefazio, rnetser by @jlost in #1016
• RHOAIENG-39707: Fix S3 TLS StorageSpec E2E Test Failures by @jlost in #1015
• [RHOAIENG-39626] - KServe E2E Tests with Skip Markers - Comprehensive… by @spolti in #1024
• [RHOAIENG-13818] - Cleanup, remove disabled GH Actions by @spolti in #1027
• chore: bump github.com/kedacore/keda/v2 from 2.16.1 to 2.17.3 by @jlost in #1034
• [RHOAIENG-38500] Remove deprecated create-release workflow by @vivekk16 in #1054
• add pipelines for odh ci builds by @MohammadiIram in #1051
• update component names by @MohammadiIram in #1059
• Sync 260121 kserve upstream odh - align git history by @spolti in #1063
• Revert "Sync 260121 kserve upstream odh - align git history" by @spolti in #1066
• Sync 260121 kserve upstream odh by @spolti in #1067
• Sync 260122 kserve upstream odh by @spolti in #1074
• Revert "Sync 260122 kserve upstream odh" by @spolti in #1076
• Sync 260122 kserve upstream odh by @spolti in #1077
• Sync 260123 kserve upstream odh by @spolti in #1079
• Sync 260128 kserve upstream odh by @spolti in #1083
• Sync 260129 kserve upstream odh by @spolti in #1086
• [RHOAIENG-35346] Add ODH operator installation support with custom PR manifests for e2e test setup by @mholder6 in #1023
• Delete .tekton directory by @MohammadiIram in #1093
• Fix serverless-to-raw.sh bug where a null display name would cause an… by @andresllh in #1108
• add script to add hardwareprofile annotations to ignorelist by @VedantMahabaleshwarkar in #1109
• Sync 260130 kserve upstream odh by @spolti in #1088
• Sync 260216 kserve upstream odh by @spolti in #1123
• Revert "Sync 260216 kserve upstream odh" by @spolti in #1134
• Sync 260216 kserve upstream odh by @spolti in #1135
• Sync KServe/master into ODH/master by @spolti in #1142
• Port SCC logic from midstream/release-v0.15 to odh-master by @pierDipi in #1145
• Sync KServe/master into ODH/master by @spolti in #1143
• refactor(constants): extract midstream-specific constants into constants_odh.go by @bartoszmajsak in #1159
• port 1: Add LLMInferenceServiceConfig templates to ODH overlay with image replacements by @KillianGolds in #1147
• port 2: Add llminferenceservices to ODH user cluster roles by @KillianGolds in #1151
• port 3: inferenceservice-config-patch from release-v0.15 to master by @KillianGolds in #1152
• build(make): add Makefile.overrides.mk for midstream-only targets by @bartoszmajsak in #1150
• Sync 260304 kserve upstream odh by @spolti in #1155
• feat(llmisvc): add distro build tag hooks for platform-specific logic by @bartoszmajsak in #1157
• Sync 260310 kserve upstream odh by @spolti in #1174
• feat(llmisvc): add Istio DestinationRules and CA-signed certs via build tags by @bartoszmajsak in #1162
• RHOAIENG-50060: Create accelerator-specific LLMInferenceServiceConfig templates with distinguishing labels by @KillianGolds in #1130
• fix(llmisvc): enforce runAsNonRoot for LLMISvcConfig templates by @bartoszmajsak in #1182
• feat(llmisvc): add Prometheus metrics collection for LLMInferenceService by @israel-hdez in #1164
• feat(odh): enable TLS in LLMInferenceServiceConfig via overlay patches by @KillianGolds in #1177
• fix(ocp/rbac): isolate distro RBAC into dedicated sub-package by @bartoszmajsak in #1181
• feat: add llmisvc setup part for openshift ci by @Jooho in #1184
• fix(odh/rbac): adds missing list verb for DRs by @bartoszmajsak in #1187
• Sync 260311 kserve upstream odh by @spolti in #1183
• fix(odh/rbac): add llminferenceserviceconfigs to user cluster roles by @KillianGolds in #1189
• fix(kustomize): adjust manifests to work with ODH by @israel-hdez in #1171
• fix: add schedulerName to IBM Spyre s390x accelerator config by @KillianGolds in #1194
• fix(llmisvc): remove ibm-aiu-setup.sh sourcing from base template by @KillianGolds in #1200
• Add xKS overlay to master by @pierDipi in #1160
• make the odh manifests configurable (#1169) by @spolti in #1175
• [RHOAIENG-44975] CVE-2026-21441: urllib3 >= 2.6.3 in storage by @jlost in #1216
• Sync 260313 kserve upstream odh by @spolti in #1195
• Add scheduler annotation for backward compatibility by @pierDipi in #1217
• Sync dev branch into release branch by @spolti in #1218
• [release-v0.17] feat: enable scheduler TLS cert reload (kserve#5260) by @pierDipi in #1224
• Build and push llmisvc controller until ODH Konflux onboarding by @pierDipi in #1232
• Fix branch names by @pierDipi in #1235
• Use correct opendatahub repository on quay by @pierDipi in #1236
• fix: prevent flaky InferenceGraph auth deletion tests by @pierDipi in #1221
• Push llmisvc-controller to GHCR temporarily by @pierDipi in #1237
• add .tekton pipelines by @spolti in #1240
• fix(llmisvc): defer migration until webhooks are serving by @bartoszmajsak in #1251
• chore: remove no longer used odh-specific workflow by @israel-hdez in #1254
• RHOAIENG-54004: add recommended-accelerators annotation and fix description key on accelerator configs by @KillianGolds in #1249
• Enable ARM builds for llmisvc controller by @pierDipi in #1255
• fix: comment out non-existent build-odh-kserve-llmisvc-controller taskRunSpec by @jlost in #1252
• Fix CI setup for llmisvc by @andresllh in #1196
• Sync 0.17 from master by @pierDipi in #1266
• fix(llmisvc): match --enable-cert-reload=true flag form in cert hash skip by @bartoszmajsak in #1269
• fix(ci): inject KSERVE_NAMESPACE into E2E test environment by @bartoszmajsak in #1253
• Fix isvc e2e tests by @pierDipi in #1258
• Cleanup scc rolebinding on force stop by @pierDipi in #1260
• fix(llmisvc): autoscaling config logging by @Gregory-Pereira in #1289
• master from kserve:master by @pierDipi in #1264
• [pull] master from kserve:master by @pull[bot] in #1193
• ci: wire llmisvc-controller into Konflux group testing (release-v0.17) by @jlost in #1292
• Bump Tekton push image tags to 3.4 by @jlost in #1248
• RHOAIENG-48857: fix port conflict in multi-container probing test by @jlost in #1100
• feat(llmisvc): add WVA autoscaling config to ODH overlay by @vivekk16 in #1297
• chore: migrate ODH vars to v5.8 replacements [RHOAIENG-11436] by @jlost in #1149
• fix(llmisvc): update Spyre configs for continuous batching by @KillianGolds in #1312
• fix(build): pin setup-envtest to release-0.19 by @jlost in #1315
• fix: clusterStorageContainer image and resources in ODH overlay by @spolti in #1311
• CI: Tekton pipeline sync by @odh-devops-app[bot] in #1308
• drop vllm --> kserve_vllm metrics renaming by @Gregory-Pereira in #1299
• feat: add ONNX model class support for MLServer by @Jooho in #1317
• [release-v0.17] feat: add ONNX model class support for MLServer by @openshift-cherrypick-robot in #1333
• feat: isolate OpenShift localmodel changes behind distro build tags for modelcache by @VedantMahabaleshwarkar in #1220
• feat(rules): add CodeRabbit rules for clean merges by @bartoszmajsak in #1318
• Sync/upstream merge part1 by @mholder6 in #1332
• Revert "Merge pull request #1332..." by @mholder6 in #1348
• ci: add Konflux pull-request and group-test pipelines for master by @jlost in #1265
• fix(llmisvc): always set status.URL with a discovered address by @pierDipi in #1351
• Sync/20260402 upstream main by @mholder6 in #1352
• feat(llmisvc): add Intel Gaudi accelerator LLMInferenceServiceConfig by @KillianGolds in #1331
• [RHOAIENG-56198] fix: rebase group-test Dockerfiles to ubi9 by @jlost in #1355
• fix: dockerfile for llmisvc to run non root on xks in odh by @andresllh in #1363
• fix(ci): [RHOAIENG-56051] harden OLM CSV waits and Kuadrant/RHCL deploy by @jlost in #1301
• feat(llmisvc): propagate imagePullSecrets from default SA by @pierDipi in #1314
• feat(llmisvc): add customizeManagerOptions hook for distro cache config by @bartoszmajsak in #1192
• fix: add back the patch to remove clusterServingRuntime by @spolti in #1313
• feat: sync release v0.17 from odh/master by @pierDipi in #1296
• add back the patch to remove clusterServingRuntime by @spolti in #1372
• ClusterStorageContainer image and resources in ODH overlay (#1311) by @spolti in #1371
• ci: improve e2e test scripts for local runs and llmisvc builds by @jlost in #1356
• ci: guard against silent build breakage from build tag drift by @bartoszmajsak in #1370
• Sync/20260403 upstream main by @mholder6 in #1359
• Sync master to release v0.17 by @vivekk16 in #1388
• fix(llmisvc): re-add ibm-aiu-setup.sh sourcing by @KillianGolds in #1391
• Sync upstream to downstream (20260410) by @Jooho in #1392
• chore: sync master to release-v0.17 by @jlost in #1393
• ci: bind pipeline-logs workspace for group test by @maskarb in #1397
• chore: eliminate trivial drifts from upstream by @bartoszmajsak in #1398
• CI: Tekton pipeline sync by @odh-devops-app[bot] in #1408
• fix(llmisvc): remove cert-hash restart annotation from scheduler by @bartoszmajsak in #1406
• fix: odh-model-controller manifest replacement by @pierDipi in #1423
• fix: wrong kserve version + missing vllm llm-d parts by @zdtsw in #1362
• CI: Tekton pipeline sync by @odh-devops-app[bot] in #1424
• fix: preserve existing auth proxy containers during RHOAI upgrades by @VedantMahabaleshwarkar in #1407
• fix(llmisvc): extend storage migration retry window by @bartoszmajsak in #1412
• refactor: isolate zero-scale and auth annotation logic via build tags by @bartoszmajsak in #1324
• fix(llmisvc): align customizeManagerOptions hook to return error by @bartoszmajsak in #1374
• refactor(test): build-tag OCP scheme registrations and drop legacy SetupEnvTest by @bartoszmajsak in #1329
• RHOAIENG-58364: Order LLMInferenceServiceConfig last in ODH overlay for upgrade safety by @pierDipi in #1427
• chore: [RHOAIENG-59102] use Konflux :odh-stable tags in params.env by @jlost in #1428
• ci: use build SA for export-pipeline-logs by @maskarb in #1400
• fix(deployment): always add OAuth proxy to new raw deployments by @israel-hdez in #1293
• Sync release from master 20260423 0828 by @pierDipi in #1437
• Revert "Sync release from master 20260423 0828" by @jlost in #1438
• Sync release from master 20260423 0828 by @pierDipi in #1439
• ci: pass GOTAGS=distro build-arg in Konflux controller pipelines by @jlost in #1440

New Contributors

• @MohammadiIram made their first contribution in #1051
• @Gregory-Pereira made their first contribution in #1289
• @maskarb made their first contribution in #1397
• @zdtsw made their first contribution in #1362

Full Changelog: odh-v2.29...odh-v3.5-EA1