KAFKA-18628: Deprecate broker.id config (KIP-1232) by mingyen066 · Pull Request #22977 · apache/kafka

mingyen066 · 2026-07-28T15:55:24Z

Deprecate the broker.id configuration for removal in Kafka 5.0.
Log a deprecation warning at broker startup when broker.id is
explicitly set, pointing users to node.id.
Pass node.id to the tiered storage plugins alongside broker.id,
and switch Kafka's own RemoteStorageManager and
RemoteLogMetadataManager implementations to read it. broker.id is
still passed for compatibility and will be dropped in 5.0.
Reword the node.id / broker.id mismatch error so it points at
removing broker.id rather than keeping both in sync.
Clean up redundant broker.id settings in test files — node.id
alone is enough via the existing synonym mechanism.

The startup warning is checked on the raw properties in
KafkaConfig.fromProps, because populateSynonyms copies node.id
into broker.id, so originals() cannot tell whether the user actually
set it. It is not guarded by doLog, since the broker startup path
calls fromProps with doLog = false.

The tiered storage plugin change is not covered by the KIP yet — I will
amend KIP-1232 to include it.

Test result:

# broker.id=1
❯ bin/kafka-server-start.sh /tmp/kip1232.properties
[2026-07-28 23:49:14,050] INFO Registered
`kafka:type=kafka.Log4jController` MBean
(org.apache.kafka.server.logger.Log4jControllerRegistration)
[2026-07-28 23:49:14,114] WARN The 'broker.id' configuration is
deprecated and will be removed in Apache Kafka 5.0. Please use 'node.id'
instead. (kafka.server.KafkaConfig$)

# node.id=1
❯ bin/kafka-server-start.sh config/server.properties
[2026-07-28 23:52:45,352] INFO Registered
`kafka:type=kafka.Log4jController` MBean
(org.apache.kafka.server.logger.Log4jControllerRegistration)
[2026-07-28 23:52:45,461] INFO Registered signal handlers for TERM, INT,
HUP (org.apache.kafka.common.utils.internals.LoggingSignalHandler)
[2026-07-28 23:52:45,462] INFO [ControllerServer id=1] Starting
controller (kafka.server.ControllerServer)

Reviewers: Gaurav Narula gaurav_narula2@apple.com, Ken Huang
s7133700@gmail.com, Chia-Ping Tsai chia7712@gmail.com, Federico
Valeri fedevaleri@gmail.com

gaurav-narula

Thanks for the PR! Mostly good, a few comments

…smatch error, parameterise test against doLog

gaurav-narula

Thanks for the updates. LGTM!

m1a2st · 2026-07-29T15:37:34Z

  * When clients connect to the cluster, they now include cluster and node information to enable detection and handling of misrouted connections. For further details, please refer to [KIP-1242](https://cwiki.apache.org/confluence/x/W4LMFw).
  * The `kafka-cluster.sh` tool now provides an `api-versions` command to display the API versions supported by the brokers or controllers, and it accepts both `--bootstrap-server` and `--bootstrap-controller`. As a result, `kafka-broker-api-versions.sh` is deprecated and will be removed in the next major release; use `kafka-cluster.sh api-versions` instead. For further details, please refer to [KIP-1220](https://cwiki.apache.org/confluence/x/-QkbFw).
+  * The `broker.id` configuration is deprecated and will be removed in Kafka 5.0. Please use `node.id` instead. For further details, please refer to [KIP-1232](https://cwiki.apache.org/confluence/x/Hgp3Fw).
+  * Tiered storage plugins are now configured with `node.id` in addition to `broker.id`. Since `broker.id` will no longer be passed to `RemoteStorageManager` and `RemoteLogMetadataManager` implementations in Kafka 5.0, plugins should read `node.id` instead. For further details, please refer to [KIP-1232](https://cwiki.apache.org/confluence/x/Hgp3Fw).


Could you also update the KIP to reflect this change?

Thanks for reviewing! I've updated the KIP to cover the tiered storage plugin config: node.id is passed alongside broker.id in 4.4, and broker.id is dropped in 5.0.

m1a2st · 2026-07-29T15:47:22Z

+     *
+     * @deprecated Use {@link #NODE_ID} instead. This key is no longer passed to plugins from Kafka 5.0 (KIP-1232).
     */
+    @Deprecated(since = "4.4", forRemoval = true)


Could you explain why the test file needs @Deprecated?

Good catch. removed.

m1a2st · 2026-07-29T15:53:45Z

+     */
+    @Deprecated(since = "4.4", forRemoval = true)
    public static final String BROKER_ID = "broker.id";
+    public static final String NODE_ID = "node.id";


Could you also update the KIP to reflect this change?

Covered by the KIP update as well — TopicBasedRemoteLogMetadataManagerConfig.BROKER_ID is deprecated in 4.4 and removed in 5.0.

chia7712 · 2026-07-31T06:44:09Z

        initializationRetryIntervalMs = (long) parsedConfigs.get(REMOTE_LOG_METADATA_INITIALIZATION_RETRY_INTERVAL_MS_PROP);
        initializationRetryMaxTimeoutMs = (long) parsedConfigs.get(REMOTE_LOG_METADATA_INITIALIZATION_RETRY_MAX_TIMEOUT_MS_PROP);
-        clientIdPrefix = REMOTE_LOG_METADATA_CLIENT_PREFIX + "_" + props.get(BROKER_ID);
+        clientIdPrefix = REMOTE_LOG_METADATA_CLIENT_PREFIX + "_" + props.get(NODE_ID);


why don't we handle the deprecation?

Object nodeIdObj = props.get(NODE_ID); if (nodeIdObj == null) { nodeIdObj = props.get(BROKER_ID); if (nodeIdObj == null) { throw new ConfigException("Both '" + NODE_ID + "' and '" + BROKER_ID + "' are missing. Please configure '" + NODE_ID + "'."); } LOG.warn("The '{}' configuration in remote log metadata manager is deprecated and will be removed in Kafka 5.0. " + "Please use '{}' instead.", BROKER_ID, NODE_ID); } else if (props.containsKey(BROKER_ID)) { LOG.warn("The '{}' configuration is deprecated and will be ignored in Kafka 5.0. " + "Please use '{}' instead.", BROKER_ID, NODE_ID); } clientIdPrefix = REMOTE_LOG_METADATA_CLIENT_PREFIX + "_" + nodeIdObj;

Done, with ConfigException. I left out the else if (props.containsKey(BROKER_ID)) branch — RemoteLogManager passes both keys unconditionally, so it would warn on every startup about a key the operator didn't set.

chia7712 · 2026-07-31T06:46:51Z

        final String transfererClass = (String) configs.get(TRANSFERER_CLASS_CONFIG);
        final String isDeleteEnabled = (String) configs.get(ENABLE_DELETE_API_CONFIG);
-        final Integer brokerIdInt = (Integer) configs.get(BROKER_ID);
+        final Integer nodeIdInt = (Integer) configs.get(NODE_ID);


Done here too.

fvaleri · 2026-07-31T08:07:59Z

-     * The ID of the broker which owns this instance of {@link LocalTieredStorage}.
+     * The ID of the node which owns this instance of {@link LocalTieredStorage}.
     */
-    public static final String BROKER_ID = "broker.id";


Should we also deprecate this one in 4.4 and remove only in 5.0? External consumers of Kafka test JAR would get a compile error.

Restored as deprecated, thanks.

Follow the deprecation cycle rather than switching the key outright: TopicBasedRemoteLogMetadataManagerConfig and LocalTieredStorage read node.id and fall back to broker.id with a warning, so callers that construct them directly keep working through 4.x. Both now fail fast when neither key is present; previously a missing id was silently stringified into the client id prefix. Restore LocalTieredStorage.BROKER_ID as deprecated instead of removing it, since the test-fixtures jar is published and external code may compile against it. Document node.id in the RemoteStorageManager and RemoteLogMetadataManager javadoc, which is the only written form of the plugin contract.

fvaleri

LGTM. Thanks.

chia7712

LGTM, and the remaining issues could be fixed by the follow-up PR

chia7712 · 2026-07-31T16:02:40Z

-            throw new InvalidConfigurationException(
-                    "Broker ID is required to configure the LocalTieredStorage manager.");
+        if (nodeIdInt == null) {
+            throw new InvalidConfigurationException(format(


This check could be moved to nodeId, and then nodeId could return int type instead of Integer

KAFKA-18628: Deprecate broker.id config (KIP-1232)

025293c

github-actions Bot added triage PRs from the community core Kafka Broker performance storage Pull requests that target the storage module tiered-storage Related to the Tiered Storage feature small Small PRs labels Jul 28, 2026

gaurav-narula reviewed Jul 28, 2026

View reviewed changes

github-actions Bot removed the triage PRs from the community label Jul 29, 2026

Address review: pass node.id to tiered storage plugins, clarify id mi…

522becf

…smatch error, parameterise test against doLog

github-actions Bot removed the small Small PRs label Jul 29, 2026

gaurav-narula approved these changes Jul 29, 2026

View reviewed changes

m1a2st reviewed Jul 29, 2026

View reviewed changes

Remove the now-unused broker.id constant from LocalTieredStorage

18cd1ae

chia7712 reviewed Jul 31, 2026

View reviewed changes

fvaleri reviewed Jul 31, 2026

View reviewed changes

fvaleri approved these changes Jul 31, 2026

View reviewed changes

chia7712 approved these changes Jul 31, 2026

View reviewed changes

chia7712 merged commit 44e4584 into apache:trunk Jul 31, 2026
27 checks passed

Uh oh!

Conversation

mingyen066 commented Jul 28, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gaurav-narula left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gaurav-narula left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fvaleri left a comment

Choose a reason for hiding this comment

Uh oh!

chia7712 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mingyen066 commented Jul 28, 2026 •

edited by github-actions Bot

Loading