🐛 Bug Report: MaxListenersExceededWarning on every response since 3.17.0 (instrumentation-router adds a finish listener per router layer)

[Graphmaxer]

📜 Description

Since 3.17.0, every HTTP response on the self-hosted api, worker and ws services emits:

(node:1) MaxListenersExceededWarning: Possible EventEmitter memory leak detected.
11 finish listeners added to [ServerResponse]. MaxListeners is 10.
Use emitter.setMaxListeners() to increase limit

The cause is @opentelemetry/instrumentation-router, which registers a prependOnceListener('finish') for every router layer a request traverses. Any request crossing more than ~10 layers passes Node's default maxListeners of 10.

This is not an actual memory leak — the listeners are attached per ServerResponse and released with it — but it produces a very large volume of useless log output on any deployment with ENABLE_OTEL=true. In our integration environment it accounted for roughly a 49% increase in the total log volume of the Novu services.

3.15.0 was unaffected with the identical ENABLE_OTEL=true configuration. Two changes in between look relevant:

• @opentelemetry/instrumentation-router 0.56.0 → 0.61.0
• NestJS 10.4.18 → 11.1.21 in 3.17.0, across api, worker and ws — precisely the three affected services

Note this is the router instrumentation, not the express one, so the ignoreLayersType: ['middleware'] configuration added to otel-init.ts for #10722 does not suppress it, nor does OTEL_NODE_DISABLED_INSTRUMENTATIONS=express.

👟 Reproduction steps

1. Run ghcr.io/novuhq/novu/api:3.18.0 with MongoDB and a Redis/Valkey reachable.
2. Set ENABLE_OTEL=true and OTEL_EXPORTER_OTLP_ENDPOINT to any endpoint (a collector need not actually be listening).
3. curl http://localhost:3000/v1/health-check
4. One MaxListenersExceededWarning is logged per request.

🕵️ Attribution of the listeners

Running the same image with NODE_OPTIONS=--require probe.js, where probe.js patches http.ServerResponse.prototype.{on,once,addListener,prependListener,prependOnceListener} and records the registering module per response:

=== 11 finish listeners on ONE ServerResponse ===
   1. node internal: node:_http_server                 [.on()]
   2. node internal: node:events                       [.prependListener()]
   3. @opentelemetry/instrumentation-router @ 0.61.0   [.prependOnceListener()]
   4. node internal: node:events                       [.prependListener()]
   5. @opentelemetry/instrumentation-router @ 0.61.0   [.prependOnceListener()]
   6. node internal: node:events                       [.prependListener()]
   7. @opentelemetry/instrumentation-router @ 0.61.0   [.prependOnceListener()]
   ... pattern continues past 19 on this route

Each node:events entry is the OpenTelemetry context manager's bound wrapper around the preceding registration.

👍 Expected behavior

Enabling OpenTelemetry should not emit a MaxListenersExceededWarning per request.

👎 Actual behavior

One warning per HTTP response on every OTel-enabled Node service.

✅ Evidence this is not a real leak

• Every occurrence reports exactly 11 listeners at the trigger point — a constant overshoot, never an escalating count.
• Container working set stays flat: api 427 → 429 MiB against an 8 GiB limit, worker 265 → 266 MiB, ws 290 → 292 MiB, over hours of traffic.
• Zero restarts, no OOM kills.

🔧 Workaround

Adding router to the disabled instrumentations removes it. Verified on api:3.18.0 over 5 requests to /v1/health-check:

OTEL_NODE_DISABLED_INSTRUMENTATIONS	Warnings
express	4
express,router	0 — boots clean, all requests 200

💡 Possible fixes upstream

1. Configure @opentelemetry/instrumentation-router in libs/application-generic/src/tracing/otel-init.ts the way instrumentation-express already is, so it does not attach a listener per layer.
2. Or raise the limit for response objects once during OTel setup (res.setMaxListeners(n) / events.setMaxListeners), which silences the warning without dropping the spans.
3. Or disable router by default for self-hosted, alongside the existing express handling.

Environment

• Novu 3.18.0, self-hosted (IS_SELF_HOSTED=true), official ghcr.io/novuhq/novu/* images
• Node 20 (image default), Kubernetes, ENABLE_OTEL=true exporting OTLP/HTTP
• Also reproduced locally with plain Docker, so it is not Kubernetes-specific

[linear-code]

NV-8574

[talhadevelopes]

Hey @Graphmaxer, nice write-up, the listener attribution trace makes this easy to act on.

I looked into the three options in otel-init.ts:

• Option 1 (mirror the express ignoreLayersType config) isn't actually available for router - @opentelemetry/instrumentation-router@0.61.0 only accepts base InstrumentationConfig, no ignoreLayersType/ignoreLayers API. It unconditionally does res.prependOnceListener('finish', endSpan) per layer in _setupSpan, so this would need an upstream fix in the instrumentation package itself, not something we can configure here.
• Option 3 (disable router instrumentation) fits the codebase's existing pattern - fs, dns, and mongoose are already disabled the same way in the same getNodeAutoInstrumentations() block. Express instrumentation already covers the useful route-handler spans per the existing comment in that file, so router's spans are redundant on top of that.

I'd propose a one-line fix:

'@opentelemetry/instrumentation-router': { enabled: false },

alongside the other disabled instrumentations in otel-init.ts.

Happy to open a PR with this plus a unit test if that sounds right, let me know.