feat: add JSON Stream (NDJSON) body processor

[fzipi]

what

Implements a new body processor for handling streaming JSON formats with per-record rule evaluation:

• NDJSON (Newline Delimited JSON)
• JSON Lines
• JSON Sequence (RFC 7464)

Streaming body processor

• Line-by-line processing for memory efficiency
• Each JSON object indexed by record number (json.0.field, json.1.field)
• Built-in DoS protection with 1024 recursion limit
• Support for nested objects and arrays
• Auto-detection of format (NDJSON vs RFC 7464) by peeking at the first 4KB
• Registered under JSONSTREAM, NDJSON, and JSONLINES aliases

Per-record rule evaluation

Instead of parsing all records into ArgsPost and evaluating Phase 2 rules once, rules are now evaluated after each complete JSON record:

• Malicious records are caught immediately without processing the rest of the stream
• ArgsPost is cleared between records so each record is evaluated in isolation
• TX variables (e.g. anomaly scores) persist across records, enabling cross-record correlation — a stream where 3 records each score below the threshold individually can
  still trigger a block when the accumulated score exceeds it
• Eval() is safe to call multiple times per phase — AllowTypePhase, Skip, and the transformation cache all reset correctly between calls
• Non-streaming body processors are completely unaffected

New StreamingBodyProcessor interface

Extends BodyProcessor with ProcessRequestRecords/ProcessResponseRecords methods that yield parsed records one at a time via callback. The transaction detects this
interface via type assertion and switches to the per-record evaluation path automatically.

Streaming relay support

Added ProcessRequestBodyFromStream(input, output) / ProcessResponseBodyFromStream(input, output) methods on the transaction for integrators building custom streaming
middleware. These read records from input, evaluate rules per record, and write clean records to output. Exposed via an experimental StreamingTransaction interface.

Usage example

SecRule REQUEST_HEADERS:Content-Type "^application/x-ndjson" \
    "id:'200007',phase:1,pass,nolog,ctl:requestBodyProcessor=JSONSTREAM"

Testing

• 24 unit tests for the body processor (single/multiple lines, nested objects, arrays, error cases, recursion limits, TX variable storage, large tokens, format
  auto-detection)
• 7 unit tests for callback-based record processing (interruption stops processing, field prefixes, RFC 7464, backward compatibility)
• 4 integration tests with real WAF rules (interruption at bad record, clean passthrough, TX variable accumulation across records, below-threshold no-block)
• Benchmark: ~5,000 ops/sec for 100-object streams

Benchmark Results (Apple M2)

ProcessRequest (buffered) vs Callback (streaming)

Scenario	Records	ProcessRequest	Callback	Throughput Speedup	Alloc Reduction
small	1	2.00 MB/s, 146 allocs	2.66 MB/s, 17 allocs	1.3x	88%
small	10	11.30 MB/s, 283 allocs	16.70 MB/s, 134 allocs	1.5x	53%
small	100	22.38 MB/s, 1,644 allocs	33.01 MB/s, 1,304 allocs	1.5x	21%
small	1,000	22.84 MB/s, 16,671 allocs	36.20 MB/s, 14,493 allocs	1.6x	13%
medium	1	8.50 MB/s, 176 allocs	11.92 MB/s, 39 allocs	1.4x	78%
medium	10	26.20 MB/s, 579 allocs	41.03 MB/s, 354 allocs	1.6x	39%
medium	100	31.71 MB/s, 4,563 allocs	52.34 MB/s, 3,505 allocs	1.7x	23%
medium	1,000	31.54 MB/s, 51,097 allocs	53.93 MB/s, 41,712 allocs	1.7x	18%
nested	1	8.53 MB/s, 178 allocs	12.04 MB/s, 41 allocs	1.4x	77%
nested	10	25.22 MB/s, 599 allocs	38.13 MB/s, 374 allocs	1.5x	38%
nested	100	30.71 MB/s, 4,763 allocs	48.66 MB/s, 3,705 allocs	1.6x	22%
nested	1,000	30.49 MB/s, 53,101 allocs	48.92 MB/s, 43,713 allocs	1.6x	18%

RFC 7464 (JSON Sequence) via Callback

Scenario	Records	Throughput	Allocs/op
small	10	16.61 MB/s	134
small	100	34.37 MB/s	1,304
medium	100	47.02 MB/s	3,505
nested	100	48.06 MB/s	3,705

Key Takeaways

• The callback-based streaming path is consistently 1.5–1.7x faster in throughput.
• Allocation counts are 13–88% lower (most dramatic at low record counts where per-collection overhead dominates).
• RFC 7464 format performance is comparable to NDJSON at the same record counts, confirming negligible format auto-detection overhead.
• The callback path avoids populating TransactionVariables collections (ArgsPost, TX vars, raw body via TeeReader), which accounts for the reduced allocations.

Record Templates

• small: {"id":1,"name":"Alice"} (24 bytes)
• medium: {"user_id":1234567890,"name":"User Name","email":"user@example.com","role":"admin","active":true,"tags":["tag1","tag2","tag3"]} (128 bytes)
• nested: {"user":{"name":"Alice","address":{"city":"NYC","zip":"10001"}},"scores":[95,87,92],"meta":{"created":"2026-01-01","active":true}} (131 bytes)

Summary by CodeRabbit

Release Notes

• New Features

  • Added streaming JSON body processor supporting NDJSON, JSON Lines, and RFC 7464 JSON Sequence formats with per-record security rule evaluation and automatic termination on malicious records

• Documentation

  • Added configuration examples and guidance for enabling the JSON stream request body processor with recommended content type settings

[codecov]

Codecov Report

❌ Patch coverage is 52.69461% with 158 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.07%. Comparing base (537abf9) to head (d4502d1).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/corazawaf/transaction.go	10.17%	148 Missing and 2 partials ⚠️
experimental/bodyprocessors/jsonstream.go	95.15%	4 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1481      +/-   ##
==========================================
- Coverage   85.30%   84.07%   -1.24%     
==========================================
  Files         174      175       +1     
  Lines        8461     8811     +350     
==========================================
+ Hits         7218     7408     +190     
- Misses        994     1146     +152     
- Partials      249      257       +8     

Flag	Coverage Δ
coraza.rule.case_sensitive_args_keys	84.04% <52.69%> (-1.24%)	⬇️
coraza.rule.mandatory_rule_id_check	84.06% <52.69%> (-1.24%)	⬇️
coraza.rule.multiphase_evaluation	83.80% <52.69%> (-1.23%)	⬇️
coraza.rule.no_regex_multiline	84.06% <52.69%> (-1.24%)	⬇️
default	84.07% <52.69%> (-1.24%)	⬇️
examples+	21.03% <24.55%> (+4.54%)	⬆️
examples+coraza.rule.case_sensitive_args_keys	83.97% <52.69%> (-1.23%)	⬇️
examples+coraza.rule.mandatory_rule_id_check	84.06% <52.69%> (-1.24%)	⬇️
examples+coraza.rule.multiphase_evaluation	83.60% <52.69%> (-1.22%)	⬇️
examples+coraza.rule.no_regex_multiline	83.91% <52.69%> (-1.23%)	⬇️
examples+memoize_builders	84.01% <52.69%> (-1.24%)	⬇️
examples+no_fs_access	81.75% <52.69%> (-1.14%)	⬇️
ftw	84.07% <52.69%> (-1.24%)	⬇️
memoize_builders	84.18% <52.69%> (-1.25%)	⬇️
no_fs_access	83.61% <52.69%> (-1.22%)	⬇️
tinygo	84.05% <52.69%> (-1.24%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR adds support for JSON Stream (NDJSON) body processing to Coraza WAF, enabling line-by-line processing of streaming JSON formats. The implementation includes a new body processor that handles NDJSON, JSON Lines, and claims support for JSON Sequence (RFC 7464).

Changes:

• New jsonStreamBodyProcessor that processes JSON objects line-by-line with memory-efficient streaming
• Built-in DoS protection via configurable recursion limits (default 1024)
• TX variable storage for raw body and line count to enable custom validation rules
• Configuration rules in coraza.conf-recommended for NDJSON content types

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File	Description
internal/bodyprocessors/jsonstream.go	Core implementation of NDJSON body processor with line-by-line parsing, recursion limits, and TX variable storage
internal/bodyprocessors/jsonstream_test.go	Comprehensive test suite with 13 test cases covering single/multiple lines, nested objects, arrays, error cases, and benchmarks
coraza.conf-recommended	Configuration rules for enabling NDJSON processing based on Content-Type headers, with optional line count limiting

[jcchavezs]

Could we add this to experimental?

[fzipi]

I thought I mentioned this. Yes, that was my idea.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

[fzipi]

@copilot Fix the conflicts first. Then add e2e tests to tests for this new body processor.

[Copilot]

@fzipi I've opened a new pull request, #1563, to work on those changes. Once the pull request is ready, I'll request review from you.

[coderabbitai]

📝 Walkthrough

Walkthrough

Introduced streaming JSON body processor support enabling per-record evaluation of NDJSON/JSON Lines/RFC 7464 request and response bodies. Added StreamingBodyProcessor interface for record-level callbacks, implemented jsonStreamBodyProcessor with auto-format detection, and integrated streaming paths into core transaction processing via new ProcessRequestBodyFromStream/ProcessResponseBodyFromStream APIs.

Changes

Cohort / File(s)	Summary
Configuration & Documentation coraza.conf-recommended	Added commented-out SecRule examples and notes for enabling JSONSTREAM request body processor for NDJSON/JSON Lines/JSON Sequence content types and configuring response body inspection.
Streaming Body Processor Implementation experimental/bodyprocessors/jsonstream.go	Implemented jsonStreamBodyProcessor with RFC 7464 record-separator auto-detection, per-record JSON parsing/flattening, recursion limit enforcement, TX variable buffering, and callback-driven record processing. Registered processor aliases: jsonstream, ndjson, jsonlines.
Streaming Processor Tests experimental/bodyprocessors/jsonstream_test.go	Added 24 test functions covering NDJSON/RFC 7464 formats, nested objects/arrays, empty-line handling, invalid JSON, recursion limits, TX variable output, processor registration, and streaming callbacks with per-record field prefixing. Includes benchmarks for processor and callback paths.
Plugin & Interface Definitions experimental/plugins/bodyprocessors.go, experimental/plugins/plugintypes/bodyprocessor.go, experimental/streaming.go	Introduced GetBodyProcessor() lookup function, StreamingBodyProcessor interface with ProcessRequestRecords/ProcessResponseRecords methods, and StreamingTransaction interface with ProcessRequestBodyFromStream/ProcessResponseBodyFromStream streaming APIs.
Transaction Streaming Implementation internal/corazawaf/transaction.go	Refactored ProcessRequestBody() and added ProcessResponseBody() streaming path detection; implemented ProcessRequestBodyFromStream() and ProcessResponseBodyFromStream() with per-record rule evaluation (Phase 2/4), argsPost/responseArgs clearing, rawRecord output writing, and errStreamInterrupted error handling for rule termination.
Middleware Documentation http/middleware.go	Added inline comment explaining Phase 2 rule evaluation differences between streaming and standard body processors.
Build & Testing Infrastructure magefile.go, streaming_integration_test.go, testing/e2e/ndjson_e2e_test.go, testing/auditlog_test.go	Added TagsMatrix() mage target for build-tag permutation generation; integration tests validating per-record interruption and TX variable persistence; E2E tests covering clean/malicious NDJSON records, SQLi detection, content-type routing, and empty-line tolerance.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant HTTP Middleware
    participant WAF Transaction
    participant Body Processor
    participant Rules Engine
    
    Client->>HTTP Middleware: POST with NDJSON body
    HTTP Middleware->>WAF Transaction: ProcessRequestBodyFromStream(reader, writer)
    
    activate WAF Transaction
    WAF Transaction->>WAF Transaction: Detect StreamingBodyProcessor
    loop For each record in stream
        WAF Transaction->>Body Processor: ProcessRequestRecords callback
        activate Body Processor
        Body Processor->>Body Processor: Parse JSON record
        Body Processor->>Body Processor: Flatten to fields
        deactivate Body Processor
        
        WAF Transaction->>WAF Transaction: Clear argsPost
        WAF Transaction->>WAF Transaction: Populate argsPost from fields
        WAF Transaction->>Rules Engine: Evaluate Phase 2 (RequestBody)
        activate Rules Engine
        Rules Engine-->>WAF Transaction: Decision (allow/deny)
        deactivate Rules Engine
        
        alt Rule interrupted
            WAF Transaction->>WAF Transaction: Set errStreamInterrupted
            WAF Transaction-->>Client: Early termination with 403
        else Passed
            WAF Transaction->>Client: Write rawRecord to output
        end
    end
    deactivate WAF Transaction
    
    Client->>Client: Receive response

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Suggested reviewers

• jcchavezs

Poem

🐰 Records cascade like clover through streams,
JSON lines flowing in procedural dreams,
Per-record verdicts, swift and lean,
A hopping processor unseen!
✨ Streaming rules at last take flight,
Coraza bounds toward new heights! 🚀

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 28.57% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title 'feat: add JSON Stream (NDJSON) body processor' directly and accurately describes the main change: the addition of a new JSON Stream body processor supporting NDJSON format.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/add-streaming-json-processor

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 10

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@coraza.conf-recommended`:
- Around line 120-122: The response MIME guidance only lists application/json
and application/x-ndjson but should also include the other documented JSON
stream types; update the note in coraza.conf-recommended to mention
application/jsonlines and application/json-seq (in addition to application/json
and application/x-ndjson) so response-inspection guidance matches the request
examples and covers all supported JSON stream MIME types.

In `@experimental/bodyprocessors/jsonstream.go`:
- Around line 175-190: The scanRecords function currently trims and reformats
the raw input before passing it to the consumer, causing canonicalization
(changing delimiters, CRLF vs LF, final newlines, and outer whitespace); instead
preserve and relay the original raw bytes/text verbatim. Modify scanRecords (and
the similar block at the later occurrence) to capture the unmodified
scanner.Text() (or the raw token) as rawRecord and pass that rawRecord unchanged
into fn rather than formatRecord(record); continue to use formatRecord only
where a formatted version is explicitly required (e.g., for parsing via
parseJSONRecord), ensuring ProcessRequestBodyFromStream and
ProcessResponseBodyFromStream receive the original rawRecord exactly as read.

In `@experimental/plugins/plugintypes/bodyprocessor.go`:
- Around line 49-56: Streaming JSON processors currently ignore the provided
BodyProcessorOptions and always use DefaultStreamRecursionLimit; update the JSON
stream implementation (experimental/bodyprocessors/jsonstream.go) used by
ProcessRequestRecords and ProcessResponseRecords to read the recursion limit
from the incoming options (e.g. options.RecursionLimit or the actual field name
on BodyProcessorOptions) and pass that value into the streaming JSON
parser/decoder instead of the hardcoded DefaultStreamRecursionLimit, falling
back to DefaultStreamRecursionLimit only when the options value is zero/unset so
configured recursion limits are honored in streaming mode.

In `@internal/corazawaf/transaction.go`:
- Around line 1235-1313: The streaming-relay path (around
bodyprocessors.GetBodyProcessor and sp.ProcessRequestRecords) lacks the same
guards as the buffered path and can bypass RequestBodyAccess /
ResponseBodyAccess, IsResponseBodyProcessable(), and
RequestBodyLimit/ResponseBodyLimit checks; mirror the checks used by
ReadRequestBodyFrom and ProcessRequestBody before calling
sp.ProcessRequestRecords: verify request/response body access flags
(RequestBodyAccess/ResponseBodyAccess), call IsResponseBodyProcessable() where
appropriate, enforce RequestBodyLimit/ResponseBodyLimit and
generateRequestBodyError + tx.WAF.Rules.Eval(...) on violations, and follow the
same fallback behavior as the non-streaming branch (use tx.ReadRequestBodyFrom /
tx.ProcessRequestBody semantics) so streaming processors cannot bypass limits or
disabled access.
- Around line 1138-1141: The streaming branch in transaction.go returns from
processRequestBodyStreaming/processResponseBodyStreaming when bodyprocessor
implements plugintypes.StreamingBodyProcessor, which skips calling the
processor's ProcessRequest/ProcessResponse that populate metadata (e.g.,
TX:jsonstream_*_body and TX:jsonstream_*_line_count) in
experimental/bodyprocessors/jsonstream.go; update the streaming path to either
call jsonStreamBodyProcessor's metadata-populating routines or explicitly invoke
the processor's ProcessRequest/ProcessResponse (or a shared helper used by
Process*Body) before/after
processRequestBodyStreaming/processResponseBodyStreaming so the same TX metadata
is emitted regardless of using StreamingBodyProcessor, keeping references to
bodyprocessor, StreamingBodyProcessor, ProcessRequest, ProcessResponse,
processRequestBodyStreaming and processResponseBodyStreaming.
- Around line 1333-1348: The code currently logs non-interruption streaming
errors (streamErr != nil && streamErr != errStreamInterrupted), sets
REQBODY/RESBODY via tx.generateRequestBodyError and tx.WAF.Rules.Eval, but then
falls through and returns (tx.interruption, nil) which hides relay I/O failures;
change both such blocks (the one around tx.generateRequestBodyError and the
similar block at the later mirror) to return the streaming error to the caller
after handling: i.e., after logging, calling tx.generateRequestBodyError and
possibly tx.WAF.Rules.Eval, return (tx.interruption, streamErr) instead of
allowing the function to return nil error so reader-side I/O failures are
propagated.

In `@magefile.go`:
- Around line 279-287: The tags slice used to build the tags matrix (the
variable named tags passed into combinations(tags)) is missing
"coraza.rule.rx_prefilter", causing the generated TagsMatrix to be stale; add
"coraza.rule.rx_prefilter" to that slice and then ensure TagsMatrix is the
single source of truth by either deriving the slice from the canonical build tag
list or updating the external workflow to read from the same central constant so
they cannot drift apart.
- Line 289: The code calls json.Marshal (see json.Marshal(combos)) but the
encoding/json package is not imported; add the import "encoding/json" to the
import block (where other stdlib imports live) so json.Marshal resolves, then
run gofmt/goimports to ensure imports are organized; update any existing import
grouping to include encoding/json alongside other standard library imports.

In `@testing/e2e/ndjson_e2e_test.go`:
- Around line 169-170: The test currently allows either http.StatusOK or
http.StatusForbidden for non-NDJSON content which is too permissive; update the
assertion in the test (replace the condition that checks resp.StatusCode against
http.StatusOK && http.StatusForbidden) to assert the single expected status code
(e.g., resp.StatusCode == http.StatusOK) and change the t.Errorf call to report
the single expected value (use resp.StatusCode and http.StatusOK in the
message); locate and modify the check referencing resp.StatusCode,
http.StatusOK, http.StatusForbidden and t.Errorf in the ndjson_e2e_test.go test
function.
- Around line 41-43: The test HTTP handler currently discards the error from
io.ReadAll(r.Body); change the handler to check the error returned by
io.ReadAll, and if non-nil respond with an appropriate failure status (e.g.,
w.WriteHeader(http.StatusInternalServerError)) and a descriptive message (and/or
t.Fatalf/log) instead of proceeding to write "OK". Locate the body-reading logic
that calls io.ReadAll(r.Body) and update error handling around that call and the
subsequent w.WriteHeader/fmt.Fprintf usage so test failures surface server-side
read errors.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 424d4e00-37aa-42e9-93d8-3b8ad2ebcabb

📥 Commits

Reviewing files that changed from the base of the PR and between 4759286 and 5a61361.

📒 Files selected for processing (12)

• coraza.conf-recommended
• experimental/bodyprocessors/jsonstream.go
• experimental/bodyprocessors/jsonstream_test.go
• experimental/plugins/bodyprocessors.go
• experimental/plugins/plugintypes/bodyprocessor.go
• experimental/streaming.go
• http/middleware.go
• internal/corazawaf/transaction.go
• magefile.go
• streaming_integration_test.go
• testing/auditlog_test.go
• testing/e2e/ndjson_e2e_test.go

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Response MIME guidance is incomplete for the documented JSON stream types.

The note mentions only application/json and application/x-ndjson, but nearby request examples also document application/jsonlines and application/json-seq. This can lead to partial response-inspection configuration.

📝 Suggested wording update

-# Note: Add 'application/json' and 'application/x-ndjson' if you want to
-# inspect JSON and NDJSON response bodies
+# Note: Add 'application/json', 'application/x-ndjson', 'application/jsonlines',
+# and 'application/json-seq' if you want to inspect JSON and JSON-stream response bodies

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	# Note: Add 'application/json' and 'application/x-ndjson' if you want to
	# inspect JSON and NDJSON response bodies
	#
	# Note: Add 'application/json', 'application/x-ndjson', 'application/jsonlines',
	# and 'application/json-seq' if you want to inspect JSON and JSON-stream response bodies
	#

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@coraza.conf-recommended` around lines 120 - 122, The response MIME guidance
only lists application/json and application/x-ndjson but should also include the
other documented JSON stream types; update the note in coraza.conf-recommended
to mention application/jsonlines and application/json-seq (in addition to
application/json and application/x-ndjson) so response-inspection guidance
matches the request examples and covers all supported JSON stream MIME types.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

rawRecord is being canonicalized before relay.

scanner.Text() drops the original delimiter, TrimSpace strips outer whitespace, and formatNDJSON/formatJSONSequence synthesize \n/RS again. ProcessRequestBodyFromStream and ProcessResponseBodyFromStream write rawRecord verbatim, so CRLF vs LF, missing final newlines, and any preserved outer whitespace are silently changed on the wire.

Also applies to: 208-214

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@experimental/bodyprocessors/jsonstream.go` around lines 175 - 190, The
scanRecords function currently trims and reformats the raw input before passing
it to the consumer, causing canonicalization (changing delimiters, CRLF vs LF,
final newlines, and outer whitespace); instead preserve and relay the original
raw bytes/text verbatim. Modify scanRecords (and the similar block at the later
occurrence) to capture the unmodified scanner.Text() (or the raw token) as
rawRecord and pass that rawRecord unchanged into fn rather than
formatRecord(record); continue to use formatRecord only where a formatted
version is explicitly required (e.g., for parsing via parseJSONRecord), ensuring
ProcessRequestBodyFromStream and ProcessResponseBodyFromStream receive the
original rawRecord exactly as read.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

BodyProcessorOptions is exposed but recursion limit is effectively bypassed in streaming implementations.

ProcessRequestRecords/ProcessResponseRecords receive options, but current JSON stream implementations still hardcode DefaultStreamRecursionLimit, so configured recursion limits are not enforced in streaming mode.

💡 Proposed fix (in experimental/bodyprocessors/jsonstream.go)

-func (js *jsonStreamBodyProcessor) ProcessRequestRecords(reader io.Reader, _ plugintypes.BodyProcessorOptions,
+func (js *jsonStreamBodyProcessor) ProcessRequestRecords(reader io.Reader, options plugintypes.BodyProcessorOptions,
 	fn func(recordNum int, fields map[string]string, rawRecord string) error) error {
-	recordCount, err := processJSONStreamWithCallback(reader, DefaultStreamRecursionLimit, fn)
+	recursionLimit := options.RequestBodyRecursionLimit
+	if recursionLimit <= 0 {
+		recursionLimit = DefaultStreamRecursionLimit
+	}
+	recordCount, err := processJSONStreamWithCallback(reader, recursionLimit, fn)
 	if err != nil {
 		return err
 	}

-func (js *jsonStreamBodyProcessor) ProcessResponseRecords(reader io.Reader, _ plugintypes.BodyProcessorOptions,
+func (js *jsonStreamBodyProcessor) ProcessResponseRecords(reader io.Reader, options plugintypes.BodyProcessorOptions,
 	fn func(recordNum int, fields map[string]string, rawRecord string) error) error {
-	recordCount, err := processJSONStreamWithCallback(reader, DefaultStreamRecursionLimit, fn)
+	recursionLimit := options.RequestBodyRecursionLimit
+	if recursionLimit <= 0 {
+		recursionLimit = DefaultStreamRecursionLimit
+	}
+	recordCount, err := processJSONStreamWithCallback(reader, recursionLimit, fn)
 	if err != nil {
 		return err
 	}

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@experimental/plugins/plugintypes/bodyprocessor.go` around lines 49 - 56,
Streaming JSON processors currently ignore the provided BodyProcessorOptions and
always use DefaultStreamRecursionLimit; update the JSON stream implementation
(experimental/bodyprocessors/jsonstream.go) used by ProcessRequestRecords and
ProcessResponseRecords to read the recursion limit from the incoming options
(e.g. options.RecursionLimit or the actual field name on BodyProcessorOptions)
and pass that value into the streaming JSON parser/decoder instead of the
hardcoded DefaultStreamRecursionLimit, falling back to
DefaultStreamRecursionLimit only when the options value is zero/unset so
configured recursion limits are honored in streaming mode.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Auto-streaming skips the processor’s metadata population.

Once these branches take Process*Records, jsonStreamBodyProcessor.ProcessRequest / ProcessResponse never run, so TX:jsonstream_*_body and TX:jsonstream_*_line_count from experimental/bodyprocessors/jsonstream.go are absent on the normal buffered Process*Body() path. That makes rule behavior depend on which transaction API was used.

Also applies to: 1665-1668

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@internal/corazawaf/transaction.go` around lines 1138 - 1141, The streaming
branch in transaction.go returns from
processRequestBodyStreaming/processResponseBodyStreaming when bodyprocessor
implements plugintypes.StreamingBodyProcessor, which skips calling the
processor's ProcessRequest/ProcessResponse that populate metadata (e.g.,
TX:jsonstream_*_body and TX:jsonstream_*_line_count) in
experimental/bodyprocessors/jsonstream.go; update the streaming path to either
call jsonStreamBodyProcessor's metadata-populating routines or explicitly invoke
the processor's ProcessRequest/ProcessResponse (or a shared helper used by
Process*Body) before/after
processRequestBodyStreaming/processResponseBodyStreaming so the same TX metadata
is emitted regardless of using StreamingBodyProcessor, keeping references to
bodyprocessor, StreamingBodyProcessor, ProcessRequest, ProcessResponse,
processRequestBodyStreaming and processResponseBodyStreaming.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Mirror the buffered-path guards before streaming relay.

These entry points never apply the same RequestBodyAccess / ResponseBodyAccess, IsResponseBodyProcessable(), or body-limit handling that ProcessRequestBody() / ProcessResponseBody() and Read*BodyFrom() enforce. That lets streaming processors bypass RequestBodyLimit / ResponseBodyLimit, and when body access is off the non-streaming fallback relays an empty buffer instead of the original stream.

Also applies to: 1355-1412

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@internal/corazawaf/transaction.go` around lines 1235 - 1313, The
streaming-relay path (around bodyprocessors.GetBodyProcessor and
sp.ProcessRequestRecords) lacks the same guards as the buffered path and can
bypass RequestBodyAccess / ResponseBodyAccess, IsResponseBodyProcessable(), and
RequestBodyLimit/ResponseBodyLimit checks; mirror the checks used by
ReadRequestBodyFrom and ProcessRequestBody before calling
sp.ProcessRequestRecords: verify request/response body access flags
(RequestBodyAccess/ResponseBodyAccess), call IsResponseBodyProcessable() where
appropriate, enforce RequestBodyLimit/ResponseBodyLimit and
generateRequestBodyError + tx.WAF.Rules.Eval(...) on violations, and follow the
same fallback behavior as the non-streaming branch (use tx.ReadRequestBodyFrom /
tx.ProcessRequestBody semantics) so streaming processors cannot bypass limits or
disabled access.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Return relay I/O failures to the caller.

The callback already wraps output write failures, but these blocks turn every non-interruption streamErr into REQBODY/RESBODY variables and still return nil. A truncated relay then looks successful to the connector, and the same suppression also hides reader-side I/O errors bubbled up by Process*Records.

Also applies to: 1428-1444

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@internal/corazawaf/transaction.go` around lines 1333 - 1348, The code
currently logs non-interruption streaming errors (streamErr != nil && streamErr
!= errStreamInterrupted), sets REQBODY/RESBODY via tx.generateRequestBodyError
and tx.WAF.Rules.Eval, but then falls through and returns (tx.interruption, nil)
which hides relay I/O failures; change both such blocks (the one around
tx.generateRequestBodyError and the similar block at the later mirror) to return
the streaming error to the caller after handling: i.e., after logging, calling
tx.generateRequestBodyError and possibly tx.WAF.Rules.Eval, return
(tx.interruption, streamErr) instead of allowing the function to return nil
error so reader-side I/O failures are propagated.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

This matrix is already stale relative to CI.

The hardcoded slice omits coraza.rule.rx_prefilter, which is still present in .github/workflows/regression.yml:25-48. If TagsMatrix is meant to be the source of truth for build tags, its output is incomplete today and will keep drifting while the workflow remains hardcoded.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@magefile.go` around lines 279 - 287, The tags slice used to build the tags
matrix (the variable named tags passed into combinations(tags)) is missing
"coraza.rule.rx_prefilter", causing the generated TagsMatrix to be stale; add
"coraza.rule.rx_prefilter" to that slice and then ensure TagsMatrix is the
single source of truth by either deriving the slice from the canonical build tag
list or updating the external workflow to read from the same central constant so
they cannot drift apart.

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify that `json.Marshal` is referenced while `encoding/json` is not imported.
# Expected: one match for `json.Marshal` and no match for `encoding/json`.
rg -n 'json\.Marshal|encoding/json' magefile.go
sed -n '8,20p' magefile.go

Repository: corazawaf/coraza

Length of output: 254

---

Add missing encoding/json import.

json.Marshal is used at line 289 but encoding/json is not imported.

Proposed fix

 import (
+	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
 	"os"
 	"os/exec"
 	"path/filepath"
 	"regexp"
 	"strings"

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	jsonData, err := json.Marshal(combos)
	import (
	"encoding/json"
	"errors"
	"fmt"
	"io"
	"os"
	"os/exec"
	"path/filepath"
	"regexp"
	"strings"

🧰 Tools

🪛 GitHub Actions: Lint (pre-commit)

[error] 289-289: Go lint failed with compile error: undefined: json.

🪛 GitHub Actions: Regression Tests

[error] 289-289: Build failed for step "go run mage.go coverage": /magefile.go:289:19: undefined: json

🪛 GitHub Check: lint

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.no_memoize)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.case_sensitive_args_keys)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.mandatory_rule_id_check,coraza.rule.case_sensitive_args_...

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.mandatory_rule_id_check)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.multiphase_evaluation,coraza.rule.case_sensitive_args_keys)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.multiphase_evaluation,coraza.rule.mandatory_rule_id_chec...

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.multiphase_evaluation,coraza.rule.mandatory_rule_id_check)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.multiphase_evaluation,coraza.rule.no_regex_multiline)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.multiphase_evaluation)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.no_regex_multiline)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, coraza.rule.rx_prefilter)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, no_fs_access,coraza.no_memoize)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest, no_fs_access)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.25.x, ubuntu-latest)

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.26.x, ubuntu-latest, coraza.rule.multiphase_evaluation,coraza.rule.mandatory_rule_id_chec...

[failure] 289-289:
undefined: json

🪛 GitHub Check: test (1.26.x, ubuntu-latest)

[failure] 289-289:
undefined: json

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@magefile.go` at line 289, The code calls json.Marshal (see
json.Marshal(combos)) but the encoding/json package is not imported; add the
import "encoding/json" to the import block (where other stdlib imports live) so
json.Marshal resolves, then run gofmt/goimports to ensure imports are organized;
update any existing import grouping to include encoding/json alongside other
standard library imports.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Handle request-body read errors in the test backend.

On Line 41, the io.ReadAll error is discarded, which can mask server-side failures and produce misleading test outcomes.

🛠️ Proposed fix

 	backend := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		body, _ := io.ReadAll(r.Body)
+		body, err := io.ReadAll(r.Body)
+		if err != nil {
+			http.Error(w, "failed to read request body", http.StatusInternalServerError)
+			return
+		}
 		w.WriteHeader(http.StatusOK)
-		fmt.Fprintf(w, "OK: %s", body)
+		if _, err := fmt.Fprintf(w, "OK: %s", body); err != nil {
+			return
+		}
 	})

As per coding guidelines: Always check and handle errors explicitly; Don't ignore errors.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@testing/e2e/ndjson_e2e_test.go` around lines 41 - 43, The test HTTP handler
currently discards the error from io.ReadAll(r.Body); change the handler to
check the error returned by io.ReadAll, and if non-nil respond with an
appropriate failure status (e.g., w.WriteHeader(http.StatusInternalServerError))
and a descriptive message (and/or t.Fatalf/log) instead of proceeding to write
"OK". Locate the body-reading logic that calls io.ReadAll(r.Body) and update
error handling around that call and the subsequent w.WriteHeader/fmt.Fprintf
usage so test failures surface server-side read errors.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

This assertion is too permissive to catch regressions.

Allowing both 200 and 403 makes the test non-diagnostic. It should assert one expected outcome for application/json under this rule set.

✅ Suggested stricter assertion

-	if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusForbidden {
-		t.Errorf("unexpected status %d for non-NDJSON content type", resp.StatusCode)
-	}
+	if resp.StatusCode != http.StatusOK {
+		t.Errorf("expected 200 for non-NDJSON content type, got %d", resp.StatusCode)
+	}

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusForbidden {
	t.Errorf("unexpected status %d for non-NDJSON content type", resp.StatusCode)
	if resp.StatusCode != http.StatusOK {
	t.Errorf("expected 200 for non-NDJSON content type, got %d", resp.StatusCode)
	}

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@testing/e2e/ndjson_e2e_test.go` around lines 169 - 170, The test currently
allows either http.StatusOK or http.StatusForbidden for non-NDJSON content which
is too permissive; update the assertion in the test (replace the condition that
checks resp.StatusCode against http.StatusOK && http.StatusForbidden) to assert
the single expected status code (e.g., resp.StatusCode == http.StatusOK) and
change the t.Errorf call to report the single expected value (use
resp.StatusCode and http.StatusOK in the message); locate and modify the check
referencing resp.StatusCode, http.StatusOK, http.StatusForbidden and t.Errorf in
the ndjson_e2e_test.go test function.