feat: add ARGS_RAW, ARGS_GET_RAW, ARGS_POST_RAW, and ARGS_NAMES_RAW collections

[fzipi]

what

Add raw (non-URL-decoded) argument collections that preserve wire-format values before any percent-decoding. This enables rules to safely apply t:urlDecodeUni without double-decoding, detect double URL encoding, and behave consistently across integrations.

New collections

Collection	Counterpart	Semantics
ARGS_GET_RAW	ARGS_GET	Query string values, not URL-decoded
ARGS_POST_RAW	ARGS_POST	Form body values, not URL-decoded
ARGS_RAW	ARGS	Union of ARGS_GET_RAW and ARGS_POST_RAW
ARGS_NAMES_RAW	ARGS_NAMES	Argument names (GET + POST), not URL-decoded
ARGS_GET_NAMES_RAW	ARGS_GET_NAMES	GET argument names, not URL-decoded
ARGS_POST_NAMES_RAW	ARGS_POST_NAMES	POST argument names, not URL-decoded

Implementation details

• Added ParseQueryBoth() to internal/url that returns both decoded and raw maps in a single pass (no double parsing)
• Fast-path in queryUnescape: skips allocation when input has no % or + (common case)
• Raw collections respect the same case-sensitivity build tag as cooked collections
• All body processors (urlencoded, multipart, JSON) populate ARGS_POST_RAW
• Purely additive — no existing behavior changes

refs

• Closes Add ARGS_RAW, ARGS_GET_RAW, ARGS_POST_RAW, and ARGS_NAMES_RAW Collections #1491
• Coordinated with ModSecurity#3501

[Copilot]

Pull request overview

This PR adds four new raw (non-URL-decoded) argument collections to Coraza: ARGS_RAW, ARGS_GET_RAW, ARGS_POST_RAW, and ARGS_NAMES_RAW. These collections preserve wire-format values before any percent-decoding, addressing a long-standing issue where applying transformations like t:urlDecodeUni to automatically-decoded variables results in double decoding. This creates both false positives and detection gaps depending on how rules are written. The new raw collections enable rule authors to safely apply URL decoding transformations exactly once, with consistent semantics across different Coraza integrations.

Changes:

• Added four new variable types (ArgsRaw, ArgsGetRaw, ArgsPostRaw, ArgsNamesRaw) with complete registration in the variables system
• Implemented ParseQueryRaw function that preserves percent-encoding and plus signs in query string parsing
• Integrated raw collections into transaction processing for GET parameters and all body processors (urlencoded, multipart, JSON)
• Added comprehensive integration tests covering URL-encoded values, double encoding, encoded names, and plus sign preservation

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
types/variables/variables.go	Exports new raw collection constants to public API
internal/variables/variables.go	Defines new raw variable types with documentation
internal/variables/variablesmap.gen.go	Adds name mapping and selection support for raw variables
internal/url/url.go	Implements ParseQueryRaw function that skips URL decoding
internal/url/url_test.go	Tests for raw query parsing covering edge cases
internal/corazawaf/transaction.go	Integrates raw collections into transaction state and GET argument extraction
internal/bodyprocessors/urlencoded.go	Populates raw collections for URL-encoded POST bodies
internal/bodyprocessors/multipart.go	Handles multipart bodies (where raw == cooked)
internal/bodyprocessors/json.go	Handles JSON bodies (where raw == cooked)
experimental/plugins/plugintypes/transaction.go	Adds raw collection methods to plugin interface
internal/operators/validate_schema_test.go	Updates mock transaction with raw collection stubs
testing/engine/args_raw.go	Comprehensive integration tests for all raw collections

[codecov]

Codecov Report

❌ Patch coverage is 84.48276% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.32%. Comparing base (a3d3d6c) to head (d404181).

Files with missing lines	Patch %	Lines
internal/corazawaf/transaction.go	83.60%	10 Missing ⚠️
internal/variables/variablesmap.gen.go	66.66%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1493      +/-   ##
==========================================
- Coverage   86.37%   86.32%   -0.06%     
==========================================
  Files         176      176              
  Lines        8810     8924     +114     
==========================================
+ Hits         7610     7704      +94     
- Misses        949      968      +19     
- Partials      251      252       +1     

Flag	Coverage Δ
coraza.no_memoize	86.40% <84.48%> (-0.06%)	⬇️
coraza.rule.case_sensitive_args_keys	86.27% <82.75%> (-0.08%)	⬇️
coraza.rule.mandatory_rule_id_check	86.31% <84.48%> (-0.05%)	⬇️
coraza.rule.multiphase_evaluation	86.06% <84.48%> (-0.05%)	⬇️
coraza.rule.no_regex_multiline	86.31% <84.48%> (-0.05%)	⬇️
default	86.32% <84.48%> (-0.06%)	⬇️
examples+	16.60% <12.06%> (-0.07%)	⬇️
examples+coraza.no_memoize	84.40% <82.75%> (-0.03%)	⬇️
examples+coraza.rule.case_sensitive_args_keys	84.40% <82.75%> (-0.03%)	⬇️
examples+coraza.rule.mandatory_rule_id_check	84.53% <84.48%> (-0.01%)	⬇️
examples+coraza.rule.multiphase_evaluation	86.06% <84.48%> (-0.05%)	⬇️
examples+coraza.rule.no_regex_multiline	84.36% <82.75%> (-0.03%)	⬇️
examples+no_fs_access	83.88% <82.75%> (-0.02%)	⬇️
ftw	86.32% <84.48%> (-0.06%)	⬇️
no_fs_access	85.68% <84.48%> (-0.05%)	⬇️
tinygo	86.32% <84.48%> (-0.06%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fzipi]

@copilot add more coverage to tests

[Copilot]

@fzipi I've opened a new pull request, #1545, to work on those changes. Once the pull request is ready, I'll request review from you.

[jptosso]

Let's replace this with lazy loading, the variable is stored by default as RAW and we lazy load it in standard ARGS variables. Otherwise there is a major performance regression

[Copilot]

Pull request overview

Copilot reviewed 16 out of 18 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

internal/corazawaf/transaction.go:1889

• Under the coraza.rule.case_sensitive_args_keys build tag, argsGetRaw/argsPostRaw are created as case-sensitive collections, but rule variable key normalization still treats ARGS_*_RAW variables as case-insensitive (see caseSensitiveVariable in internal/corazawaf/rule.go). This will cause keyed selections like ARGS_GET_RAW:Foo to be lowercased and fail to match when raw keys preserve case. Add the new RAW variables to the same case-sensitivity handling as their cooked counterparts (including regex/key normalization paths).

	v.serverPort = collections.NewSingle(variables.ServerPort)
	v.highestSeverity = collections.NewSingle(variables.HighestSeverity)
	v.statusLine = collections.NewSingle(variables.StatusLine)
	v.duration = collections.NewSingle(variables.Duration)
	v.resBodyError = collections.NewSingle(variables.ResBodyError)
	v.resBodyErrorMsg = collections.NewSingle(variables.ResBodyErrorMsg)
	v.resBodyProcessorError = collections.NewSingle(variables.ResBodyProcessorError)
	v.resBodyProcessorErrorMsg = collections.NewSingle(variables.ResBodyProcessorErrorMsg)

	v.filesSizes = collections.NewMap(variables.FilesSizes)
	v.filesTmpContent = collections.NewMap(variables.FilesTmpContent)
	v.multipartFilename = collections.NewMap(variables.MultipartFilename)

internal/corazawaf/transaction.go:728

• Comment grammar: "an url encoded" should be "a URL-encoded" (and consider capitalizing URL) to match the terminology used elsewhere in the codebase and docs.

	tx.ruleRemoveByID[id] = struct{}{}
}

---

You can also share your feedback on Copilot code review. Take the survey.

[fzipi]

Let's replace this with lazy loading, the variable is stored by default as RAW and we lazy load it in standard ARGS variables. Otherwise there is a major performance regression

I like this. Honestly, we can even just store everything as RAW, and then if you ask for ARGS it is just applying the urlDecode transformation once.