-
Notifications
You must be signed in to change notification settings - Fork 1.9k
feat(parsers): add PICUS Breach and Attack Simulation CSV parser #14984
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
skywalke34
wants to merge
6
commits into
DefectDojo:dev
Choose a base branch
from
skywalke34:picus-parser
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
c0a14fa
feat(parser): add PICUS Breach and Attack Simulation parser
skywalke34 6e35e28
test(parser): add PICUS parser fixtures and unit tests
skywalke34 52edddb
feat(parser): register PICUS deduplication config
skywalke34 f92f866
docs(parser): add PICUS parser documentation
skywalke34 11d5bef
feat(parser): enrich PICUS mitigation with control posture and triage…
skywalke34 c8593b1
docs(parser): document PICUS enriched mitigation field
skywalke34 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| --- | ||
| title: "PICUS Scan" | ||
| toc_hide: true | ||
| --- | ||
|
|
||
| The [Picus Security](https://www.picussecurity.com/) parser for DefectDojo supports imports from CSV format. Picus is a Breach and Attack Simulation (BAS) platform that runs simulated attacks against an environment and reports whether existing security controls prevented, logged, and alerted on each simulated action. This document details how Picus result CSV exports are mapped into DefectDojo Findings, which fields are parsed, and the BAS-specific transformation notes. | ||
|
|
||
| ## Supported File Types | ||
|
|
||
| The Picus parser accepts CSV file format. Picus exports a separate CSV per attack vector (for example Email, Endpoint, Network, and Web), but all share an identical column schema, so the same parser handles every export. | ||
|
|
||
| To import Picus results into DefectDojo: | ||
|
|
||
| 1. Log into your Picus console | ||
| 2. Run or open the simulation whose results you want to import | ||
| 3. Export the results as CSV (one file per attack vector) | ||
| 4. Save each file with a `.csv` extension | ||
| 5. Upload each CSV to DefectDojo using the "PICUS Scan" scan type | ||
|
|
||
| DefectDojo imports a single scan file at a time and does not unpack archives. If the export is delivered as a `.rar` or `.zip`, extract it first and import each CSV individually. Each file becomes its own import, which keeps the attack vectors (Email, Endpoint, Network, Web) grouped separately. | ||
|
|
||
| ## Default Deduplication Hashcode Fields | ||
|
|
||
| Picus findings deduplicate using the hashcode algorithm on a single stable [hashcode field](https://docs.defectdojo.com/en/working_with_findings/finding_deduplication/about_deduplication/), the tool-native action identifier: | ||
|
|
||
| - vuln_id_from_tool (populated from the Picus `actionId`) | ||
|
|
||
| ### Sample Scan Data | ||
|
|
||
| Sample Picus scans can be found in the [sample scan data folder](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/picus). | ||
|
|
||
| ## Link To Tool | ||
|
|
||
| - [Picus Security](https://www.picussecurity.com/) | ||
| - [Picus Documentation](https://docs.picussecurity.com/) | ||
|
|
||
| ## CSV Format | ||
|
|
||
| ### Total Fields in CSV | ||
|
|
||
| - Total data fields: 63 | ||
| - Total data fields parsed into dedicated Finding fields or the structured description: 20 | ||
| - Remaining fields (detection-integration, protocol, hash, and tab-link columns) are not currently mapped | ||
|
|
||
| ### CSV Format Field Mapping Details | ||
|
|
||
| <details> | ||
| <summary>Click to expand Field Mapping Table</summary> | ||
|
|
||
| | Source Field | DefectDojo Field | Notes | | ||
| | ---------------------- | ------------------------- | -------------------------------------------------------------------------------------- | | ||
| | threatName + actionName | title | Combined as "threatName - actionName"; truncated to 500 characters with "..." if longer | | ||
| | threatSeverity | severity | Mapped to DefectDojo severity levels; defaults to Info if unrecognized | | ||
| | threatPreventionResult | active | "Not Blocked" sets active=True (control gap); any other value sets active=False | | ||
| | threatPreventionResult | mitigation | Drives a recommendation describing whether controls blocked the simulated attack | | ||
| | actionId | vuln_id_from_tool | Native Picus action identifier; drives hashcode deduplication across re-imports | | ||
| | cve | unsaved_vulnerability_ids | Comma-separated CVEs split into a list; omitted when empty | | ||
| | cwe | cwe | Parsed to integer when the field contains digits; omitted otherwise | | ||
| | actionMitreTactic | unsaved_tags | Added as a tag when present | | ||
| | actionMitreTechnique | unsaved_tags | Added as a tag when present | | ||
| | actionMitreSubtechnique | unsaved_tags | Added as a tag when present | | ||
| | attackCategory | unsaved_tags | Added as a tag when present | | ||
| | affectedProducts | component_name | The affected product reported by the simulation | | ||
| | threatName | description | Included in the structured description table | | ||
| | actionName | description | Included in the structured description table | | ||
| | actionDescription | description | Included in the structured description table | | ||
| | attackModules | description | Included in the structured description table | | ||
| | threatDetectionLogResult | description | Included in the structured description table | | ||
| | threatDetectionAlertResult | description | Included in the structured description table | | ||
| | affectedOs | description | Included in the structured description table | | ||
| | affectedPlatforms | description | Included in the structured description table | | ||
| | actionPayload | description | Included in the structured description table | | ||
|
|
||
| </details> | ||
|
|
||
| ### Additional Finding Field Settings (CSV Format) | ||
|
|
||
| <details> | ||
| <summary>Click to expand Additional Settings Table</summary> | ||
|
|
||
| | Finding Field | Default Value | Notes | | ||
| | --------------- | ----------------------------- | --------------------------------------------------------------------------- | | ||
| | static_finding | False | Picus results reflect runtime simulation behavior, not static analysis | | ||
| | dynamic_finding | True | Picus results reflect runtime simulation behavior | | ||
| | active | True when "Not Blocked" | A simulated attack that was not blocked is an open control gap | | ||
|
|
||
| </details> | ||
|
|
||
| ## Special Processing Notes | ||
|
|
||
| ### Breach and Attack Simulation Semantics | ||
|
|
||
| Picus is a BAS tool, so each CSV row is a simulated attack *action* rather than a discovered vulnerability. The value to DefectDojo is whether a security control stopped the simulated attack. The parser imports every action as a Finding and uses `threatPreventionResult` to decide the finding's active state: | ||
|
|
||
| - `Not Blocked` → active Finding (the control failed to stop the attack — an open gap) | ||
| - `Blocked` (or any other value) → inactive Finding (the control mitigated the attack) | ||
|
|
||
| This preserves the full simulation history while surfacing unmitigated gaps as the actionable findings. | ||
|
|
||
| ### Severity Mapping | ||
|
|
||
| Severity is taken from the `threatSeverity` column (the inherent risk of the threat scenario), not the per-action `severity` column: | ||
|
|
||
| - `Critical` → Critical | ||
| - `High` → High | ||
| - `Medium` → Medium | ||
| - `Low` → Low | ||
| - `Info` / `Informational` → Info | ||
|
|
||
| Any unrecognized value defaults to Info. | ||
|
|
||
| ### Title Format | ||
|
|
||
| Finding titles combine the threat and action names as "threatName - actionName". When only a threat name is present, it is used alone. Titles longer than 500 characters are truncated to 497 characters with a "..." suffix. | ||
|
|
||
| ### Description Construction | ||
|
|
||
| The parser builds a markdown table containing the threat, action, attack category, MITRE references, detection/prevention results, affected asset details, payload, and the Picus simulation/action identifiers. Empty source fields are omitted, and pipe characters in values are escaped so the table renders correctly. | ||
|
|
||
| ### Deduplication | ||
|
|
||
| Deduplication uses the hashcode algorithm keyed solely on `vuln_id_from_tool`, which the parser populates from the native Picus `actionId`. The `actionId` is stable across simulation runs (the same attack action keeps the same identifier), while `simulationRunId` changes on every run. Keying on `actionId` alone — and deliberately excluding `simulationRunId` — means that when a later run's CSV is re-imported into the same test, each action matches its prior finding. This lets DefectDojo update the status of existing findings (for example, closing an action that was previously "Not Blocked" once a control begins blocking it) rather than creating duplicates. Picus simulations span an asset class such as Network or Email, so deduplication is expected to operate within a single asset/engagement rather than across asset types. | ||
|
|
||
| ### Unmapped Fields | ||
|
|
||
| Detection-integration, protocol-level prevention/detection, file-hash, signature, and tab-link columns are retained in the source CSV but are not currently mapped to Finding fields. The most operationally relevant columns are surfaced either as dedicated Finding fields or within the structured description table. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| import csv | ||
| import io | ||
|
|
||
| from dojo.models import Finding | ||
|
|
||
| SEVERITY_MAPPING = { | ||
| "Critical": "Critical", | ||
| "High": "High", | ||
| "Medium": "Medium", | ||
| "Low": "Low", | ||
| "Info": "Info", | ||
| "Informational": "Info", | ||
| } | ||
|
|
||
|
|
||
| class PicusParser: | ||
|
|
||
| """Parser for Picus Breach and Attack Simulation (BAS) CSV result exports.""" | ||
|
|
||
| def get_scan_types(self): | ||
| return ["PICUS Scan"] | ||
|
|
||
| def get_label_for_scan_types(self, scan_type): | ||
| return scan_type | ||
|
|
||
| def get_description_for_scan_types(self, scan_type): | ||
| return "Import Picus Breach and Attack Simulation results (CSV)." | ||
|
|
||
| def get_findings(self, filename, test): | ||
| content = filename.read() | ||
| if isinstance(content, bytes): | ||
| content = content.decode("utf-8-sig") | ||
| reader = csv.DictReader(io.StringIO(content), delimiter=",", quotechar='"') | ||
|
|
||
| return [self._build_finding(row, test) for row in reader] | ||
|
|
||
| def _build_finding(self, row, test): | ||
| def get(key): | ||
| return (row.get(key) or "").strip() | ||
|
|
||
| threat_name = get("threatName") | ||
| action_name = get("actionName") | ||
| prevention = get("threatPreventionResult") | ||
|
|
||
| title = f"{threat_name} - {action_name}" if action_name else threat_name | ||
| if len(title) > 500: | ||
| title = title[:497] + "..." | ||
|
|
||
| severity = SEVERITY_MAPPING.get(get("threatSeverity"), "Info") | ||
|
|
||
| description = self._build_description(get) | ||
| mitigation = self._build_mitigation(get) | ||
|
|
||
| finding = Finding( | ||
| test=test, | ||
| title=title, | ||
| severity=severity, | ||
| description=description, | ||
| mitigation=mitigation, | ||
| component_name=get("affectedProducts") or None, | ||
| # In BAS, a finding is active when the attack was NOT blocked. | ||
| active=prevention == "Not Blocked", | ||
| static_finding=False, | ||
| dynamic_finding=True, | ||
| ) | ||
|
|
||
| # actionId is Picus's native, run-stable action identifier; it drives | ||
| # hashcode deduplication so the same action matches across re-imports. | ||
| action_id = get("actionId") | ||
| if action_id: | ||
| finding.vuln_id_from_tool = action_id | ||
|
|
||
| cwe = get("cwe") | ||
| if cwe.isdigit(): | ||
| finding.cwe = int(cwe) | ||
|
|
||
| cves = [c.strip() for c in get("cve").split(",") if c.strip()] | ||
| if cves: | ||
| finding.unsaved_vulnerability_ids = cves | ||
|
|
||
| tags = self._build_tags(get) | ||
| if tags: | ||
| finding.unsaved_tags = tags | ||
|
|
||
| return finding | ||
|
|
||
| def _build_tags(self, get): | ||
| tags = [] | ||
| for key in ("actionMitreTactic", "actionMitreTechnique", "actionMitreSubtechnique", "attackCategory"): | ||
| value = get(key) | ||
| if value and value not in tags: | ||
| tags.append(value) | ||
| return tags | ||
|
|
||
| def _build_mitigation(self, get): | ||
| prevention = get("threatPreventionResult") | ||
| if prevention == "Not Blocked": | ||
| return ( | ||
| "The simulated attack was NOT blocked by existing preventive controls. " | ||
| "Review and tune the relevant security controls to block this technique." | ||
| ) | ||
| if prevention == "Blocked": | ||
| return "The simulated attack was blocked by existing preventive controls." | ||
| return "" | ||
|
|
||
| def _build_description(self, get): | ||
| fields = [ | ||
| ("Threat", "threatName"), | ||
| ("Action", "actionName"), | ||
| ("Action Description", "actionDescription"), | ||
| ("Attack Category", "attackCategory"), | ||
| ("Attack Modules", "attackModules"), | ||
| ("Threat Severity", "threatSeverity"), | ||
| ("Prevention Result", "threatPreventionResult"), | ||
| ("Detection Log Result", "threatDetectionLogResult"), | ||
| ("Detection Alert Result", "threatDetectionAlertResult"), | ||
| ("MITRE Tactic", "actionMitreTactic"), | ||
| ("MITRE Technique", "actionMitreTechnique"), | ||
| ("MITRE Sub-technique", "actionMitreSubtechnique"), | ||
| ("Affected OS", "affectedOs"), | ||
| ("Affected Platforms", "affectedPlatforms"), | ||
| ("Affected Products", "affectedProducts"), | ||
| ("Action Payload", "actionPayload"), | ||
| ("Simulation Run Id", "simulationRunId"), | ||
| ("Action Id", "actionId"), | ||
| ] | ||
| lines = ["| Field | Value |", "| --- | --- |"] | ||
| for label, key in fields: | ||
| value = get(key) | ||
| if value: | ||
| value = value.replace("|", "\\|") | ||
| lines.append(f"| {label} | {value} |") | ||
| return "\n".join(lines) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| simulationRunId,threatId,threatName,attackModules,threatSeverity,threatReleaseDateEpoch,threatReleaseDateDateTimeUTC,threatPreventionResult,threatDetectionLogResult,threatDetectionAlertResult,actionId,actionName,actionDescription,affectedOs,affectedPlatforms,affectedProducts,isPrivileged,actionMitreTactic,actionMitreTechnique,actionMitreSubtechnique,nistCapabilities,actionPayload,actionPayloadOutputTabLink,actionPreventionResult,actionPreventionSources,actionFileName,actionSha1,actionSha256,actionMd5,cve,cwe,attackStartedTimeUTC,attackEndedTimeUTC,attackLoggedTimeUTC,attackAlertedTimeUTC,actionProtocol,actionProtocolPreventionResult,actionProtocolPreventionSources,actionProtocolDetectionLogResult,actionProtocolDetectionAlertResult,actionDetectionIntegration,actionDetectionLogResult,actionDetectionAlertResult,attackCategory,actionLogsTabLink,genericMitigationsTabLink,vendorName,platform,productVersion,signatureId,signatureName,severity,signatureVersion,detectionContentTabLink,attackerUser,tenantId,subscription,location,rewindStatus,output,executionSteps,rewindSteps,manualRewindSteps | ||
| 2001,1,Fileless Malware via PowerShell,Endpoint,Critical,,02/10/2023,Not Blocked,Not Logged,Not Alerted,1001,PowerShell Download Cradle Execution,Executes an in-memory PowerShell download cradle to retrieve a payload without touching disk,Windows,Platform X,Product Y,No,TA0002,T1059,T1059.001,,,,,,,,,,CVE-2021-44228,78,02/10/2023,02/10/2023,,,,,,,,,,,Malicious Code,https://sample[.]com,https://sample[.]com,Vendor X,Platform X,v1,10001001,SIG-T1059,High,v1,,Anonymous1,341,,,,,,, | ||
| 2001,1,Credential Dumping,Endpoint,High,,02/10/2023,Blocked,Logged,Alerted,1002,LSASS Memory Dump via comsvcs.dll,Dumps LSASS process memory to extract credentials,Windows,Platform X,Product Y,No,TA0006,T1003,T1003.001,,,,,,,,,,,200,02/10/2023,02/10/2023,,,,,,,,,,,Data Theft,https://sample[.]com,https://sample[.]com,Vendor X,Platform X,v1,10001002,SIG-T1003,High,v1,,Anonymous1,341,,,,,,, | ||
| 2001,1,Lateral Movement Attempt,Endpoint,Medium,,02/10/2023,Not Blocked,Logged,Not Alerted,1003,SMB Share Enumeration,Enumerates accessible SMB shares across the subnet,Windows,Platform X,Product Y,No,TA0008,T1021,T1021.002,,,,,,,,,,,,02/10/2023,02/10/2023,,,,,,,,,,,Reconnaissance,https://sample[.]com,https://sample[.]com,Vendor X,Platform X,v1,10001003,SIG-T1021,Medium,v1,,Anonymous1,341,,,,,,, | ||
| 2001,1,Benign Beacon Test,Endpoint,Low,,02/10/2023,Blocked,Logged,Alerted,1004,Low Risk Connectivity Check,Performs a low-risk outbound connectivity check,Windows,Platform X,Product Y,No,TA0011,T1071,,,,,,,,,,,,,02/10/2023,02/10/2023,,,,,,,,,,,Network Attack,https://sample[.]com,https://sample[.]com,Vendor X,Platform X,v1,10001004,SIG-T1071,Low,v1,,Anonymous1,341,,,,,,, | ||
| 2001,1,XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,Endpoint,Critical,,02/10/2023,Not Blocked,Not Logged,Not Alerted,1005,YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY,Threat and action names are intentionally long to validate title truncation behavior,Windows,Platform X,Product Y,No,TA0040,T1486,,,,,,,,,,,,,02/10/2023,02/10/2023,,,,,,,,,,,Malicious Code,https://sample[.]com,https://sample[.]com,Vendor X,Platform X,v1,10001005,SIG-T1486,Critical,v1,,Anonymous1,341,,,,,,, | ||
| 2001,1,Phishing Payload Delivery,Endpoint,High,,02/10/2023,Not Blocked,Not Logged,Alerted,1006,Macro-Enabled Document Drop,Delivers a macro-enabled document that downloads a second-stage payload,Windows,Platform X,Product Y,No,TA0001,T1566,T1566.001,,,,,,,,,,CVE-2017-11882,119,02/10/2023,02/10/2023,,,,,,,,,,,Social Engineering,https://sample[.]com,https://sample[.]com,Vendor X,Platform X,v1,10001006,SIG-T1566,Medium,v1,,Anonymous1,341,,,,,,, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| simulationRunId,threatId,threatName,attackModules,threatSeverity,threatReleaseDateEpoch,threatReleaseDateDateTimeUTC,threatPreventionResult,threatDetectionLogResult,threatDetectionAlertResult,actionId,actionName,actionDescription,affectedOs,affectedPlatforms,affectedProducts,isPrivileged,actionMitreTactic,actionMitreTechnique,actionMitreSubtechnique,nistCapabilities,actionPayload,actionPayloadOutputTabLink,actionPreventionResult,actionPreventionSources,actionFileName,actionSha1,actionSha256,actionMd5,cve,cwe,attackStartedTimeUTC,attackEndedTimeUTC,attackLoggedTimeUTC,attackAlertedTimeUTC,actionProtocol,actionProtocolPreventionResult,actionProtocolPreventionSources,actionProtocolDetectionLogResult,actionProtocolDetectionAlertResult,actionDetectionIntegration,actionDetectionLogResult,actionDetectionAlertResult,attackCategory,actionLogsTabLink,genericMitigationsTabLink,vendorName,platform,productVersion,signatureId,signatureName,severity,signatureVersion,detectionContentTabLink,attackerUser,tenantId,subscription,location,rewindStatus,output,executionSteps,rewindSteps,manualRewindSteps |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be
""or maybeNone?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. I've reworked
_build_mitigationso it no longer falls through to a barereturn ""(just pushed in 11d5bef). It now aggregates the prevent → log → alert control posture plus any available Picus mitigation/triage references (mitigation guidance, detection content, payload output, action logs, detection signature), emitting only the fields that are present. In the edge case where none of those exist it now returnsNonerather than"", so the field stays unset instead of being persisted as an empty string.