README.md

Security and Compliance Work Group

Use Cases and Threat Modeling Subgroup

Important

• The Use Cases and Threat Modeling subgroup meets weekly on Tuesdays @12pm US Eastern, 16:00 GMT. Please use the calendar link below and click the "Need an invite?" link:
  • LF AI & Data calendar entry
  • Slack: #security-use-cases-and-threat-modeling
  • Meeting Agendas/Notes: Google Doc

Leadership

Chairs:

• Matt Colman (IBM)
  • mtcolman@uk.ibm.com
  • Slack: @Matt Colman
• Petra Vukmirovoc (Numan)
  • Slack: @petra vukmirovic

Mission

To produce interoperable AI security and compliance standards through its lifecycle-wide use cases that with corresponding controls and risk mitigation measures informed by a library of real world threat models, make them directly consumable by automated security, compliance, and governance workflows.

Goals

1. Identify and publish at least 3 pilot use cases + threat models in OWASP Threat Model Library
2. Align on the use case - threat model (threats, controls, risks) - standard workflow

Guidance

Threat Statements

Threat statements need to be written with the following structure:

"Actor A ('the who') does B ('action') to C ('technical component') which results in doing D ('the reason') to E ('the resource')"

E.g.: "Attacker ('the who') tampers ('malicious action') with REST service parameters ('technical component') to view ('the reason') unauthorised customer data ('the resource')"

Threat Perspective

Threats need to be written with actors included within the diagram as the focal point, for example on a diagram that shows a developer flow, this threat:

External Attacker publishes a malicious version of the official plugin to an unofficial marketplace which results in developers installing it and sending code to an attacker-controlled backend.

should be written as:

Developer uses an unofficial marketplace and unknowingly installs a malicious version of the official plugin into the IDE. This results in code being sent to an external, attacker-controlled backend.

NIST 800-30 Taxonomy Alignment

The AI tool will enrich threat statements with NIST 800-30 rather than reformat them, e.g. we would send the following to the LLM when asking it to align CAPEC to the scenario:

Prompt
Determine which CAPEC align(s) to this threat scenario

Developer uses an unofficial marketplace and unknowingly installs a malicious version of the official plugin into the IDE. This results in code being sent to an external, attacker-controlled backend.

becomes:

Prompt
Determine which CAPEC align(s) to this threat scenario, NIST 800-30 threat taxonomy alignment has been supplied to provide further clarity alongside the threat statement. Ensure to primarily focus on the details in the threat statement.

Developer uses an unofficial marketplace and unknowingly installs a malicious version of the official plugin into the IDE. This results in code being sent to an external, attacker-controlled backend.

NIST 800-30 taxonomy alignment:

• Threat Source: Adversarial - Individual - Insider
• Threat Events: Deliver/insert/install malicious capabilities - Insert tampered critical components into systems / Deliver/insert/install malicious capabilities - Insert targeted malware into organizational systems
• Adverse Impacts: HARM TO ASSETS - Loss of intellectual property / HARM TO ASSETS - Damage to or loss of information assets

Process to follow

Human:

1. Create dataflow diagram
2. Generate threats (supply the guidance above for reference)
   • Ensure threats are written following the structure guidance above (amend any that aren't)
   • Ensure threats are written following the perspective guidance above (amend any that aren't)

AI:

1. Enrich human-generated threats with NIST 800-30 taxonomy (threat source, threat event, and adverse impact)
2. Map enriched statements to CAPEC & CWE
3. Map CAPEC & CWE to Controls (NIST 800-53 and CSA AICM)

Current Work

We have threat modelled the use case "AI Assistant Code Generator" (based on using a code-generating AI assistant as a plugin in IDE).

The DFD lives here on Mural.

The Threat Model lives here on google Docs (please request access if needed)

We are currently creating an automated, AI driven process to utilise AI to:

• Enrich threat scenario text to ensure high quality
• Enrich scenario with NIST 800-30 threat taxonomy
• Map threat scenario to Common Attack Pattern Enumerations and Classifications (CAPEC)
• Map the CAPEC to NIST SP800-53 Security and Privacy Controls for Information Systems and Organizations controls
• Map NIST SP800-53 controls to Cloud Security Alliance AI Controls Matrix (CSA AICM) controls (where the threat is AI specific/focused)

Note: The way we define Threat Intent (as per NIST 800-30) is either: Intent by proxy or Direct intent. So a Vendor can be adversarial even if it is by proxy through an attacker if they are compromised

This gives us the following mapping: