OSCAL 1.2.0 JSON Schema: Missing properties field in object definitions

[brandtkeller]

Describe the bug

The OSCAL 1.2.0 complete schema (oscal_complete_schema.json) contains multiple object definitions that have required fields specified but are missing the corresponding properties field. This appears to be a regression from version 1.1.3, where these same definitions included properties fields.

Who is the bug affecting

For me - it affects my ability to use existing tooling to generate types for programmatic use. I have yet to explore other implications of this schema state.

What is affected by this bug

Metaschema, OSCAL Content

How do we replicate this issue

To find all affected definitions in the schema, you can use this jq query:

jq '[.. | objects | select(type == "object" and has("type") and .type == "object" and has("title") and has("required") and (has("properties") | not)) | .title] | unique' oscal_complete_schema.json

Expected behavior (i.e. solution)

In OSCAL 1.2.0

{
  "additionalProperties": false,
  "description": "Relates the finding to a set of referenced observations that were used to determine the finding.",
  "required": [
    "observation-uuid"
  ],
  "title": "Related Observation",
  "type": "object"
}

Problem: The required field lists "observation-uuid" but there is no properties field defining what observation-uuid is.

In OSCAL 1.1.3 (WORKING)

{
  "additionalProperties": false,
  "description": "Relates the finding to a set of referenced observations that were used to determine the finding.",
  "properties": {
    "observation-uuid": {
      "$ref": "#/definitions/UUIDDatatype",
      "description": "A machine-oriented identifier reference to an observation defined in the list of observations.",
      "title": "Observation Universally Unique Identifier Reference"
    }
  },
  "required": [
    "observation-uuid"
  ],
  "title": "Related Observation",
  "type": "object"
}

I would expect that the properties are present for required or non-required fields.

Other comments

I am far from a schema expert (more so metaschema). Maybe this schema is entirely valid and I have overlooked some aspect of generating types from the schema.

Revisions

No response

[iMichaela]

@brandtkeller - Is this a problem only with the complete schema or with the individual schemas too? Such analysis would determine if the problem is with the tooling (which uses new functions) or the models because some common file had an update which caused the error. No metaschema definition files were modified from v1.1.3 to v1.2.0., except for an old PR from FedRAMP that should not have caused it.
Further investigations are necessary.

[iMichaela]

@brandtkeller - All schemas are valid, and so are the metaschema definition files. The resulting schemas are validated by the pipeline and not released unless they are valid. But if props existed before, they should still be there since nobody removed them.

[iMichaela]

The difference between 1.1.3 (left) and 1.2.0 (right) appears only in the assessment results JSON schemas. The XSD schemas show no difference and the assessment metaschema definition files have not been changed between these versions. It is a JSON specific schema generation issue.

[iMichaela]

Initial review shows that the related-observation assembly is locally defined two times in the oscal_assessemnt-cmmon_metaschema.xml. This is NOT a new definition. It has been defined in this way for almost 3 years, but the new toolchain is less tolerant and most likely causes the error.

line: 832

  <define-assembly name="related-observation" max-occurs="unbounded">
    <formal-name>Related Observation</formal-name>
    <description>Relates the finding to a set of referenced observations that were used to determine the finding.</description>
    <group-as name="related-observations" in-json="ARRAY"/>
    <define-flag name="observation-uuid" as-type="uuid" required="yes">
      <formal-name>Observation Universally Unique Identifier Reference</formal-name>
      <!-- Identifier Reference -->
      <description>A <a href="https://pages.nist.gov/OSCAL/concepts/identifier-use/#machine-oriented">machine-oriented</a> identifier reference to an observation defined in the list of observations.</description>
    </define-flag>
  </define-assembly>

line: 1299

   <define-assembly name="related-observation" max-occurs="unbounded">
      <formal-name>Related Observation</formal-name>
      <description>Relates the finding to a set of referenced observations that were used to determine the finding.</description>
      <group-as name="related-observations" in-json="ARRAY"/>
      <define-flag name="observation-uuid" as-type="uuid" required="yes">
         <formal-name>Observation Universally Unique Identifier Reference</formal-name>
         <!-- Identifier Reference -->
         <description>A <a href="https://pages.nist.gov/OSCAL/concepts/identifier-use/#machine-oriented">machine-oriented</a> identifier reference to an observation defined in the list of observations.</description>
      </define-flag>
    </define-assembly>

In oscal_poam_metaschema.xml:

line: 150

            <define-assembly name="related-observation" max-occurs="unbounded">
                <formal-name>Related Observation</formal-name>
                <description>Relates the poam-item to a set of referenced observations that were used to determine the finding.</description>
                <group-as name="related-observations" in-json="ARRAY"/>
                <define-flag name="observation-uuid" as-type="uuid" required="yes">
                    <formal-name>Observation Universally Unique Identifier Reference</formal-name>
                    <!-- Identifier Reference -->
                    <description>A <a href="https://pages.nist.gov/OSCAL/concepts/identifier-use/#machine-oriented">machine-oriented</a> identifier reference to an observation defined in the list of observations.</description>
                </define-flag>
            </define-assembly>

INITIAL CONCLUTION:
The bug does not manifest in the POAM schema where the same assembly is defined once locally while oscal_assessment-common_metaschema.xml is also imported. Eliminating the duplications is recommended. Testing is necessary.

[iMichaela]

Fixed by PR #2180