feat: structural validator expansion — exports, delay-load, VS_VERSIONINFO, resource hierarchy (v0.7.5)

Substantial expansion of IOCX's structural validator suite. Four new
parser/validator pairs (exports, delay-load imports, VS_VERSIONINFO,
resource hierarchy), 24 new reason codes, security-relevant metadata
enrichment on Optional Header and per-resource fields. 1370 tests at
100% coverage.

## Structural validators

* pe_exports / exports: 40-byte IMAGE_EXPORT_DIRECTORY, EAT, ENPT, EOT
  decoded from bytes via struct.unpack_from. Forwarder detection follows
  the PE-spec rule (address RVA within export directory range).
* parser_delay_imports / validator_delay_imports: 32-byte
  IMAGE_DELAY_IMPORT_DESCRIPTOR, INT, IAT decoded from bytes. PE32+/PE32
  thunk sizing determined from OPTIONAL_HEADER.Magic; v1/v0 attribute
  mode captured explicitly; bound state detected by bound_iat_rva != 0.
* pe_version_info / validator_version_info: envelope, VS_FIXEDFILEINFO,
  StringFileInfo, VarFileInfo decoded from bytes. Deterministic leaf
  selection by (name_id, language_id) across multiple RT_VERSION entries.
* Resource hierarchy enforcement: resource validator now tracks tree
  depth and enforces Type → Name → Language layering.

All new parsers never raise; sub-structure failures emit tombstone tags
in truncations[] and per-entry errors[]. Validators use priority-resolved
sub-reasons via details["reason"], scoped by details["table"] where
applicable. Absence of any of these directories is not treated as a
structural defect.

## Reason codes (24 new)

Resource hierarchy: RESOURCE_DIRECTORY_LANGUAGE_NOT_ID,
RESOURCE_DATA_AT_INVALID_DEPTH.

VS_VERSIONINFO: RESOURCE_VERSIONINFO_INVALID_{HEADER, FIXEDINFO,
STRINGFILEINFO, VARFILEINFO}.

Exports: EXPORT_DIRECTORY_INVALID_HEADER, EXPORT_DIRECTORY_OUT_OF_BOUNDS,
EXPORT_TABLE_TRUNCATED, EXPORT_NAME_RVA_INVALID, EXPORT_NAME_NOT_ASCII,
EXPORT_NAME_POINTER_TABLE_UNSORTED, EXPORT_NAME_ORDINAL_INDEX_INVALID,
EXPORT_ORDINAL_OUT_OF_RANGE, EXPORT_FUNCTION_RVA_INVALID,
EXPORT_FORWARDER_MALFORMED.

Delay-load: DELAY_IMPORT_DIRECTORY_INVALID_HEADER,
DELAY_IMPORT_DIRECTORY_OUT_OF_BOUNDS, DELAY_IMPORT_TABLE_TRUNCATED,
DELAY_IMPORT_DESCRIPTOR_INVALID, DELAY_IMPORT_DLL_NAME_INVALID,
DELAY_IMPORT_INT_IAT_MISMATCH, DELAY_IMPORT_ATTRIBUTES_LEGACY_VA_MODE,
DELAY_IMPORT_ENTRY_INVALID.

## Public metadata enrichment

Optional Header gains dll_characteristics (raw + decoded flags + unknown
bits), win32_version_value, loader_flags, stack_reserve_size,
stack_commit_size, heap_reserve_size, heap_commit_size.

Header block gains subsystem_name (from IMAGE_SUBSYSTEM_*) and
machine_name (from IMAGE_FILE_MACHINE_*; table covers all 29 documented
machine types).

Resources now expose a structured ResourceEntry per resource with type,
name, language, language_name, codepage, size, entropy, rva, raw_offset,
and per-entry errors. Entropy rounded to 4 dp for snapshot stability;
output sorted by (type, language, rva). Resources with unreadable data
now emitted with error tombstones (size_invalid, rva_invalid,
data_out_of_bounds, raw_offset_invalid) rather than silently dropped.

## Schema typing

New TypedDicts: ExportStruct, DelayImportStruct, VersionInfoStruct with
sub-types. InternalMetadata gains export_struct, delay_import_struct,
version_info_struct as Optional[...]. New constants module
iocx.parsers.pe_constants with SUBSYSTEM_NAMES, MACHINE_NAMES,
DLL_CHARACTERISTICS_FLAGS, DLL_CHARACTERISTICS_KNOWN_MASK.

## Dispatcher order

validate_resources → validate_version_info → validate_exports →
validate_delay_imports → validate_entropy.

## Fixes and refactors

* Resource parser now guards pe.get_offset_from_rva against
  pefile.PEFormatError and AttributeError; a corrupt RVA produces a -1
  sentinel that flows through the validator's existing data_raw < 0
  arm. No new reason code.
* Resource entropy now sliced from get_memory_mapped_image() using RVA
  (correct) rather than raw file offset. Caught before snapshot
  stamping.
* Resource validator no longer silently returns when a directory's own
  RVA falls outside .rsrc.
* _decode_langid returns None for undecodable LANGIDs (previously the
  magic string "unknown"). Removed the < 0x0400 early-return guard that
  was rejecting valid neutral-sublang LANGIDs.
* analyse_extended refactored: removed duplicated _SUBSYSTEM_MAP and
  _MACHINE_MAP tables (parser layer is now single source of truth),
  removed legacy subsystem_human and machine_human fields, and made
  resource entropy aggregates tolerate entries with entropy: None.

## Breaking changes

* Consumers checking language_name == "unknown" must update to
  language_name is None.
* subsystem_human and machine_human removed from extended metadata;
  use subsystem_name and machine_name (parser layer).
* Optional Header default convention: new fields use None for missing
  (distinct from 0). Existing Optional Header fields retain default-to-
  zero behaviour for backward compatibility.
* Snapshot refresh required for fixtures with resources, optional
  header, or extended metadata. Mechanical via existing regeneration
  tooling.

## Coverage and verification

* 100% line and branch coverage on all new modules.
* Defensive-path coverage via monkeypatched struct.error injection.
  One # pragma: no cover applied to a defensive return in the exports
  validator's _first_unsorted_index helper (documented unreachable).
* Delay-load parser cross-checked against dumpbin /imports on
  mspaint.exe: 107 imports from gdiplus.dll with byte-exact agreement
  on DLL name, hint values, IAT addresses, ordering, and bound state.
* ~650 new tests bringing suite total to 1370 tests.

## Architectural notes

Producer-facing structural metadata (export_struct, delay_import_struct,
Load Config) is internal by design, feeding validators and heuristics
rather than the public IOC schema. Consumers needing structural
information rely on validator reason codes and the existing consumer-
facing metadata fields (exports, delayed_imports, etc.).

Public schema promotion of version_info_struct (containing consumer-
facing metadata like CompanyName, ProductVersion, OriginalFilename) is
deferred to a coordinated future release with corresponding fixture
corpus refresh.

## Deferred to next release

* TLS Directory parser and validator (originally deferred from this
  release).
* ~25 single-anomaly fixtures targeting the new reason codes, including
  negative-control fixtures.
* Cross-tool divergence study using the new fixtures, particularly for
  delay-load's three spec-interpretation questions (v0/v1 mode, INT/IAT
  mirror vs bound assumption, declared-size vs walk-to-terminator).
* SUBLANG table refinement for multilingual LCID edge cases.
* Heuristic work items filed: CFG declaration consistency,
  implausibility framework, intra-field DllCharacteristics flag
  dependencies.

Author: malx-labs

CHANGELOG.md

@@ -1,4 +1,127 @@
+# **v0.7.5 - Structural validator expansion**
+**Released: 2026‑07‑01**
+
+This release substantially expands IOCX's structural validator suite with four new parser/validator pairs (export tables, delay-load imports, VS_VERSIONINFO, and the resource directory's Type → Name → Language hierarchy), enriches public metadata with security-relevant Optional Header and per-resource fields, and adds 24 new structural reason codes across exports, resources, and delay-load. All new code lands with 100% line and branch coverage backed by real-binary verification.
+
+## Added
+
+### New structural validators
+
+- **Export table parser and validator** (pe_exports, exports). Decodes the 40-byte IMAGE_EXPORT_DIRECTORY, EAT, ENPT, and EOT purely from bytes; emits ten new reason codes covering directory, name-pointer, and function-entry anomalies. Forwarder detection follows the PE-spec rule (address RVA within export directory range). Absence of an export directory is not treated as a defect.
+- **Delay-load import parser and validator** (parser_delay_imports, validator_delay_imports). Decodes the 32-byte IMAGE_DELAY_IMPORT_DESCRIPTOR, INT, and IAT purely from bytes; emits eight new reason codes covering directory, descriptor, and entry anomalies. PE32+ vs PE32 thunk sizing determined once from OPTIONAL_HEADER.Magic; v1 vs v0 attribute mode captured explicitly rather than coerced; bound state detected by bound_iat_rva != 0. Absence of a delay-load directory is not treated as a defect.
+- **VS_VERSIONINFO parser and validator** (pe_version_info, validator_version_info). Decodes the version-info envelope, VS_FIXEDFILEINFO, StringFileInfo, and VarFileInfo purely from bytes; emits four new reason codes covering header, FixedInfo, StringFileInfo, and VarFileInfo malformations. Leaf selection across multiple RT_VERSION entries is deterministic, sorted by (name_id, language_id). Absence of RT_VERSION is not treated as a defect.
+- **Resource hierarchy enforcement.** Resource validator now tracks tree depth and enforces the PE spec's Type → Name → Language layering, emitting two new reason codes (RESOURCE_DIRECTORY_LANGUAGE_NOT_ID, RESOURCE_DATA_AT_INVALID_DEPTH).
+
+### Reason codes added (24 total)
+
+- **Resource hierarchy (2):** RESOURCE_DIRECTORY_LANGUAGE_NOT_ID, RESOURCE_DATA_AT_INVALID_DEPTH
+- **VS_VERSIONINFO (4):** RESOURCE_VERSIONINFO_INVALID_HEADER, _INVALID_FIXEDINFO, _INVALID_STRINGFILEINFO, _INVALID_VARFILEINFO
+- **Exports (10):** EXPORT_DIRECTORY_INVALID_HEADER, _OUT_OF_BOUNDS, EXPORT_TABLE_TRUNCATED, EXPORT_NAME_RVA_INVALID, _NOT_ASCII, _POINTER_TABLE_UNSORTED, _ORDINAL_INDEX_INVALID, EXPORT_ORDINAL_OUT_OF_RANGE, EXPORT_FUNCTION_RVA_INVALID, EXPORT_FORWARDER_MALFORMED
+- **Delay-load (8):** DELAY_IMPORT_DIRECTORY_INVALID_HEADER, _OUT_OF_BOUNDS, DELAY_IMPORT_TABLE_TRUNCATED, DELAY_IMPORT_DESCRIPTOR_INVALID, DELAY_IMPORT_DLL_NAME_INVALID, _INT_IAT_MISMATCH, _ATTRIBUTES_LEGACY_VA_MODE, DELAY_IMPORT_ENTRY_INVALID
+
+All new codes follow the established pattern: priority-resolved sub-reasons surfaced via details["reason"], with sub-table scoping via details["table"] where applicable.
+
+### Public metadata enrichment
+
+- **Optional Header fields.** New fields in the optional_header block: dll_characteristics (raw value), dll_characteristics_flags (decoded flag names sorted by bit position), dll_characteristics_unknown_bits (hex string for unrecognised bits), win32_version_value, loader_flags, stack_reserve_size, stack_commit_size, heap_reserve_size, heap_commit_size.
+- **Header decoding.** New subsystem_name field in the header block, decoded from IMAGE_SUBSYSTEM_* (e.g., "WINDOWS_CUI"); returns null for unknown values. Raw subsystem integer unchanged. New machine_name field decoded from IMAGE_FILE_MACHINE_* (e.g., "AMD64", "I386", "ARM64"); the supporting MACHINE_NAMES table covers all 29 documented machine types.
+- **Resource metadata.** The resources field now exposes a structured ResourceEntry per resource covering type, name, language, language_name, codepage, size, entropy, rva, raw_offset, and errors. Per-resource Shannon entropy is rounded to 4 decimal places. Output is sorted by (type, language, rva) for snapshot stability. Resources whose data bytes cannot be read are no longer silently dropped; they are emitted with errors populated (tags: size_invalid, rva_invalid, data_out_of_bounds, raw_offset_invalid).
+
+### Schema typing
+
+- New TypedDicts: ExportStruct, ExportDirectoryHeader, ExportFunctionEntry, ExportNamePointerEntry, DelayImportStruct, DelayImportDescriptor, DelayImportEntry, VersionInfoStruct and its sub-types (FixedFileInfo, StringFileInfo, StringTable, VarFileInfo, VarEntry, Translation).
+- InternalMetadata.resources_struct is now Optional[ResourcesStruct] with a fully-typed ResourceEntry shape replacing List[Any]. InternalMetadata.export_struct, delay_import_struct, and version_info_struct are typed as Optional of their respective structs.
+- New constants module iocx.parsers.pe_constants houses SUBSYSTEM_NAMES, MACHINE_NAMES, DLL_CHARACTERISTICS_FLAGS, and the derived DLL_CHARACTERISTICS_KNOWN_MASK.
+
+## Changed
+
+### Validator dispatcher
+
+Three new validators registered in the structural validator chain, in order:
+
+```
+validate_resources → validate_version_info → validate_exports → validate_delay_imports → validate_entropy
+```
+
+This completes the resource and import/export structural validator clusters ahead of the entropy/derived layer.
+
+### Parser robustness
+
+- **Guarded RVA→offset conversion.** pe.get_offset_from_rva calls are now guarded against pefile.PEFormatError and AttributeError. A corrupt RVA produces a -1 sentinel in raw_offset rather than propagating the exception. The validator's existing data_raw < 0 arm maps this to the existing RESOURCE_DATA_OUT_OF_BOUNDS reason code; no new code introduced.
+- **Resource entropy now computed over the correct byte range.** Previously sliced get_memory_mapped_image() with the raw file offset; now correctly uses the RVA. Caught before snapshot stamping; entropy values match the previous release's behaviour for all existing fixtures.
+
+### Schema and decoding semantics
+
+- **ResourceEntry schema expanded.** Fields are now total=False Optional to accommodate per-entry computation failures. New fields: codepage, errors. The name, rva, and raw_offset fields are now populated where they were previously declared but absent from output.
+- **_decode_langid returns None for undecodable LANGIDs** (previously returned the magic string "unknown"). The early-return guard if langid < 0x0400 was removed; it was rejecting valid neutral-sublang LANGIDs (LANGID 0x0001 now correctly decodes as "ar" for Arabic).
+- **Optional Header missing-field convention.** New Optional Header fields use None as the default value when pefile cannot extract the field, distinct from 0 which indicates the binary's actual value. This is a deliberate semantic split for security-relevant fields where "missing" and "zero" carry different meaning. Existing Optional Header fields retain their default-to-zero behaviour for backward compatibility.
+
+### Extended analyser refactor
+
+- **analyse_extended cleaned up.** Removed duplicated _SUBSYSTEM_MAP and _MACHINE_MAP lookup tables now that decoded names come from the parser layer. The legacy subsystem_human and machine_human fields are removed from extended metadata output. Resource entropy summary statistics (entropy_min, entropy_max, entropy_avg) now compute over only entries with computed entropy values, excluding entropy: None entries.
+
+## Fixed
+
+- **Resource validator no longer silently returns** when a directory's own RVA falls outside .rsrc. Previously suppressed any reporting for malformed directory placement.
+
+## Documentation
+
+- **Reason codes reference** extended with new top-level sections for Resource Hierarchy Anomalies, Resource Version-Info Anomalies, Export Anomalies, and Delay-Load Import Anomalies, each with dedicated sub-reason taxonomy sections documenting the details["reason"] and details["table"] contracts.
+- **Validator documentation** gained sections 2.5 (VS_VERSIONINFO), 2.6 (exports), and 2.7 (delay-load imports), each with explicit determinism rationale. Section 2.7 frames the three spec-interpretation questions (v0 vs v1 attribute mode, INT/IAT mirror vs bound assumption, declared-size vs walk-to-terminator) that produce cross-tool divergence.
+- **Schema reference** documents the new Optional Header fields (including the dll_characteristics_unknown_bits role in preserving complete information about non-decoded bits) and the mixed-default convention with rationale.
+- **analyse_extended module purpose** documented inline via module docstring clarifying that the module performs shape conversion and derived-statistics computation only, not new information extraction.
+- **_decode_langid semantics** documented inline (primary language and sublang decomposition, fallback to default region, fallback to primary-language-only, fallback to None).
+
+## Internal
+
+- **100% line and branch coverage** on all new modules: pe_version_info, validator_version_info, pe_exports, exports, parser_delay_imports, validator_delay_imports, _parse_optional_header, _parse_header, pe_constants, and resource validator additions.
+- **Defensive-path coverage** via monkeypatched struct.error injection across all four parsers. Narrow-except negative tests confirm parser exception handling does not silently swallow exceptions outside the documented catch list.
+- **One # pragma: no cover** applied to a defensive return in the exports validator's _first_unsorted_index helper, documented inline as unreachable from the caller.
+- **End-to-end binary verification.** Delay-load parser cross-checked against mspaint.exe via dumpbin /imports: 107 imports decoded from gdiplus.dll's delay-load directory with byte-exact agreement on DLL name, hint values, IAT addresses, ordering, and bound state across both tools.
+
+**Total: ~650 new tests bringing the suite to 1370 tests.**
+
+## Compatibility
+
+### No remapping of existing reason codes
+
+Existing fixture expected outputs are unchanged for binaries that don't exercise the new pathways.
+
+### Snapshot refresh required
+
+The following changes will produce diffs in fixture expected outputs and require a coordinated snapshot refresh:
+
+- Resource fixtures gain new ResourceEntry keys (codepage, errors, name, rva, raw_offset); resources with errors now appear where they were previously silently dropped.
+- language_name no longer returns "unknown"; consumers checking language_name == "unknown" must update to language_name is None.
+- Every fixture with an optional header gains new Optional Header keys (dll_characteristics, _flags, _unknown_bits, win32_version_value, loader_flags, stack/heap sizing fields).
+- header block gains subsystem_name and machine_name keys.
+- Extended metadata no longer emits subsystem_human and machine_human; consumers should use subsystem_name and machine_name (the parser layer is now the single source of truth for both).
+
+All refreshes are mechanical via the existing fixture regeneration tooling.
+
+### No public IOC schema changes in this release
+
+export_struct and delay_import_struct are exposed only in internal metadata by design - they feed validators, not consumer schema. Public IOC schema exposure for version-info is deferred to a coordinated future release with corresponding fixture corpus refresh.
+
+## Known scheduled work
+
+- **Single-anomaly fixtures** targeting the new reason codes across resources, version-info, exports, and delay-load (~25 fixtures planned). Includes negative-control fixtures (exp_forwarder_to_ordinal_valid, delay_well_formed_bound_modern, delay_well_formed_unbound_with_ordinal_import) demonstrating that validators do not false-positive on healthy spec-valid inputs.
+- **Public IOC schema promotion** of version_info_struct planned for a future release with corpus refresh. Contains consumer-facing metadata (CompanyName, ProductVersion, OriginalFilename, FileDescription) with established IOC value.
+- **Internal-only structural data.** export_struct, delay_import_struct, and Load Config metadata remain internal by design — they exist to feed validators and heuristics, not the public IOC schema. Consumers needing structural information about these directories should rely on the validators' reason codes and (for exports/delay-load) the existing consumer-facing metadata fields.
+- **SUBLANG table refinement.** The current implementation models sublang values as language-independent, which is incorrect for multilingual edge cases (sublang 0x02 means UK English with primary English, but Swiss German with primary German). A flat LCID → BCP-47 mapping is the structural fix; deferred since current behaviour is correct for the common case.
+- **TLS Directory parser and validator.** Originally deferred from this release; planned for the next release.
+- **Cross-tool divergence study** using the new fixtures. Delay-load is a particularly strong divergence candidate because the three identified spec-interpretation questions are known to produce inconsistent output across pefile, LIEF, Ghidra, and IDA. Tracked separately as a methodology contribution opportunity.
+- **Filed work items**:
+   - pefile_usage_policy.md documenting the deterministic-subset usage pattern.
+   - PE_CFG_DECLARATION_INCONSISTENT heuristic (DllCharacteristics ↔ Load Config GuardFlags cross-validator consistency).
+   - PE_FIELDS_IMPLAUSIBLE heuristic for "cannot exist in wild" binaries.
+   - PE_DLL_CHARACTERISTICS_INCONSISTENT heuristic for intra-field flag dependency violations (e.g., HIGH_ENTROPY_VA without DYNAMIC_BASE).
+   - Existing-field default migration (the 0 vs None inconsistency in OptionalHeaderInfo).
+
+---
+
 # **v0.7.4.1 — Windows‑Compatible PE Detection Hotfix**
+**Released: 2026‑05‑28**
 
 IOCX v0.7.4.1 removes the `python-magic` dependency, improves PE detection accuracy, and reduces IOCX’s attack surface.
 
@@ -19,6 +142,7 @@ IOCX v0.7.4.1 removes the `python-magic` dependency, improves PE detection accur
 ---
 
 # **v0.7.4 — Advanced Directory Parsing & Metadata Expansion**
+**Released: 2026‑05‑26**
 
 IOCX v0.7.4 significantly expands static PE coverage with advanced directory parsing, extended metadata extraction, and deterministic structural validation. This release improves correctness across modern compiler outputs while preserving IOCX’s static‑only, zero execution design.
 

README-pypi.md

@@ -40,21 +40,19 @@ If you need predictable, automatable IOC extraction — IOCX is built for you.
 
 ---
 
+## Version highlights (v0.7.5)
+
+- Added detection for malformed exports, delay-load tables, resources, VS_VERSIONINFO, and Optional Header fields via 24 structural reason codes
+- Surfaces security metadata — DLL characteristics flags, subsystem/machine decoding, per-resource Shannon entropy
+- Never crashes on malformed input — byte-level parsing with structured error tombstones
+- 1370 tests at 100% coverage — deterministic output, snapshot-stable, cross-verified against `dumpbin`
+
 ## Version highlights (v0.7.4.1)
 
 - Removed the `python-magic` dependency, which caused import failures on Windows systems
 - Added a pure‑Python file‑type detector for full cross‑platform portability
 - No behavioural changes to IOC extraction
-- The `--min-length` consistency fix is planned for **v0.7.5**
-
-## Version highlights (v0.7.4)
-
-- Full **Load Config Directory** parsing and validation
-- Extended Optional Header metadata for downstream heuristics
-- Structural anomaly heuristics (GuardCF, unmapped cookie, SEH issues)
-- Faster, more resilient PE Analysis
-- Raw IOC extraction remains world-class
-- Zero regressions across all workloads
+- The `--min-length` consistency fix is planned for **v0.7.6**
 
 ---
 

README.md

@@ -9,7 +9,7 @@
 
 <p align="center">
   <a href="https://pypi.org/project/iocx/"><img src="https://img.shields.io/pypi/v/iocx?logo=pypi&logoColor=white"></a>
-  <img src="https://img.shields.io/badge/tests-945_passed-brightgreen">
+  <img src="https://img.shields.io/badge/tests-1370_passed-brightgreen">
   <img src="https://img.shields.io/badge/coverage-100%25-brightgreen">
   <img src="https://img.shields.io/badge/python-3.12-blue">
   <a href="https://github.com/iocx-dev/iocx/actions"><img src="https://img.shields.io/github/actions/workflow/status/iocx-dev/iocx/ci.yml?label=build"></a>
@@ -202,6 +202,15 @@ Fast path — no PE parsing.
 <summary><strong>Show Version History</strong></summary>
 <br>
 
+### **v0.7.5 — Structural Validator Expansion**
+- Four new PE structural validators — exports, delay-load imports, VS_VERSIONINFO, and resource hierarchy
+- 24 new reason codes with priority-resolved sub-reason taxonomies
+- Deterministic byte-level parsing — no reliance on pefile's lazy interpretation
+- Security-relevant metadata — DLL characteristics, subsystem/machine name decoding, per-resource entropy
+- 1370 tests at 100% coverage — end-to-end verified against `dumpbin` on real binaries
+
+---
+
 ### **v0.7.4.1 — Windows Compatibility Hotfix**
 - Removed the `python-magic` dependency, which caused import failures on Windows systems
 - Added a pure‑Python file‑type detector for full cross‑platform portability