[Bee] Incorrect reading order for Key Value regions

[yqliving]

Requested feature

For some content in the PDF document, it includes multiple columns or blocks with key and value regions, and the structure is not recognized as tables; the reading order should be key first, value second, one by one, but in the markdown results, they are mixed together.

For example:
file content

result:

There are several testing documents:

1. Summary section in test_kvp_reading_order.pdf, programmatic PDF
2. form section in AB.FILES.038670001-02-PDF_A_MockUp-page-5.pdf, Form data, scanned PDF, but the section is not recognized as table.

[dosubot]

This is a known limitation with how the standard PDF pipeline handles FORM and KEY_VALUE_REGION layout labels. When the layout model detects these regions, they're processed as ContainerElement groups in the reading order model [1] — the child elements within the container are linearized using a generic reading order algorithm that doesn't account for the spatial key→value pairing pattern (left-to-right or label-above/value-below).

Several related issues track this same class of problem:

• #3454 — Table Incorrectly Identified as Form Area in Scanned PDF
• #3653 — How to fix bad results on some pages that look like a Form [2]

There are currently no pipeline configuration options to force key-value regions to be treated differently or to override the reading order within these containers [3].

Workaround — use the VLM pipeline, which bypasses traditional layout detection and handles form-like layouts better:

from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import VlmPipelineOptions, VlmConvertOptions
from docling.pipeline.vlm_pipeline import VlmPipeline

vlm_options = VlmConvertOptions.from_preset("granite_docling")
pipeline_options = VlmPipelineOptions(vlm_options=vlm_options)

converter = DocumentConverter(
    format_options={
        InputFormat.PDF: PdfFormatOption(
            pipeline_cls=VlmPipeline, pipeline_options=pipeline_options
        )
    }
)
result = converter.convert("your_document.pdf")
print(result.document.export_to_markdown())

Note: The VLM pipeline requires a GPU for reasonable performance and is slower than the standard pipeline.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[zeweihan]

Great, precise bug report — and the key-value reading-order case is where doc parsers do their most dangerous silent damage, because it looks like a cosmetic ordering nit when it's actually a binding break.

1. Reading-order error on key/value regions is a key↔value binding break, not a shuffle.
Every token survives (smoke test "are all the words present?" passes), and OCR is correct on each token. What silently changes is which value attaches to which key. That's the same failure family as table cell-to-column misattribution (#3698) and OCR-merged-word repair (#3654): content survives, structure/attribaution silently remaps, no error, full retrieval confidence downstream. The OP's screenshot is the textbook signature — nothing is missing, the dates/labels just swapped owners.

2. KEY_VALUE_REGION that is not recognized as a table is the worst case, not an edge case.
When the layout model does label a region TABLE, the table-structure model runs and produces explicit (row, col) bindings that survive linearization — the key→value edge is encoded structurally. When it's labeled KEY_VALUE_REGION but processed as a ContainerElement linearized by generic reading order (as dosubot notes), the only thing carrying the key→value edge is spatial proximity + alignment — and that's exactly what reading-order linearization discards. So the regions where binding is hardest to recover are the ones where the structural fallback is weakest. For regulated corpora this is backwards: form-like intake pages, signature/date blocks, compliance checklists, and exhibit cover sheets are predominantly KEY_VALUE_REGION rather than ruled tables.

3. Blast radius is asymmetric and lands on operative fields.
These are the pairs disputes turn on. Consider a two-column block: Effective Date: 2024-03-01 | Termination Date: 2025-12-31. If reading order interleaves them, the two dates can swap keys and every token is still present, the markdown still renders two dates next to two labels — but the binding is reversed, which is the opposite legal meaning. Same shape for checkbox pairs (☐ Amendment / ☑ Original) where the checked/unchecked state is the document's operative assertion, and for First Name/Last Name signature fields. None of these produce a degradation signal; the consumer gets a fluent, confident answer resting on a swapped binding.

4. Companion signal — make the pairing reconstructable, not just linearized.
The cheapest high-leverage fix isn't a better reading-order model; it's to preserve the key→value edge explicitly so that even when linearization order is wrong the binding is recoverable. Emit a per-KEY_VALUE_REGION pairing annotation:

{
  "key_span": {text, bbox, page},
  "value_span": {text, bbox, page},
  "pairing_basis": "spatial" | "alignment" | "reading_order",
  "confidence": 0.0–1.0
}

Then: (a) downstream RAG/chunkers can re-derive key→value from spatial/alignment even when reading_order linearization scrambled it; (b) low-pairing_basis=reading_order confidence routes to a review queue instead of silently producing interleaved prose; (c) a defensibility test becomes possible — "every value in this retrieved answer must trace back to a key via an explicit pairing edge" — which is exactly the invariant reading-order-only output currently cannot satisfy.

The unifying thread with the table/heading/OCR-misattribution bugs: content survives, the binding silently remaps, and the only thing that catches it is preserving the structural edge that linearization throws away. Key→value is one more load-bearing edge; serializing it explicitly is the fix that generalizes across the whole class.