Support a remote OpenAI-compatible API backend for chart extraction (ChartExtractionApiOptions)

[c-vongpaseut]

Requested feature

Allow Docling's chart extraction enrichment (do_chart_extraction) to run against a remote OpenAI-compatible Chat Completions endpoint (e.g. vLLM, Ollama, TGI, LM Studio) — the same way picture description already supports remote inference via PictureDescriptionApiOptions + enable_remote_services=True.

Concretely: an API-backed variant of ChartExtractionModelOptions (e.g. ChartExtractionApiOptions) exposing url, headers, params, prompt, timeout, and concurrency, so the chart→ CSV/code/summary step can be offloaded to a self-hosted VLM behind an HTTP API instead of loading ibm-granite/granite-vision-*-chart2csv locally through transformers.

[zeweihan]

Strong +1 — this is the right asymmetry to close. PictureDescriptionApiOptions already proved the remote-API pattern works; chart extraction is arguably a better fit for it than picture description, because the chart→CSV/code step is the most model-dependent (and most reproducibility-sensitive) part of the whole enrichment path.

One design point worth nailing down while the option shape is still open, from a regulated-domain (financial/legal corpora) angle: the reconstructed TableData emitted from a remote VLM is an operative data source, not a convenience cache. A payment-waterfall chart, a fee-tier bar chart, or an allocation schedule that gets rebuilt into structured rows is now the only copy of those numbers in the DoclingDocument — downstream RAG treats it as authoritative and cites it back to "Figure 3". So the moment chart extraction becomes an offloadable remote call, three provenance questions get silently promoted to load-bearing:

1. Which model produced these numbers? Remote endpoints drift (granite-vision-chart2csv → a server-side swap to a different VLM) with no change to caller code. The reconstructed fee tiers can shift between two runs of the same PDF and nothing in the PictureItem explains why. A chart_extraction_model / resolved-model field on the emitted item (mirroring what PictureDescriptionApiOptions already implies but rarely records) makes "which model read this chart on date X" answerable for an auditor — exactly the gap that bites when a chart-derived amount ends up in a disputed clause.

2. Reconstruction confidence is the wrong default to drop. Local granite-vision-*-chart2csv at least has a fixed, inspectable model identity. A remote endpoint adds a failure class where the VLM returns plausible-but-wrong CSV (smooth, well-formed, confidently-shaped numbers that don't match the raster) — visually indistinguishable from a correct extraction, no degradation signal, and the structured TableData looks healthier than the source image. Cheap companion: carry a chart_render_status / reconstruction-source flag on the item (remote-vlm vs local-transformers vs cached) so the shape of the derivation is at least inspectable, not just its existence.

3. Round-trip parity with feat(msexcel): parse native charts into classified pictures #3756. The native-chart PR (feat(msexcel): parse native charts into classified pictures #3756) already reconstructs TableData from series + shared categories; a remote-VLM path produces structurally-equivalent TableData but via a different derivation (pixels→VLM→CSV vs cell-reference→cached-workbook→TableData). Two items carrying the same TableData shape but different reconstruction provenance should stay distinguishable — same parent anchor, but a consumer that re-runs extraction for audit needs to know whether it's re-deriving from authoritative workbook values or re-asking a VLM whose answer may legitimately differ on the second pass.

Concrete ask that's cheap to bake in now: whatever ChartExtractionApiOptions lands as, have it stamp {resolved_model, endpoint_host, reconstruction_source} onto the emitted PictureItem (or its prov) rather than treating the remote call as a transparent transport. That keeps "I offloaded chart extraction to my own vLLM" from silently becoming "I can no longer reconstruct which derivation produced these numbers." Same provenance philosophy as the formula_render_status / text_coverage companions already in flight — never let a detected region's derivation vanish without a flag.