Skip to content

ROB: Decode all stream filters with short DecodeParms#3913

Open
VectorPeak wants to merge 1 commit into
py-pdf:mainfrom
VectorPeak:rob-short-decodeparms
Open

ROB: Decode all stream filters with short DecodeParms#3913
VectorPeak wants to merge 1 commit into
py-pdf:mainfrom
VectorPeak:rob-short-decodeparms

Conversation

@VectorPeak

@VectorPeak VectorPeak commented Jul 6, 2026

Copy link
Copy Markdown

This is a small robustness change for malformed PDFs. In normal PDFs, /DecodeParms should line up with /Filter, so this probably does not affect common valid files. I also think it is reasonable if maintainers decide this edge case is not worth handling.

What Problem This Solves

/Filter is the ordered decoding pipeline for a stream, while /DecodeParms only provides the per-filter options for that pipeline. In the normal case, they have the same length. If /DecodeParms is omitted entirely, pypdf already treats every filter as using default parameters.

The malformed edge case is when /DecodeParms is present but shorter than /Filter. decode_stream_data() currently walks the two arrays with zip(filters, decode_parms), and zip() stops as soon as the shorter sequence is exhausted. That means the decode loop can stop early instead of continuing with default parameters for the remaining filters.

For example:

/Filter [/ASCIIHexDecode /FlateDecode]
/DecodeParms []

or:

/Filter [/ASCIIHexDecode /FlateDecode]
/DecodeParms [<<>>]

The stream data still needs both decoding steps: first ASCIIHexDecode, then FlateDecode. With an empty /DecodeParms array, the current loop does not run any filter at all. With a one-entry /DecodeParms array, it only runs ASCIIHexDecode, so the output is still Flate-compressed bytes instead of the original payload. This PR keeps walking the full /Filter list and uses default parameters when a trailing /DecodeParms entry is missing.

Change

  • Iterate over every filter in the /Filter chain.
  • Use empty decode parameters for missing trailing /DecodeParms entries.
  • Preserve the existing NullObject handling and normal behavior for missing or fully aligned /DecodeParms.

Evidence

The added regression test builds a stream with two filters:

ASCIIHexDecode -> FlateDecode

It covers these /DecodeParms shapes:

missing
[{}, {}]
[{}]
[]

I also checked the same shape with a tiny generated PDF, so this is not only a direct StreamObject test. The PDF has one page whose content stream is encoded as Flate data wrapped in ASCIIHex data, with the stream dictionary intentionally using a short /DecodeParms array:

/Filter [/ASCIIHexDecode /FlateDecode]
/DecodeParms []

and:

/Filter [/ASCIIHexDecode /FlateDecode]
/DecodeParms [<<>>]

Reading that file through the normal path:

PdfReader(pdf_path)
  -> reader.pages[0]["/Contents"].get_object()
  -> stream.get_data()
  -> decode_stream_data()

now returns the original page content stream:

b'BT /F1 12 Tf 72 720 Td (Hello malformed DecodeParms) Tj ET\n'

Simulating the old zip(filters, decode_parms) loop shows the difference:

DecodeParms: []
old zip-style output sample: b'789C730A51D077335430'

DecodeParms: [<<>>]
old zip-style output sample: b'x\x9cs\nQ\xd0w3T04R\x08IS07\x02"\x03'

So with an empty /DecodeParms array, the old loop leaves the data at the ASCIIHex layer. With a one-entry /DecodeParms array, it runs ASCIIHexDecode but leaves the data at the Flate-compressed layer. The fixed loop keeps decoding through the full /Filter list and reaches the original content bytes.

Possible call chain / impact

A reader can reach this path when stream data is decoded through decode_stream_data(), including malformed PDFs whose stream dictionary has an aligned /Filter array but a short /DecodeParms array.

This PR only changes the fallback behavior for missing trailing decode-parameter entries. It does not change predictor handling, individual filter implementations, stream writing, or already aligned /Filter + /DecodeParms cases.

Validation

  • mamba run -n prw-pypdf-py3.14 python -m pytest tests/test_filters.py -k "short_decode_parms or flate or asciihex" -q - passed, 23 tests
  • mamba run -n prw-pypdf-py3.14 ruff check pypdf/filters.py tests/test_filters.py - passed
  • git diff --check - passed

I also tried the full tests/test_filters.py file locally. It did not complete in this workspace because several cached PDF/image fixtures under tests/pdf_cache are missing; the focused regression and nearby filter tests above pass.

AI assistance

AI assistance was used to identify and review this small robustness change. The final patch and PR text were reviewed before submission.

@VectorPeak
VectorPeak force-pushed the rob-short-decodeparms branch from cf8bac6 to bae737f Compare July 6, 2026 13:50
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.86%. Comparing base (efc511b) to head (bae737f).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3913   +/-   ##
=======================================
  Coverage   97.86%   97.86%           
=======================================
  Files          57       57           
  Lines       10738    10739    +1     
  Branches     2006     2006           
=======================================
+ Hits        10509    10510    +1     
  Misses        127      127           
  Partials      102      102           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant