Skip to content

Add generated TUS protocol canary#114

Draft
kvz wants to merge 95 commits into
mainfrom
tus-gen
Draft

Add generated TUS protocol canary#114
kvz wants to merge 95 commits into
mainfrom
tus-gen

Conversation

@kvz

@kvz kvz commented May 26, 2026

Copy link
Copy Markdown
Member

Why

This is the Python companion canary for the generated TUS protocol SDK work. It proves tus-py-client can consume contract-owned protocol facts, generated conformance fixtures, and checked devdock examples without duplicating TUS wire knowledge in the Python repo.

Experimental Status

Experimental work: this PR is part of an exploratory contract/generated SDK effort, and is intentionally kept as a Draft while the approach is validated across TUS clients.

What

  • Adds generated protocol constants and contract fixtures for the TUS Python client.
  • Adds handwritten checked examples that read generated scenario JSON and prove real Transloadit/tusd behavior via the shared devdock runner.
  • Adds generated start-option validation facts plus a small Python interpreter for those facts, so conflicting options fail before transport while messages remain contract-owned.
  • Adds generated detailed-error facts plus Python request/response error wrapping for create-upload failures, keeping detailed message text and missing-value markers contract-owned.
  • Adds a reusable local conformance-plan server for Python examples and proves relative Location resolution from contract-owned request/response fixtures.
  • Adds generated method-override facts and routes sync/async chunk requests through a generated method plan, proving override_patch_method with POST plus X-HTTP-Method-Override: PATCH.
  • Adds generated URL-storage key policy helpers and updates FileStorage to use them for new records while preserving legacy raw-key lookup/update behavior.
  • Adds generated protocol-selection helpers for per-protocol request headers, upload body content types, and upload-complete headers, proving draft-05 creation-with-upload without hard-coded wire headers.
  • Adds generated abort-policy constants and Python abort mechanics, proving abortUpload by aborting an active create request and aborting a patch request followed by DELETE termination of a known upload.
  • Adds a path input-source proof for the contract-owned tusNodePathInputSource scenario by writing the scenario content to a temporary file and uploading it through Python's file_path surface.
  • Adds retry-state mechanics and proof for the contract-owned tusRetryStateTransitions scenario: retry_delays drives retry count/delay in milliseconds, on_should_retry(error, retry_attempt) observes/overrides retry decisions, and retry attempts reset after offset recovery observes server-side progress.
  • Adds parallel-upload concat support and proof for the contract-owned tusParallelUploadConcat scenario: Python creates partial uploads, uploads file ranges, reports aggregate callbacks, then creates the final concat upload from the partial URLs.
  • Keeps generated output rewrite-and-compare checked from the generator side.

Latest Verification

  • Branch tus-gen is pushed at 0969ad6.
  • python3 -m py_compile examples/api2-devdock-tus-parallel-upload-concat/main.py tusclient/uploader/baseuploader.py tusclient/uploader/uploader.py tusclient/protocol_generated.py passed.
  • python3 -m pytest tests/test_uploader.py -q passed with 34 tests.
  • python3 -m pytest -q passed with 73 tests.
  • The external devdock wrapper passed all 24 checked Python/TUS examples, including tusParallelUploadConcat, tusRetryStateTransitions, tusNodePathInputSource, tusAbortUpload, tusAbortUploadAfterStoredUrl, tusProtocolVersionSelectionDraft05CreationWithUpload, transloaditTusAssemblyFileUrlStorageBackend, tusOverridePatchMethod, and tusRelativeLocationResolution.
  • GitHub Actions passed for 0969ad6 across ubuntu/windows Python 3.8-3.13.

Related TUS PRs

kvz and others added 5 commits June 12, 2026 17:22
The abort/termination retry loop now comes from the shared statement-IR
procedure (terminate-upload-with-retry): tusclient/abort_generated.py owns
the algorithm, and TusClient.terminate_upload delegates to it with an empty
retry budget to keep its historical single-attempt behavior. The HTTP
terminate request stays a handwritten transport shim.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The uploader's per-chunk patch-with-retry now runs on the generated
upload_chunk_with_retry (tusclient/upload_chunks_generated.py), emitted
from the shared TUS protocol contract retry core. The handwritten
_do_request/_retry_or_cry recursion is delegated: the patch transport
stays in _perform_patch_request (TusRequest machinery, accepted-state
absorption), offset recovery failures re-enter the retry decision
iteratively exactly as the recursion did, and the no-status-filter
retry semantics plus the _retried counter surface are preserved. The
legacy retries/retry_delay options map onto the delays-indexed budget
via _upload_retry_delays_ms. The async uploader keeps its own loop for
now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AsyncUploader's handwritten _do_request/_retry_or_cry recursion delegates
to the generated async upload_chunk_with_retry, the same walked per-chunk
retry procedure as the sync runtime with await at the patch and sleep
effects. The retry decision helpers stay imported from the sync modules,
and the legacy _retry_limit/_retry_delay_seconds/_should_retry helpers
leave baseuploader with their last consumer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Method-override interpretation is keyed by operationId, and the contract
derives each override's sourceMethod from that same operation
(tusClientMethodOverrideModels), so after an operationId match the
source_method comparison could never fire for contract-consistent
callers — both generated call sites pass the UPLOAD_CHUNK constants
derived from the very operation the override names. This converges the
Python interpretation with the TypeScript runtime
(tusMethodOverrideForOperation over the sourceMethod-less
tusClientRuntimeMethodOverrideModels). source_method stays the
no-override fallback method, mirroring the TS plan-method fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant