Pluggable JSON serialization/deserialization backend

[pauloxnet]

Code of Conduct

• I agree to follow Django's Code of Conduct

Feature Description

Make Django's JSON serialization and deserialization pluggable, so a project can choose which JSON library Django uses internally, the same way it can already choose a password hasher (PASSWORD_HASHERS + django[argon2]) or a storage backend (STORAGES). The standard library json module stays the default; switching to another library (e.g. orjson, or any future one) is opt-in.

Problem

Django core calls json.dumps / json.loads directly in roughly 40 places (http/response.py, core/serializers/json.py, core/signing.py, contrib/admin, contrib/messages, contrib/staticfiles, db/models/fields/json.py, forms/fields.py, utils/html.py, views/i18n.py, test/client.py, and others). A handful are already customizable per-call (JsonResponse(encoder=, json_dumps_params=), JSONField(encoder=, decoder=), serialize(cls=...), SESSION_SERIALIZER, SERIALIZATION_MODULES), but most are hardcoded to the stdlib json module.

The user-facing surfaces (JsonResponse, json_script, JSONField, Client, the serializers) are already pluggable per-call, and a third-party package like django-orjson by @adamchainz, already covers them opt-in. The genuine gap is the internal hardcoded call sites in contrib/admin, contrib/messages, contrib/staticfiles, views/i18n.py, db/models/fields/composite.py, forms/utils.py, etc. Those cannot be swapped from a third-party package without monkey-patching, which is fragile and not something the ecosystem should require of a project with millions of lines of code. The same class of gap was identified for the multipart parser in #105, and resolved there by making the parser pluggable on HttpRequest.

Request or proposal

proposal

Additional Details

This follows the template established by #105 / PR #20498 (pluggable multipart parser) and the PASSWORD_HASHERS + django[argon2] precedent: keep the current behavior as the default, define a uniform interface, and let users opt in via a setting rather than per-call overrides.

Per the guidance @carltongibson gave on #105 — define a uniform interface first, default to the current implementation, and treat the opt-in as a stepping stone — the first PRs would introduce no new setting and no behavior change: only route the hardcoded json calls through a single helper in django.utils.json (which already exists in main with normalize_json, added for django.tasks — currently a tiny ~15-line module that step 1 would expand). A later, DEP-gated step would add the opt-in setting and an optional packaging extra.

The proposal is library-agnostic. orjson is the most common choice in the ecosystem today, but msgspec-based backends (used by django-bolt and django-rapid by @FarhanAliRaza ) and others could target the same interface. The first implementation does not need to update every internal consumer (admin, messages, staticfiles, etc.) in one go: it only needs to establish the pluggability point, so that a project can install a library and have it picked up project-wide without changing application imports and without a dedicated wrapper package.

The practical difference for a large project is decisive. Today, adopting orjson via django-orjson means subclassing JsonResponse in every view, changing the test base class in every test module, and passing serializer= to every signing.dumps() call — potentially thousands of file changes in a large project — and the internal surfaces (admin, messages, staticfiles, i18n) remain unreachable without monkey-patching. Even medium-sized community projects like djangoproject.com (Django's own website, public code) or Django Packages would need to change every view import and test base class to adopt orjson today. After step 3, the same adoption is pip install django[orjson] plus three lines in settings.py: zero view changes, zero test changes, zero signing changes, and the previously unreachable surfaces are covered automatically. Rollback is removing three lines. This is the same push-vs-pull distinction as PASSWORD_HASHERS (configure once, framework pulls the backend into every call site) versus subclassing every hasher class manually.

Beyond speed, a non-stdlib backend like orjson also serializes datetime, UUID, dataclass, and enum.Enum natively — so JsonResponse({"created_at": timezone.now()}) works without passing encoder=DjangoJSONEncoder. That is a capability gain, not only a ~10x speed gain on dumps.

Related:

• Add pluggable request body parsers #105 — pluggable request body parsers (idea that inspired this approach)
• django/django#20498 — merged PR making the multipart parser pluggable on HttpRequest
• Trac #36841 — Trac ticket for #20498
• django/deps#88 — DEP 0017 (Content Type Parsing, renumbered from DEP 0015)
• DEP 0007 — Dependency Policy (governs how optional extras like django[orjson] would be added; note: orjson does not support PyPy, but since it's an optional extra, PyPy users simply don't install it and keep the stdlib default)
• Add project metadata for all optional dependencies #157 — Add project metadata for all optional dependencies (complementary proposal)
• SESSION_SERIALIZER — existing per-area serializer setting
• SERIALIZATION_MODULES — existing pluggable serializer registry
• Forum: "Testing django.tasks and understanding json Encoding" (Jan 2026) — TASK_SERIALIZER discussion where jacobtylerwalls noted consensus for a swappable serializer setting
• json_module parameter in django-modern-rest (PR #859) — closest published API contract for a swappable JSON module
• Trac #17942 — historical rejection of a global JSON_RESPONSE_DEFAULT_ENCODER setting; this proposal differs because it's a backend dict (like STORAGES), not a single-value encoder setting
• DjangoJSONEncoder — the encoder whose default behavior changes on opt-in (step 3)

Implementation Suggestions

A progressive sequence, in the style of #105 / #20498:

1. Refactor only — route the hardcoded json.dumps / json.loads calls in django/ through django.utils.json.dumps / .loads. No new setting, no behavior change. Default still stdlib. Delivered as several small PRs, one per coherent cluster (admin, serializers, HTTP/test, forms, etc.), each with its own release note, so regressions stay bisectable and reviews tractable. Two surfaces are explicitly carved out and stay on stdlib unconditionally, because swapping them is load-bearing rather than a perf win: core/signing.py (signed-token determinism depends on separators=(",", ":"); orjson does not accept separators as a parameter and cannot guarantee byte-identical output, which signed tokens require) and the bare json.loads(param) calls in db/models/fields/json.py lookup compilation (KeyTransform and friends parse values that came from the database; routing them through a swappable backend would silently change ORM lookup behavior). These calls keep using stdlib json directly even after step 3.
2. Introduce a backend abstraction — a BaseJsonBackend interface (dumps, loads) with a single StdlibJsonBackend implementation that preserves today's behavior exactly (including DjangoJSONEncoder as default, separators=(",", ":") in signing, the latin-1 encode step). Still no setting, still stdlib-only. BaseJsonBackend is documented as a public but stable API: third-party backends may depend on it, so signature changes require a deprecation cycle (the same standard STORAGES backends already meet).
3. Opt-in backend (DEP-gated) — a JSON_BACKENDS setting on the STORAGES shape (Django 4.2), plus optional packaging extras (e.g. django[orjson]) so a project can pip install django[orjson] and point the setting at an alternative backend. Per-call encoder= / decoder= / cls= keep working unchanged (the active backend transparently falls back to stdlib for that single call when a custom JSONEncoder/JSONDecoder is passed, since not every library supports that contract); the opt-in only affects the common path. Two behavior changes on opt-in must be documented in the DEP's "Backwards Compatibility" section: (a) JSONField(encoder=...) and JSONField(decoder=...) are never accelerated by the backend switch — they always fall back to stdlib for correctness, so users should not expect speedup there; (b) DjangoJSONEncoder is no longer the implicit default for JsonResponse, json_script, and the test client when a non-stdlib backend is active, so output for datetime / Decimal / UUID changes to the backend's native formatting. The bundled OrjsonJsonBackend ships a default= function replicating DjangoJSONEncoder's handling for the types orjson does not cover natively (Decimal → str, timedelta → duration_iso_string, Promise → str), limiting the practical output diff to datetime formatting — but it is still an intentional, documented output change that snapshot tests will surface. Step 3 will be accompanied by a reproducible benchmark and a blog post, per the contributing guide's "performance changes need benchmarks" rule.

Each step is independently mergeable and backwards-compatible. Step 3 requires a DEP in django/deps, on the model of DEP 0017.

[github-actions]

Thank you pauloxnet for sharing your idea! We have a lot of them so please be patient. You can see the current queue here.

Community instructions

For commenters, please use the emoji reactions on the issue to express support, and/or concern easily. Please use the comments to ask questions or contribute knowledge about the idea. It is unhelpful to post comments of "I'd love this" or "What's the state of this?"

Reaction Guide

• 👍 This is something I would use
• 👎 This is something that would cause problems for me or Django
• 😕 I’m indifferent to this
• 🎉 This is an easy win

[nanorepublica]

Just a note that I will happily help with this where in applies to contrib/messages - I am very slowly working on an internal refactor of it. Proposal is here: #155

[pauloxnet]

Just a note that I will happily help with this where in applies to contrib/messages - I am very slowly working on an internal refactor of it. Proposal is here: #155

Thanks @nanorepublica! That would be great: having you on board for the contrib/messages part is exactly the kind of overlap that makes this easier. Happy to coordinate when the time comes.

[adamchainz]

Thanks for seeing my project and writing this proposal @pauloxnet.

Beyond speed, a non-stdlib backend like orjson also serializes datetime, UUID, dataclass, and enum.Enum natively — so JsonResponse({"created_at": timezone.now()}) works without passing encoder=DjangoJSONEncoder. That is a capability gain, not only a ~10x speed gain on dumps.

Django's JsonResponse uses DjangoJSONEncoder by default, so the claim is false for datetime and UUIDs. Still true for dataclass and Enum.

That said, this "capability gain" also shows the biggest sticking point of such a migration the two packages are not interchangeable. If a project decides to use orjson and then swap back to stdlib json for some reason, it cannot be done in a safe manner. Any call site that depends on serializing a dataclass or Enum, for example, will fail.

1. core/signing.py (signed-token determinism depends on separators=(",", ":"); orjson does not accept separators as a parameter and cannot guarantee byte-identical output, which signed tokens require)

Thanks for pointing this out, but it's not quite true. Signed tokens do not depend on byte-identical output.

But what I did find out is that Django's JSON signing serializer decodes with latin-1. This means that a migration from orjson to stdlib json can trigger mojibake of non-ASCII data. See the notes added to django-orjson's docs pages for signing and sessions, from this PR.

---

Because of the above issues (extra serialized types and potential mojibake), I'm not so confident in the proposal. While orjson offers significant speed gains, adopting it carries migration risks and limitations. Keeping the migration blast radius limited through the piecemeal approach of django-orjson allows you to speed up your hotspots, like views returning JsonResponses, without introducing risk to the other call sites.

[pauloxnet]

Thanks for the thorough review @adamchainz , really appreciate you digging into the details.

On the type-support point, you're right: DjangoJSONEncoder already handles datetime and UUID, so the capability gain I claimed was overstated. The real gap is only dataclass and Enum. I'll correct that in the proposal.

That said, the gap itself suggests an idea I'd like your thoughts on: what if step 2 also enriches DjangoJSONEncoder itself to handle dataclass (via dataclasses.asdict()) and Enum (via .value)? That would mean every backend, including the default one, handles the same types. The only difference between backends would be speed, not capability. Switching from orjson back to stdlib would be safe because the default would also handle dataclass and Enum. It also has a side benefit: DjangoJSONEncoder handling those types is a feature gain for all Django users, not just those who opt into orjson. I'm planning to update the proposal along these lines, but wanted to check with you first.

On the mojibake point: thanks for documenting that so carefully in django-orjson PR #15. I looked into the root cause in django/core/signing.py: the JSONSerializer encodes with latin-1 and decodes with latin-1, which was introduced in 2012 (commit 58a086acfb) as part of the Python 3 bytes-interface porting. Since json.dumps() with default ensure_ascii=True always produces ASCII, latin-1 and UTF-8 produce identical bytes, so the choice was indistinguishable at the time. But with orjson (which outputs UTF-8 with raw non-ASCII), the latin-1 decode causes the silent mojibake you identified.

I verified that changing latin-1 to utf-8 in JSONSerializer.dumps() and loads() is backward-compatible: existing signed tokens contain only ASCII bytes (because ensure_ascii=True), which decode identically in both encodings. And json.loads() in Python 3 accepts bytes directly and decodes them as UTF-8, so loads() could simplify to json.loads(data) without any explicit decode step.

I'm thinking of opening a separate Trac ticket for that fix, it's independent of this proposal and would benefit the signing infrastructure regardless of whether JSON_BACKENDS ever lands. If applied, it would also make the signing/sessions carve-out in this proposal unnecessary. Would you be supportive of that, or do you see reasons to keep latin-1?

[jacobtylerwalls]

On the type-support point, you're right: DjangoJSONEncoder already handles datetime and UUID, so the capability gain I claimed was overstated.

Ah, but you mentioned tasks as well, and tasks doesn't use DjangoJSONEncoder. We're going to want pluggable serialization for tasks, I think, per the linked discussions.