You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+29-5Lines changed: 29 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,8 @@
1
1
# GIPHY Bandwidth Saver
2
2
3
-
GIPHY Bandwidth Saver is a local demo proxy that validates GIPHY media with the origin while avoiding redundant media downloads when the same GIF rendition has already been observed. It is intentionally not a login proxy, not a crawler, and not an offline mirror. The design keeps GIPHY in the request path for freshness and removal semantics while reducing repeated transfer of identical media bytes.
3
+
GIPHY Bandwidth Saver is an MIT-licensed reference implementation for a revalidating GIPHY media cache. It includes a local website/API proxy so you can demo the behavior against the live GIPHY website, plus conformance harnesses that other implementations can adapt to prove request accounting, revalidation, replacement, deletion, and range behavior.
4
+
5
+
This repository is intentionally not a drop-in production proxy, not a login proxy, not a crawler, and not an offline mirror. The reusable idea is narrower: keep GIPHY in the request path for freshness and removal semantics while avoiding redundant media-byte downloads when the same GIF rendition has already been observed.
@@ -20,7 +22,7 @@ This project demonstrates a corporate-friendly middle ground:
20
22
21
23
The result complies with the spirit of origin-controlled media delivery: GIPHY continues to receive validation traffic with enough request context to inform popularity-driven ranking and removal decisions, while consumers avoid unnecessary bandwidth spend and preserve user anonymity.
22
24
23
-
## Runtime
25
+
## Demo Quick Start
24
26
25
27
This project uses pnpm and Node 24.
26
28
@@ -37,7 +39,9 @@ Then open:
37
39
https://web.localhost:8080
38
40
```
39
41
40
-
The server listens on `127.0.0.1` and uses local HTTPS with HTTP/2 by default. That default matters: without HTTP/2, the browser-to-proxy side no longer behaves like the CDN path and media loading looks much slower than the cache design actually is. You can override runtime paths with:
42
+
This starts the local website proxy for `giphy.com`, rewrites page/API/media URLs to local hostnames, and routes media requests through the revalidating media cache. Scroll the proxied GIPHY homepage to load GIFs, then scroll back through already-seen media while watching the TUI. Repeated media should still be revalidated upstream, but upstream `304` responses should let the proxy serve local bytes and count avoided downloads.
43
+
44
+
The server listens on `127.0.0.1` and uses local HTTPS with HTTP/2 by default. That default matters for the demo: without HTTP/2, the browser-to-proxy side no longer behaves like the CDN path and media loading looks much slower than the cache design actually is. You can override runtime paths with:
41
45
42
46
```sh
43
47
PORT=8081 CACHE_DIR=cache pnpm dev
@@ -118,7 +122,7 @@ The web and API proxies rewrite GIPHY host strings based on response content typ
118
122
119
123
## Code Boundaries
120
124
121
-
The demo-only website/API rewriting code is intentionally isolated from the media cache. This keeps the reusable bandwidth-saving path obvious.
125
+
The demo-only website/API rewriting code is intentionally isolated from the media cache. This keeps the reusable bandwidth-saving reference path obvious.
122
126
123
127
Demo proxy files:
124
128
@@ -139,6 +143,8 @@ Shared infrastructure:
139
143
140
144
Architecture tests assert that media cache modules do not import demo rewrite helpers and demo rewrite modules do not import cache storage.
141
145
146
+
For another implementation, the media-cache contract is the part to study first. The website/API proxy is scaffolding for making the reference implementation easy to run in a browser.
147
+
142
148
## Upstream Networking
143
149
144
150
All upstream GIPHY calls use an explicit HTTPS keep-alive agent instead of relying on Node's global defaults. The agent keeps HTTP/1.1 connections reusable across requests, caps socket fan-out, and keeps a bounded pool of idle sockets:
@@ -276,7 +282,7 @@ Important edge-case contracts:
276
282
277
283
### Test Harness
278
284
279
-
The E2E tests use a simulated target media server and a conformance client:
285
+
The E2E tests double as validation harnesses for the reference implementation. They use a simulated target media server and a conformance client:
280
286
281
287
- The target server accepts a response program per asset: repeated `304`s, replacement `200`s, deletion `404` / `410` steps, weighted transition windows, delays, and payload bodies.
282
288
- The client sends unique fingerprints on every request and can vary method, range headers, `If-Range`, and browser validators.
The stress scenarios cover tens of thousands of unique fingerprints across shared media GIF IDs, renditions (`giphy`, `200`, `200w`, `100`), and file types: GIFs as `.gif`, `.mp4`, and `.webp`; Clips as `.mp4`; Stickers as `.gif` and `.webp`.
298
304
305
+
To validate your own media proxy, adapt the harness model documented in [docs/media-proxy-conformance-harness.md](docs/media-proxy-conformance-harness.md): send each client request with a unique `x-giphy-conformance-fingerprint`, preserve that header on upstream validation/download requests, record target-server receipts, and compare planned fingerprints, client responses, upstream receipts, and final cache state. The lowest-friction path is to reuse the target server, synthetic client, scenario definitions, and assertion ledger under `test/support/`, then replace this repo's proxy startup with your proxy under test.
306
+
307
+
Use these helper scripts when shaping or debugging another implementation:
308
+
309
+
-`pnpm cache:summary [path/to/metadata.sqlite]` summarizes a local cache database into safe aggregate fixture shape data.
310
+
-`pnpm media:characterize <media-url> [more URLs]` probes live media URL semantics such as `HEAD`, validators, negative responses, and range behavior without storing payload bytes.
311
+
299
312
### Browser Cache Validation
300
313
301
314
To validate the browser cache layer in Chrome or Edge:
@@ -317,6 +330,17 @@ Expected browser behavior:
317
330
318
331
If you do not see browser disk-cache hits or browser-facing `304` responses, first confirm `Disable cache` is off. If it is off, the most likely cause is a certificate error preventing Chrome from caching and reusing images for that local HTTPS origin. In that case, the proxy disk cache is still working and will still avoid upstream media downloads after origin revalidation, but browser-to-proxy transfers will continue and the TUI will overstate end-user network bandwidth saved. Use `pnpm tls:mkcert` or `TLS_CERT_FILE` / `TLS_KEY_FILE` to run with a locally trusted certificate when validating browser cache behavior.
319
332
333
+
## Exercises Left To The Reader
334
+
335
+
This reference implementation keeps several production concerns explicit instead of hiding them behind demo code:
336
+
337
+
- Production deployment: packaging this behind a real CDN, load balancer, container image, process supervisor, or multi-region topology is out of scope.
338
+
- Multi-node cache sharing: the demo uses local SQLite plus disk files. A production implementation needs its own answer for shared storage, eviction, backpressure, and node-local hot caches.
339
+
- Policy integration: authentication, authorization, rate limiting, tenant isolation, abuse controls, and audit logging are not implemented here.
340
+
- Observability: the TUI is useful for local demos, but production use needs metrics, tracing, alerting, and structured logs.
341
+
- Standalone harness packaging: the conformance harness is written so another proxy can adapt it, but it is not currently published as a separate npm package or CLI.
342
+
- Browser automation: browser-cache validation is documented as a manual Chrome/Edge workflow. A production-quality validation suite could add Playwright coverage for DevTools-visible disk-cache and `304` behavior.
0 commit comments