Skip to content

CONTRIBUTIONS.md

Latest commit

 

History

History
88 lines (67 loc) · 3.73 KB

CONTRIBUTIONS.md

File metadata and controls

88 lines (67 loc) · 3.73 KB
Raw

Open Source Contributions — Token-Aware Rate Limiter Project

Started: 2026-03-16

Standalone Project

Item Details
Repo joaquinhuigomez/token-aware-rate-limiter
What Reference implementation: atomic Redis Lua token bucket, Jain's fairness index, real-time WebSocket dashboard, multi-tenant rate limiting for LLM APIs
Stack Python, FastAPI, Redis, Chart.js
Tests 25 passing (bucket algorithm, multi-tenant, API, fairness)
Status Public, complete

PR #1 — LiteLLM Bug Fix (highest impact)

Item Details
PR BerriAI/litellm#23775
Fixes #18730 — Concurrent requests bypass TPM rate limits
Target repo LiteLLM (39k stars)
What Fixed TOCTOU race condition in parallel_request_limiter.py. TPM was only updated after request completion, so concurrent requests all read current_tpm=0 and bypassed the limit. Fix: reserve estimated tokens (max_tokens) at pre-call time, reconcile with actual usage on completion, release on failure.
Files changed litellm/proxy/hooks/parallel_request_limiter.py, 1 new test file (4 tests)
Status ⏳ OPEN — awaiting maintainer review
Next steps Maintainers may request changes. Watch for CI results and review comments.

How to check status

gh pr view 23775 --repo BerriAI/litellm --json state,reviews,statusCheckRollup

PR #2 — LiteLLM Fairness Proposal

Item Details
Comment BerriAI/litellm#20996 (comment)
Issue #20996 — Team-Level Dynamic Fair-Share Rate Limiting
What Proposed Jain's fairness index as the metric for their dynamic fair-share feature. Linked to standalone implementation as reference. Offered to write a follow-up PR if maintainers are interested.
Status ⏳ POSTED — awaiting maintainer response
Next steps If they express interest, write a PR adding fairness index to their Prometheus callback system.

How to check status

gh issue view 20996 --repo BerriAI/litellm --comments | tail -20

PR #3 — Awesome-LLMOps Listing

Item Details
PR tensorchord/Awesome-LLMOps#299
Target repo Awesome-LLMOps (5.7k stars)
What Added token-aware-rate-limiter to the LLMOps section of their curated list.
Status ⏳ OPEN — awaiting maintainer review

How to check status

gh pr view 299 --repo tensorchord/Awesome-LLMOps --json state

Notifications

Yes, you will receive email notifications for all of these:

  • GitHub sends emails to juangoo43@gmail.com (or whatever email is on your GitHub account) for:
    • PR review comments and approvals
    • CI check results (pass/fail)
    • Maintainer requests for changes
    • Merge or close events
  • Make sure notifications are enabled: github.com/settings/notifications
  • You can also check all activity with: gh notifications --repo BerriAI/litellm

Quick status check (all at once)

echo "=== LiteLLM PR ===" && gh pr view 23775 --repo BerriAI/litellm --json state,reviews,mergedAt
echo "=== Awesome-LLMOps PR ===" && gh pr view 299 --repo tensorchord/Awesome-LLMOps --json state,mergedAt
echo "=== Fairness Issue ===" && gh issue view 20996 --repo BerriAI/litellm --json comments --jq '.comments | length'