Skip to content
View madhugoutham's full-sized avatar
馃幆
Focusing
馃幆
Focusing
1 follower · 2 following

Achievements

Achievement: Pull Sharkx2Achievement: QuickdrawAchievement: Pair Extraordinaire

Achievements

Achievement: Pull Sharkx2Achievement: QuickdrawAchievement: Pair Extraordinaire

Organizations

@llm-d

Block or report madhugoutham

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don鈥檛 include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user鈥檚 behavior. Learn more about reporting abuse.

Report abuse
madhugoutham/README.md

Madhu Goutham 馃憢

Senior ML Engineer at Red Hat AI Inference Engineering. Kubernetes-native distributed LLM inference.

Working on llm-d 路 KServe 路 Gateway API Inference Extension 路 vLLM

Interests P/D disaggregation 路 inference scheduling 路 KServe LLMInferenceService 路 EPP scorers 路 KEDA autoscaling

Pinned Loading

  1. llm-d/llm-d llm-d/llm-d Public

    Achieve state of the art inference performance with modern accelerators on Kubernetes

    Shell 4k 678

  2. vllm vllm Public

    Forked from vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python 1

  3. llm-d-latency-predictor llm-d-latency-predictor Public

    Forked from llm-d/llm-d-latency-predictor

    Latency prediction service for ML-model based scoring with llm-d-inference-scheduler

    Python

  4. llm-d-router llm-d-router Public

    Forked from llm-d/llm-d-router

    llm-d Router: The intelligent entry point for inference requests

    Go