Skip to content
@llm-d-incubation

llm-d incubation

Incubating components of llm-d, a Kubernetes-native high-performance distributed LLM inference framework

Popular repositories Loading

  1. llm-d-infra llm-d-infra Public

    llm-d helm charts and deployment examples

    Go Template 54 55

  2. llm-d-modelservice llm-d-modelservice Public

    helm charts for deploying models with llm-d

    Go Template 30 58

  3. llm-d-planner llm-d-planner Public

    Python 12 9

  4. llm-d-fast-model-actuation llm-d-fast-model-actuation Public

    Kubernetes controllers for fast model actuation using vLLM sleep/wake and launcher-based model swapping

    Go 10 14

  5. batch-gateway batch-gateway Public

    The batch gateway is an llm-d implementation of the OpenAI batch inference API

    Go 8 14

  6. py-inference-scheduler py-inference-scheduler Public

    Python based inference-scheduler for Reinforcement Learning

    Python 7 1

Repositories

Showing 10 of 14 repositories
  • batch-gateway Public

    The batch gateway is an llm-d implementation of the OpenAI batch inference API

    llm-d-incubation/batch-gateway’s past year of commit activity
    Go 8 Apache-2.0 14 22 5 Updated Apr 10, 2026
  • llm-d-fast-model-actuation Public

    Kubernetes controllers for fast model actuation using vLLM sleep/wake and launcher-based model swapping

    llm-d-incubation/llm-d-fast-model-actuation’s past year of commit activity
    Go 10 Apache-2.0 14 53 9 Updated Apr 10, 2026
  • llm-d-planner Public
    llm-d-incubation/llm-d-planner’s past year of commit activity
    Python 12 Apache-2.0 9 25 4 Updated Apr 10, 2026
  • weight-propagation-interface Public
    llm-d-incubation/weight-propagation-interface’s past year of commit activity
    Python 1 Apache-2.0 0 0 0 Updated Apr 10, 2026
  • py-inference-scheduler Public

    Python based inference-scheduler for Reinforcement Learning

    llm-d-incubation/py-inference-scheduler’s past year of commit activity
    Python 7 Apache-2.0 1 3 1 Updated Apr 9, 2026
  • llm-d-async Public

    Asynchronous Processor for Inference Gateway. Orchestrator of queues

    llm-d-incubation/llm-d-async’s past year of commit activity
    Go 6 Apache-2.0 8 15 3 Updated Apr 9, 2026
  • llm-d-skills Public
    llm-d-incubation/llm-d-skills’s past year of commit activity
    2 Apache-2.0 2 1 3 Updated Apr 2, 2026
  • llm-d-infra Public

    llm-d helm charts and deployment examples

    llm-d-incubation/llm-d-infra’s past year of commit activity
    Go Template 54 Apache-2.0 55 13 (1 issue needs help) 9 Updated Apr 2, 2026
  • llm-d-modelservice Public

    helm charts for deploying models with llm-d

    llm-d-incubation/llm-d-modelservice’s past year of commit activity
    Go Template 30 58 15 (1 issue needs help) 12 Updated Mar 28, 2026
  • hermes Public

    Hermes is a cluster configuration scanning and self-test generation tool for llm-d inference workloads

    llm-d-incubation/hermes’s past year of commit activity
    Rust 0 3 7 (1 issue needs help) 10 Updated Mar 19, 2026
View all repositories

Top languages

Loading…

Most used topics

Loading…