Top llm-serving tools
Sustained, decay-weighted momentum leaders tagged “llm-serving”.
- 193· newvllmA high-throughput and memory-efficient inference and serving engine for LLMsamdblackwell+1858.1
- 630· newllm-action本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）ai-dev-toolsllm+452.6
- 707· newrayRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.data-sciencedeep-learning+1851.6
- 713· newvllm-ascendCommunity maintained hardware plugin for vLLM on Ascendai-dev-toolsascend+851.6
- 1107· newrtp-llmRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.ai-dev-toolsgpt+646.2
- 1389· newBentoMLRun any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.ai-dev-toolsai-inference+2741.6
- 1392· newloraxMulti-LoRA inference server that scales to 1000s of fine-tuned LLMsai-dev-toolsfine-tuning+1041.6
- 1780· newsndr_core_engineSNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + Control Center GUI.rag-evalawq+1919.3
- 2211· newhelix♾️ Private Agent Fleet with Spec Coding. Each agent gets their own GPU-accelerated desktop. Run Claude, Codex, Gemini and open models on a full private AI Stack ♾️ai-agentsrag-eval+1819.2
- 2142· newsuperduperSuperduper: End-to-end framework for building custom AI applications and agents.ai-dev-toolsrag-eval+2019.2
- 2155· newaiciAICI: Prompts as (Wasm) Programsai-dev-toolsai+1219.2
The weekly drop
Get the weekly rising list
The top risers across agents, MCP servers and dev tools, once a week in your inbox. No spam, unsubscribe anytime.