llm-serving rising this week
Tools tagged “llm-serving” gaining momentum, ranked by rate of change.
- 171· newvllmA high-throughput and memory-efficient inference and serving engine for LLMsamdblackwell+1858.1
- 576· newllm-action本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）ai-dev-toolsllm+452.9
- 651· newvllm-ascendCommunity maintained hardware plugin for vLLM on Ascendai-dev-toolsascend+851.9
- 645· newrayRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.data-sciencedeep-learning+1851.9
- 1017· newrtp-llmRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.ai-dev-toolsgpt+646.9
- 1333· newloraxMulti-LoRA inference server that scales to 1000s of fine-tuned LLMsai-dev-toolsfine-tuning+1042.7
- 1330· newBentoMLRun any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.ai-dev-toolsai-inference+2742.7
- 1775· newsndr_core_engineSNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + Control Center GUI.rag-evalawq+1920.0
- 2212· newhelix♾️ Private Agent Fleet with Spec Coding. Each agent gets their own GPU-accelerated desktop. Run Claude, Codex, Gemini and open models on a full private AI Stack ♾️ai-agentsrag-eval+1819.9
- 2143· newsuperduperSuperduper: End-to-end framework for building custom AI applications and agents.ai-dev-toolsrag-eval+2019.9
- 2156· newaiciAICI: Prompts as (Wasm) Programsai-dev-toolsai+1219.9
The weekly drop
Get the weekly rising list
The top risers across agents, MCP servers and dev tools, once a week in your inbox. No spam, unsubscribe anytime.