Top llm-inference tools
Sustained, decay-weighted momentum leaders tagged “llm-inference”.
- 425· newgitleaksFind secrets with Gitleaks 🔑ai-dev-toolsai-powered+1955.1
- 465· newlemonadeLemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zkmcp-serversai+1854.7
- 630· newllm-action本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）ai-dev-toolsllm+452.6
- 673· newKVarNKVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.ai-agentsagentic-ai+651.9
- 712· newplanoPlano is an AI-native proxy and data plane for agentic apps — with built-in orchestration, safety, observability, and smart LLM routing so you stay focused on your agents core logic. ai-dev-toolsai-gateway+1651.6
- 707· newrayRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.data-sciencedeep-learning+1851.6
- 757· newopenevolveOpen-source implementation of AlphaEvolveai-dev-toolsalpha-evolve+1850.8
- 1060· newnanocoderAn open coding agent for your terminal, built by a community collective rather than a company. Bring your own model, keep your code on your machine, and owe nothing to anyone.ai-agentsai-dev-tools+846.2
- 1070· newOpenAlpha_EvolveOpenAlpha_Evolve is an open-source Python framework inspired by the groundbreaking research on autonomous coding agents like DeepMind's AlphaEvolve.ai-dev-toolsalphacode+1646.2
- 1122· newopenvinoOpenVINO™ is an open source toolkit for optimizing and deploying AI inferenceaicomputer-vision+1746.2
- 1244· newAtomic-ChatLocal AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer.mcp-serversai-chat+1941.7
- 1389· newBentoMLRun any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.ai-dev-toolsai-inference+2741.6
- 1392· newloraxMulti-LoRA inference server that scales to 1000s of fine-tuned LLMsai-dev-toolsfine-tuning+1041.6
- 1469· newtaOSSelf-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).ai-agentsagent-framework+1740.5
- 1776· newleanctxDrop-in prompt compression for production LLM apps. Cut your token bill 40-60% without changing your code. Python SDK, LLMLingua-2, MIT.rag-evalanthropic+1119.3
- 1780· newsndr_core_engineSNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + Control Center GUI.rag-evalawq+1919.3
- 2039· newblastOpen-source VMs-as-a-serviceai-agentsbrowser-automation+219.2
- 2272· newekoEko (Eko Keeps Operating) - Build Production-ready Agentic Workflow with Natural Language - eko.fellou.aiai-agentsai-dev-tools+2019.2
- 2142· newsuperduperSuperduper: End-to-end framework for building custom AI applications and agents.ai-dev-toolsrag-eval+2019.2
- 2231· newmonocleMonocle is a framework for tracing GenAI app code. This repo contains implementation of Monocle for GenAI apps written in Python. ai-agentsagentic-ai+1219.2
- 2219· newedslDesign, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.ai-agentsanthropic+1719.2
- 2170· newllmflowsLLMFlows - Simple, Explicit and Transparent LLM Appsai-dev-toolsrag-eval+1319.2
- 2155· newaiciAICI: Prompts as (Wasm) Programsai-dev-toolsai+1219.2
- 2218· newLLM-VMirresponsible innovation. Try now at https://chat.dev/ai-agentsartificial-intelligence+919.2
The weekly drop
Get the weekly rising list
The top risers across agents, MCP servers and dev tools, once a week in your inbox. No spam, unsubscribe anytime.