Glonce

litelm reimplements LiteLLM's core in about 2,900 lines

toolsThe Glonce editorsHacker News🔥59Glonce rating59Impact50Novelty40Relevance85Surprise30Players20Credibility78
litelm reimplements LiteLLM's core in about 2,900 lines

litelm is a new Python library, released as a lighter alternative to LiteLLM, a package that routes calls to large language model providers and translates between their message formats. According to litelm's own documentation, LiteLLM's actual routing and translation logic is a small part of its total codebase: more than 100,000 lines are proxy servers, a caching layer, cost tracking and other features that most users never touch. litelm reimplements just that core call path, model routing, message translation, streaming, tool use and embeddings, in about 2,900 lines of code with two runtime dependencies, the openai and httpx packages. It deliberately leaves out LiteLLM's Router class, its proxy server and its caching layer.

litelm's API is built to mirror LiteLLM's: the same function names, arguments and response types, so the documentation describes migrating an existing LiteLLM codebase as a one-letter change to the import, from the seven-letter litellm to the six-letter litelm. It routes to 19 providers using a provider/model-name string, such as "openai/gpt-4o" or "groq/llama-3.1-70b-versatile", and any OpenAI-compatible endpoint can be used through an api_base parameter, which the documentation shows working against self-hosted servers such as vLLM, Ollama and LM Studio. Every call has an async counterpart, acompletion, aembedding, aresponses and atext_completion; streaming and tool or function calling are supported; and errors from any provider are mapped into a shared exception hierarchy, including ContextWindowExceededError, RateLimitError and AuthenticationError, so calling code can catch one set of exceptions regardless of which provider raised them. Provider SDKs beyond the two default dependencies, such as the anthropic SDK or boto3 for Bedrock, are installed as optional pip extras rather than bundled by default.

To back its compatibility claim, the maintainer published an attestation dated 2026-09-11 describing an audit of LiteLLM's own routing and formatting changes across the commit range 649eb2d through 9a715df2: 360 core-path commits were triaged, upstream tests were inspected for potentially relevant behavior, and the resulting gaps were fixed test-first. The attestation reports that litelm's own local test suite passed 262 tests and skipped 55, that all 45 available live-provider tests passed, and that all 10 DSPy smoke tests passed, all against the current dependency lock. Separately, the project states that, measured against a scoped LiteLLM baseline at commit 9a715df2, litelm passes 75 ported tests with no remaining actionable assertion or runtime failures, and that its DSPy drop-in support has all 7 execution paths, Predict, CoT, typed signatures, streaming, embeddings, tool use and multi-output, proven to work live. The maintainer states explicitly that this attests only litelm's declared routing, formatting and DSPy surface, not full compatibility with LiteLLM as a whole.

litelm describes itself as human-directed, AI-assisted software. Much of the code was written using Claude Code with Claude Opus 4.6 and 4.7, and code written from 2026-05-14 onward has instead been produced through a tool called Pi using GPT-5.5; the documentation states that its compatibility claims rest on the test suites and on maintainer review, not on how the code was written. litelm is labeled alpha software; no license or version number beyond that label is given in the material available, and no performance or latency comparison against LiteLLM is provided, only the differences in code size and dependency count.

Key facts

  • litelm reimplements LiteLLM's core routing and message-translation logic in about 2,900 lines of code with two dependencies (openai, httpx), versus more than 100,000 lines in LiteLLM's own proxy, caching and other features.
  • Its API mirrors LiteLLM's exactly, so the documentation describes switching as a one-letter change to the import, from the seven-letter litellm to the six-letter litelm.
  • It routes to 19 providers via a provider/model-name syntax and reaches any OpenAI-compatible endpoint, including self-hosted vLLM, Ollama and LM Studio servers, through an api_base parameter.
  • A 2026-09-11 maintainer attestation reports 262 passing local tests (55 skipped), 45 passing live-provider tests and 10 passing DSPy smoke tests, after triaging 360 of LiteLLM's core-path commits for compatibility gaps.
  • The project discloses its code was written first with Claude Code using Claude Opus 4.6 and 4.7 and, from 2026-05-14, through a tool called Pi using GPT-5.5; it is labeled alpha, with no license or version number stated.

Why it matters

LiteLLM is a package that Python projects use to call multiple large language model providers through one interface, but according to litelm's own documentation, the actual routing and message-translation logic is a small part of LiteLLM's total size: more than 100,000 lines are a proxy server, a caching layer, cost tracking and other features that most users never touch. litelm's premise is that isolating just that core call path, in about 2,900 lines of code with two dependencies, gives developers who only need routing and translation a far smaller piece of software to install, read and depend on than the full LiteLLM package.

Who it affects

This targets Python developers and teams who already call multiple LLM providers, such as OpenAI, Anthropic, Bedrock and Groq, through LiteLLM's interface, particularly if they use only its routing, translation, streaming, tool-calling and embeddings functions rather than its proxy server or caching layer. It also targets users of self-hosted inference servers such as vLLM, Ollama and LM Studio, which litelm reaches through the same OpenAI-compatible api_base parameter used for hosted providers, and users of the DSPy framework, for which litelm claims verified drop-in support across all 7 of its execution paths: Predict, CoT, typed signatures, streaming, embeddings, tool use and multi-output.

How to use it

litelm installs with pip install litelm, which pulls in the core package plus its two dependencies, openai and httpx; pip install litelm[anthropic], litelm[bedrock] or litelm[all] add the corresponding provider SDKs. Calls take a provider/model-name string, for example litelm.completion("openai/gpt-4o", messages=[...]) or, with stream=True, litelm.completion("groq/llama-3.1-70b-versatile", messages=[...]), with provider API keys read from environment variables such as OPENAI_API_KEY and ANTHROPIC_API_KEY, or passed directly as an api_key argument. An api_base argument points the same call at any OpenAI-compatible endpoint, including self-hosted vLLM, Ollama or LM Studio servers. Every function has an async counterpart, acompletion, aembedding, aresponses and atext_completion; tool and function calling take the same tools and tool_choice arguments as the OpenAI API; and errors from any provider are mapped to a shared set of exceptions, ContextWindowExceededError, RateLimitError and AuthenticationError. Because the API mirrors LiteLLM's function names, arguments and response types, the documentation describes migrating an existing LiteLLM codebase as a one-letter change to the import, from the seven-letter litellm to the six-letter litelm.

How solid is it

The maintainer published an attestation dated 2026-09-11 stating that LiteLLM's own routing and formatting changes were reviewed across the commit range 649eb2d through 9a715df2, that the audit triaged 360 core-path commits, inspected upstream tests for potentially relevant behavior, and fixed the resulting compatibility gaps test-first. Under that attestation, litelm's local scoped test suite passed 262 tests and skipped 55, all 45 available-provider live tests passed, and all 10 DSPy smoke tests passed, all run against the current dependency lock. The project separately states that, against a scoped LiteLLM baseline at commit 9a715df2, litelm passes 75 ported tests with no remaining actionable assertion or runtime failures, and that all 7 of DSPy's execution paths, Predict, CoT, typed signatures, streaming, embeddings, tool use and multi-output, have been proven to work live. The maintainer is explicit that the attestation covers only litelm's declared routing, formatting and DSPy surface, not full compatibility with LiteLLM, and the project labels itself alpha software. It also discloses that most of its code was written using Claude Code with Claude Opus 4.6 and 4.7, and that code written from 2026-05-14 onward has instead been produced through a tool called Pi using GPT-5.5, stating that its compatibility claims rest on the test suites and maintainer review rather than on how the code was written.

Risks and caveats

litelm is labeled alpha software with no version number given beyond that label, and no license is stated in the material reviewed. The maintainer's compatibility attestation covers only litelm's declared routing, formatting and DSPy surface, explicitly not full compatibility with LiteLLM, and describes a one-way audit of LiteLLM's own commits and tests rather than any statement of adoption, production use or endorsement by LiteLLM's own project or maintainers. The source gives no performance or latency comparison between the two libraries, only differences in code size and dependency count, and it does not say whether the 49 fast upstream contract tests run by one script are the same tests as, or different from, the 75 ported tests reported as passing, nor what LiteLLM's own full test-suite total is, so the material available does not indicate what share of LiteLLM's behavior is actually exercised. litelm also drops LiteLLM's Router class, proxy server and caching layer by design, so teams that depend on those features would need to keep using LiteLLM itself or build replacements of their own.

“This attests litelm's declared routing/formatting/DSPy surface only, not full litellm compatibility.”

— the maintainer's 2026-09-11 attestation