LiteLLM is an open-source LLM Gateway with 28K+ stars on GitHub and trusted by companies like NASA, Rocket Money, Samsara, Lemonade, and Adobe. We’re rapidly expanding and seeking a founding full-stack engineer to help scale the platform. We’re based in San Francisco.
LiteLLM provides an open-source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format. We just hit $2.5M ARR and have raised a $1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund. You can find more information on our website, Github and Technical Documentation.
Companies use LiteLLM Enterprise once they put LiteLLM into production and need enterprise features like Prometheus metrics (production monitoring) and need to give LLM access to a large number of people with SSO (secure sign on) or JWT (JSON Web Tokens).
As a Founding Backend Engineer, you'll be responsible for ensuring LiteLLM unifies the format for calling LLM APIs in the OpenAI spec. This involves writing transformations to convert API requests from OpenAI spec to various LLM provider formats. You'll work directly with the CEO and CTO on critical projects including:
- Migrating key systems from httpx to aiohttp for 10x higher throughput
- Adding support for Anthropic and Bedrock
- Anthropic 'thinking' parameter
- Handling provider-specific quirks like OpenAI o-1 streaming limitations
- Scaling aggregate spend computation for 1M+ logs
- Implementing cost tracking and logging for Anthropic API
The tech stack includes Python, FastAPI, JS/TS, Redis, Postgres, S3, GCS Storage, Datadog, and Slack API.
- 1-2 years of backend/full-stack experience with production systems
- Passion for open source and user engagement
- Experience scaling high-performance infrastructure
- Strong work ethic and ability to thrive in small teams
- Eagerness to shape growing infrastructure