LLM operations tools with an API

Figure and bar: visits/mo, estimate · year: online since

1. #3O

   OpenRouter

   Routes API requests to 500 language models across 80 providers through a unified interface.

   224k
   Active
   From $5.50/mo
   2023
2. #4A

   AI21

   Routes requests between open-source and frontier models to reduce token costs while maintaining quality.

   103k
   Active
   From $0.20 per use
   2017
3. #7M

   Mistral

   Provides frontier language models, agents, and infrastructure for building custom AI systems.

   63.6k
   Active
   From $14.99/user/mo
   2023
4. #9M

   Modal

   Runs inference, training, and batch processing workloads on GPUs with instant autoscaling and sub-second cold starts.

   52.1k
   Active
   From $0/mo
5. #10T

   Truefoundry

   Routes requests to language models and agents with observability, access controls, and deployment across cloud or on-premises infrastructure.

   46.2k
   Active
   From $25/user/mo
6. #11R

   Replicate

   Runs and fine-tunes machine learning models through an API with a single line of code.

   37k
   Active
   From $0.00 per use
7. #12T

   Together AI

   Runs open-source language models with serverless inference, batch processing, and GPU clusters.

   32.7k
   Active
   2022
8. #14G

   GreenPT

   Provides AI chat, APIs, and specialized tools running on European renewable energy infrastructure with transparent energy tracking.

   28k
   Active
   From ≈ $5.06/mo
   2025
9. #16W

   Weights & Biases

   Tracks machine learning experiments, manages models and datasets, and monitors AI applications in production.

   21.2k
   Active
   From $60/mo
   2017
10. #17G

    GMI Cloud

    Runs AI model inference and agents on serverless or dedicated GPU infrastructure with automatic scaling.

    19.7k
    Active
    From $2 per use
11. #19L

    LiteLLM

    Routes requests across 140+ LLM providers and 1800+ models through a single API gateway with cost optimization and access controls.

    15.2k
    Active
    Free plan
    2023
12. #22L

    Liquid AI

    Distributes small foundation models optimized to run on devices without sending data to servers.

    11.3k
    Active
    Free plan
    2023
13. #24F

    Fireworks AI

    Hosts open-weight language and image models for fast inference via API, with fine-tuning and dedicated GPU deployments.

    8.7k
    Active
    From $0.01 per use
14. #25L

    Langfuse

    Traces, evaluates, and monitors language model applications in production with observability and prompt management.

    7.5k
    Active
    From $29/mo
15. #28B

    Barndoor AI

    Routes AI model and agent requests through access controls, data-loss prevention and audit logging before they reach the model.

    6.5k
    Active
    Contact sales
16. #30B

    Baseten

    Deploys and serves AI models in production with optimized inference across cloud infrastructure.

    5.9k
    Active
    Free plan
    2019
17. #31P

    Portkey

    Routes requests to 1600 language models through a unified API with observability, guardrails, and prompt management.

    5.9k
    Active
    From $49/mo
18. #33S

    Supermemory MCP

    Stores and retrieves long-term memory for AI agents through an API, indexing conversations and files and injecting relevant context into prompts.

    4.6k
    Active
    From $19/mo
19. #34M

    Maxim AI

    Simulates, evaluates and monitors AI agents and LLM apps, with test datasets, automated and human evaluators, production tracing and an open-source gateway.

    4.2k
    Active
    Free plan
20. #35H

    Humanloop

    Humanloop has joined Anthropic and is sunsetting its platform.

    3.6k
    Maintained
    Free plan
21. #36H

    Helicone

    Routes, debugs, and analyzes LLM API calls with observability and monitoring.

    2.7k
    Active
    From $79/mo
22. #39A

    Agent Forge

    Provides decentralized GPU compute infrastructure and coordinates autonomous AI agents across a network.

    1.1k
    Active
    From $20/mo
23. #40L

    LLM Gateway Chat

    Routes requests across 40 LLM providers through a unified API with cost tracking and automatic failover.

    989
    Active
    From $5 per use
24. #41A

    Aleph Alpha

    Develops specialized sovereign large language models for business-critical applications with customization and optimization services.

    820
    Active
    2019
25. #42O

    OmniRoute

    Routes requests across 352 model providers through a single OpenAI-compatible endpoint, falling back automatically when a provider is unavailable or out of quota.

    641
    Active
26. #44C

    Cyfuture AI

    Provides GPU infrastructure, model inference, fine-tuning, and AI applications including chatbots, voice agents, and RAG systems.

    397
    Active
    From $0.60 per use
    2001
27. #45F

    FastRouter.ai

    Routes requests across 200+ LLMs through a single OpenAI-compatible API with intelligent failover and cost optimization.

    319
    Active
    From $199/mo
28. #46O

    OminiGate

    Routes requests to text, image, and video models from multiple providers through a single API endpoint.

    298
    Active
    From $0.01 per use
    2025
29. #47W

    wafer

    Optimizes language model inference serving for lower latency and higher throughput.

    224
    Active
30. #48M

    MegaLLM

    Routes requests across 70 language models, automatically selecting the cheapest and fastest provider.

    167
    Active
31. #49R

    Regolo.ai

    Serves OpenAI-compatible language models from EU data centers with zero data retention.

    162
    Active
    From ≈ $44/mo
32. #53E

    EvoX Genesis

    GPU-accelerated runtime for self-evolving agents, models, and workflows with evolutionary algorithms.

    132
    Active
33. #56A

    AI Stats

    Routes requests across AI model providers through a unified OpenAI-compatible API gateway.

    97
    Active
    From $1 per use
34. #57G

    GPU FinderMCP · open

    Compares on-demand and spot cloud GPU prices, egress fees and stock levels across 31 providers for hourly-refreshed searches.

    83
    Active
35. #63E

    EmpirioLabs AIMCP · sign-in

    Hosts open-source models on its own GPUs and serves proprietary model endpoints through a chat and API interface.

    46
    Active
    From $19.90/mo
    2025
36. #64F

    flatkey.ai

    Routes requests across 100+ AI models and 1000+ tools through a single API key and unified billing.

    38
    Active
    From $10/mo
37. #67A

    A4F

    Routes requests to hundreds of language models through a single API with automatic provider failover.

    35
    Active
    From $25/mo
38. #68R

    RedPill

    Routes AI inference requests through verified trusted execution environments with encrypted paths and cryptographic proofs.

    32
    Active
    From $0.01 per use
39. #69C

    CostPerPrompt

    Displays current pricing for 313 language models across vendors and calculates monthly API costs.

    32
    Active
    Free
40. #70S

    Sunbird AIMCP · sign-in

    Open-source conversational language model and speech API for African languages, plus satellite site-finding for solar mini-grids.

    32
    Active
    2019
41. #71N

    Nerra

    Builds a shared memory layer from your company tools and serves it to AI agents over MCP.

    31
    Active
    2024
42. #74P

    Plano

    Routes prompts and orchestrates agents with observability, guardrails, and model selection.

    26
    Active
    Open source
43. #76A

    ApexApi

    Routes requests to 130 AI models through a single API key and OpenAI-compatible endpoint.

    24
    Active
    From $10 per use
44. #78P

    Phoenix Grove Systems™

    Provides access to open-source language models with privacy-first architecture and persistent memory across conversations.

    22
    Active
45. #79C

    CoralBricks

    Serves open-source language models through an OpenAI-compatible API optimized for coding and research agents.

    22
    Active
46. #82T

    TrustedRouter

    Routes requests across AI model providers through one OpenAI-compatible API with verifiable privacy and no prompt logging.

    19
    Active
    From $5.50 per use
47. #83F

    FloTorch

    Routes requests across multiple LLMs and manages RAG pipelines with observability and caching for enterprise deployments.

    18
    Maintained
48. #84M

    memgram

    Stores and retrieves contextual memory for AI agents across conversations with inspection and governance.

    18
    From $29/mo