LLM operations tools that don't train on your data

Figure and bar: visits/mo, estimate · year: online since

1. #3O

   OpenRouter

   Routes API requests to 500 language models across 80 providers through a unified interface.

   224k
   Active
   From $5.50/mo
   2023
2. #7M

   Mistral

   Provides frontier language models, agents, and infrastructure for building custom AI systems.

   63.6k
   Active
   From $14.99/user/mo
   2023
3. #12T

   Together AI

   Runs open-source language models with serverless inference, batch processing, and GPU clusters.

   32.7k
   Active
   2022
4. #14G

   GreenPT

   Provides AI chat, APIs, and specialized tools running on European renewable energy infrastructure with transparent energy tracking.

   28k
   Active
   From ≈ $5.06/mo
   2025
5. #32O

   OpenLayer Agents

   Tests and monitors machine learning and LLM systems with configurable evaluations before release and tracing, cost and policy checks in production.

   4.7k
   Active
   Free plan
   2021
6. #38P

   Patronus

   Generates simulated digital environments and benchmarks for training and evaluating AI agents.

   1.7k
   Active
   Contact sales
7. #40L

   LLM Gateway Chat

   Routes requests across 40 LLM providers through a unified API with cost tracking and automatic failover.

   989
   Active
   From $5 per use
8. #43L

   LangWatch

   Tests and evaluates AI agents through simulation, tracing, and scoring before production deployment.

   452
   Active
   From ≈ $33/user/mo
9. #46O

   OminiGate

   Routes requests to text, image, and video models from multiple providers through a single API endpoint.

   298
   Active
   From $0.01 per use
   2025
10. #62C

    ConstellationMCP · sign-in

    Trains post-trained and fine-tuned models on a firm's own documents, such as credit agreements, and deploys them inside its network.

    47
    Active
11. #63E

    EmpirioLabs AIMCP · sign-in

    Hosts open-source models on its own GPUs and serves proprietary model endpoints through a chat and API interface.

    46
    Active
    From $19.90/mo
    2025
12. #72P

    Plurai

    Generates test scenarios and evaluates AI agents against policy violations and edge cases in production.

    29
    Active
    From $0.15 per use
13. #76A

    ApexApi

    Routes requests to 130 AI models through a single API key and OpenAI-compatible endpoint.

    24
    Active
    From $10 per use
14. #77P

    Paralleliq

    Identifies idle GPU capacity and tier mismatches in inference fleets to reduce cost per token without touching production.

    24
    Active
    Free plan
15. #78P

    Phoenix Grove Systems™

    Provides access to open-source language models with privacy-first architecture and persistent memory across conversations.

    22
    Active
16. #79C

    CoralBricks

    Serves open-source language models through an OpenAI-compatible API optimized for coding and research agents.

    22
    Active
17. #82T

    TrustedRouter

    Routes requests across AI model providers through one OpenAI-compatible API with verifiable privacy and no prompt logging.

    19
    Active
    From $5.50 per use
18. #84M

    memgram

    Stores and retrieves contextual memory for AI agents across conversations with inspection and governance.

    18
    From $29/mo
19. #85S

    Septor Labs

    Routes requests across language models, embeddings, and agents through a single OpenAI-compatible endpoint with caching and observability.

    17
    Active
    From $14/mo
    2025
20. #86C

    cortecs

    Routes requests to multiple large language models through a GDPR-compliant gateway with policy-based provider selection.

    16
    Active
    Pay per use
21. #90I

    Inferbase

    Routes requests across language models, enforces policies, and logs every decision.

    16
    Active
    Free plan
22. #91L

    LiberaGPT

    Runs large language models entirely offline on iPhone and Android devices without subscriptions or data collection.

    14
    Active
    Free
23. #94O

    OpenInfer

    Schedules and orchestrates AI inference workloads across heterogeneous hardware from cloud to edge devices.

    13
    Active
24. #102R

    RunInfra

    Serves open-source language models with prompt caching to reduce inference cost and latency.

    5
    Active
    From $0.05 per use
    2024
25. #105R

    Retrace

    Records and replays agent execution to fork failed steps, test fixes, and enforce runtime safety guardrails.

    4
    Active
    From $29/mo
26. #106G

    Gridlight

    Runs AI inference on your own hardware with model routing, cost governance, and data residency controls.

    3
    Active
    Contact sales
27. #111R

    Rainwater Labs

    Builds customizable foundation models designed for secure, air-gapped deployment with embedded alignment and compliance.

    3
    Active
28. #113B

    Bike4MindMCP · open

    Runs chat, agent jobs and interactive artifacts across hosted frontier models and self-hosted open-weight models from one workbench.

    3
    Active
    From $30/mo
29. #117A

    AgentPrizm

    Provides persistent, auditable memory layer for AI agents via API and MCP protocol.

    2
    Active
    From $19/mo
30. #126K

    KourierMCP · open

    Serves open-weight language models through an OpenAI-compatible API endpoint running on the company's own GPU clusters.

    2
    Active
    From $20/mo
31. #131F

    Fortaify

    Runs language models, image generation, and voice processing locally on your device without sending data to the cloud.

    1
    Active
    From $8.99/mo
32. #142R

    ReOpenly

    Trains compact open-weight models on your own examples, then hosts them behind an OpenAI-compatible API.

    1.5k linking domains
    From $0.03 per use
33. #150Y

    Your Own AI

    Runs large language models locally on your machine with optional cloud routing for frontier models.

    60 linking domains
    Active
    From $200/yr
34. #155N

    Nadir

    Routes LLM requests to the cheapest model tier that meets quality thresholds, reducing inference costs.

    668 ★
    Active
    From $25 per use
35. #157A

    ApiLink

    Routes requests to 369 AI models from 45 providers through a single OpenAI-compatible API endpoint.

    34 linking domains
    Active
36. #164D

    Deep Variance

    Audits GPU cluster capacity and usage against contracts, reporting shortfalls as dollar ranges.

    30 linking domains
    Active
37. #171Z

    zizka.ai

    Stores agent execution traces, replays sessions, and detects behavior drift in production.

    29 linking domains
    Active
38. #174P

    Prompt Smartly AI

    Optimizes and refines prompts for AI language models to improve output quality.

    22 linking domains
    From $5/mo
39. #176P

    Promptyx

    Organizes, versions, and evaluates prompts and workflows across multiple language models.

    26 linking domains
    Active
    From $29/mo
40. #177R

    RPCS-1 Agent Tuner

    Tunes language model behavior by adjusting response style parameters like tone, directness, and memory handling.

    25 linking domains
    Active
    Free plan
41. #181Z

    zenture

    Evaluates AI response quality, compares model outputs, and refines prompts in a unified workspace.

    20 linking domains
    Active
    From ≈ $8.99/mo
42. #182N

    NeoSmith AI

    Distills small language models from your LLM traffic, one specialized model per agent in agentic workflows.

    21 linking domains
    Active
    Free
43. #183P

    Pinstripes

    Runs quantized frontier models on dedicated infrastructure with transparent precision specifications.

    Emerging
    Active
    From $5/mo
44. #186S

    SentinelGateway

    Routes requests across multiple LLM providers with automatic failover, semantic caching, and PII redaction.

    Emerging
    Active
    From $29/mo
    2024
45. #197T

    Tensorant

    Fine-tunes and serves open-weight models on your cloud GPUs with dataset management and evaluation.

    Little sign of use
    Active
    From $99/mo