LLM operations tools that state SOC 2

Figure and bar: visits/mo, estimate · year: online since

1. #3O

   OpenRouter

   Routes API requests to 500 language models across 80 providers through a unified interface.

   224k
   Active
   From $5.50/mo
   2023
2. #15A

   Arize

   Observes agent behavior, evaluates performance with evals, and suggests improvements through automated debugging.

   27.2k
   Active
   From $50/mo
3. #16W

   Weights & Biases

   Tracks machine learning experiments, manages models and datasets, and monitors AI applications in production.

   21.2k
   Active
   From $60/mo
   2017
4. #25L

   Langfuse

   Traces, evaluates, and monitors language model applications in production with observability and prompt management.

   7.5k
   Active
   From $29/mo
5. #26U

   Uniphore

   Enterprise platform that prepares business data, fine-tunes domain-specific language models and runs agents that execute workflows across operations and customer service.

   6.8k
   Active
   2008
6. #27H

   HoneycombMCP · sign-in

   Stores and queries distributed traces, logs and metrics in real time to debug production services, including LLM and agent calls.

   6.5k
   Active
   From $150/mo
   2016
7. #28B

   Barndoor AI

   Routes AI model and agent requests through access controls, data-loss prevention and audit logging before they reach the model.

   6.5k
   Active
   Contact sales
8. #31P

   Portkey

   Routes requests to 1600 language models through a unified API with observability, guardrails, and prompt management.

   5.9k
   Active
   From $49/mo
9. #32O

   OpenLayer Agents

   Tests and monitors machine learning and LLM systems with configurable evaluations before release and tracing, cost and policy checks in production.

   4.7k
   Active
   Free plan
   2021
10. #40L

    LLM Gateway Chat

    Routes requests across 40 LLM providers through a unified API with cost tracking and automatic failover.

    989
    Active
    From $5 per use
11. #43L

    LangWatch

    Tests and evaluates AI agents through simulation, tracing, and scoring before production deployment.

    452
    Active
    From ≈ $33/user/mo
12. #56A

    AI Stats

    Routes requests across AI model providers through a unified OpenAI-compatible API gateway.

    97
    Active
    From $1 per use
13. #59S

    ScorecardMCP · sign-in

    Tests and evaluates AI agents against thousands of realistic scenarios to provide rapid feedback on performance.

    76
    Active
    From $299/mo
    2024
14. #62C

    ConstellationMCP · sign-in

    Trains post-trained and fine-tuned models on a firm's own documents, such as credit agreements, and deploys them inside its network.

    47
    Active
15. #64F

    flatkey.ai

    Routes requests across 100+ AI models and 1000+ tools through a single API key and unified billing.

    38
    Active
    From $10/mo
16. #68R

    RedPill

    Routes AI inference requests through verified trusted execution environments with encrypted paths and cryptographic proofs.

    32
    Active
    From $0.01 per use
17. #72P

    Plurai

    Generates test scenarios and evaluates AI agents against policy violations and edge cases in production.

    29
    Active
    From $0.15 per use
18. #79C

    CoralBricks

    Serves open-source language models through an OpenAI-compatible API optimized for coding and research agents.

    22
    Active
19. #94O

    OpenInfer

    Schedules and orchestrates AI inference workloads across heterogeneous hardware from cloud to edge devices.

    13
    Active
20. #97Z

    ZOLIX Lite

    Analyzes cloud and AI infrastructure spending to detect waste and recommend cost optimizations across AWS, Azure, GCP, and OCI.

    9
    Active
    Free plan
    2024
21. #102R

    RunInfra

    Serves open-source language models with prompt caching to reduce inference cost and latency.

    5
    Active
    From $0.05 per use
    2024
22. #106G

    Gridlight

    Runs AI inference on your own hardware with model routing, cost governance, and data residency controls.

    3
    Active
    Contact sales
23. #113B

    Bike4MindMCP · open

    Runs chat, agent jobs and interactive artifacts across hosted frontier models and self-hosted open-weight models from one workbench.

    3
    Active
    From $30/mo
24. #117A

    AgentPrizm

    Provides persistent, auditable memory layer for AI agents via API and MCP protocol.

    2
    Active
    From $19/mo
25. #118株

    株式会社イネブラ

    Routes requests to multiple LLM providers through a unified API gateway with cost optimization and model switching.

    2
    Active
    From ≈ $6.21/mo
26. #132N

    Narev

    Aggregates and compares pricing for language models across multiple providers via API.

    1
    Active
    Free plan
27. #136A

    ASE(Auditome Sovereign Engine)

    Evaluates AI actions against policy before execution and issues cryptographically signed decision receipts for audit trails.

    1
    Maintained
    From $495 once
28. #155N

    Nadir

    Routes LLM requests to the cheapest model tier that meets quality thresholds, reducing inference costs.

    668 ★
    Active
    From $25 per use
29. #186S

    SentinelGateway

    Routes requests across multiple LLM providers with automatic failover, semantic caching, and PII redaction.

    Emerging
    Active
    From $29/mo
    2024
30. #189F

    ForgeOpus

    Provides no-code tools for fine-tuning language models and building neural networks visually.

    Emerging
    Maintained
31. #193A

    Aurora Gateway

    Routes requests to any LLM provider with caching, failover, and usage tracking.

    Emerging
    Active
    Open source
    2026
32. D

    DecionisMCP · sign-in

    Evaluates AI agent and automated system actions against policy, returning allow, escalate or block with signed decision records.

    27
    Active
    From $49/mo
    2026