Fiddler AI

Fiddler takes in traces from agent frameworks, OpenTelemetry, S3 and AI gateways (Kong, AgentGateway, LiteLLM), reads model data from Snowflake, S3, SageMaker, Databricks and MLflow, and sends alerts to Datadog and PagerDuty. These move telemetry in and alerts out, and no connectors that let an agent take authenticated actions in outside systems are documented.

SourceFiddler, docs.fiddler.ai llms.txtread 2026-09-21

No workflows that sequence, branch or retry an agent's steps are documented; Fiddler observes, evaluates and guards agents that run elsewhere.

SourceFiddler, fiddler.ai llms.txtread 2026-09-21

No retrieval structure over the customer's knowledge that grounds an agent's answers is documented; Fiddler evaluates RAG applications built elsewhere, including faithfulness and hallucination checks.

SourceFiddler, docs.fiddler.ai llms.txtread 2026-09-21

Fiddler Guardrails enforce policy inline on an agent's request and response path in under 100 milliseconds, detecting hallucinations, toxicity, PII and PHI leakage, prompt injection and jailbreaks, and for coding agents redact PII, PHI and secrets inline through the customer's existing LLM gateway; enforceable policy is one of the platform's five pillars.

SourceFiddler, fiddler.ai llms.txt and homepage, and docs.fiddler.ai llms.txt (AI gateway guardrail enforcement)read 2026-09-21

Fiddler's SOC 2 Type II report covers security, confidentiality and availability, is audited annually and is available under NDA, and it states HIPAA compliance; customers authenticate through SSO providers and grant granular authorizations with role-based access control, data is encrypted with AES-256 at rest and TLS 1.2 or higher in transit, and customer data is kept and deleted under a retention policy.

SourceFiddler, fiddler.ai/securityread 2026-09-21

Visibility is hierarchical, from application to session, agent, trace and span, with native SDKs for LangGraph, LangChain, Google ADK and Strands, OpenTelemetry ingestion, coding-agent tracing of LLM calls and tool use through Claude Code and existing gateways, alerts routed to Datadog and PagerDuty, and customer data kept under a stated retention policy.

SourceFiddler, fiddler.ai llms.txt, docs.fiddler.ai llms.txt and fiddler.ai/securityread 2026-09-21

No session, conversation or long-term memory that an agent reads back as context is documented; Fiddler records sessions and traces for observability.

SourceFiddler, fiddler.ai llms.txtread 2026-09-21

Deployment runs as SaaS, in the customer's VPC, on premises, in AWS GovCloud or air-gapped, and as a Partner AI App inside the customer's Amazon SageMaker, with Fiddler's Centor evaluation models running entirely in the customer's environment.

SourceFiddler, fiddler.ai homepage and llms.txt, and docs.fiddler.ai llms.txtread 2026-09-21

No ready-made agents, workflows or templates that a buyer adopts and runs are documented; Fiddler ships pre-built evaluators, a metrics library and use-case cookbooks.

SourceFiddler, docs.fiddler.ai llms.txt and fiddler.ai llms.txtread 2026-09-21

Guardrails evaluate every agent request and response inline as it passes, continuous evaluations score production traces, and alert rules fire on monitored metrics, so Fiddler's evaluation and enforcement start from each event with no person initiating them.

SourceFiddler, fiddler.ai llms.txt and homepage, and docs.fiddler.ai llms.txtread 2026-09-21

Custom prompts and a customer choice of judge model drive Fiddler's LLM-as-a-judge evaluators, beside its own Centor Models that run in the customer's environment, so the customer chooses the model behind Fiddler's own evaluation features. Which judge models are supported is not published.

SourceFiddler, docs.fiddler.ai llms.txt (Advanced Prompt Specs) and fiddler.ai llms.txtread 2026-09-21

Fiddler documents native SDKs (its OpenTelemetry SDK plus LangGraph, LangChain, Google ADK, Strands and Evals SDKs), a Python client for alerts and other resources, and a Fiddler MCP server that lets MCP-compatible assistants query applications, traces and evaluator results.

SourceFiddler, docs.fiddler.ai llms.txtread 2026-09-21

The Fiddler Evals SDK runs experiments on LLM applications with pre-built evaluators and custom metrics, datasets with golden labels and side-by-side comparison, custom LLM-as-a-judge evaluators, and continuous evaluation in production over more than 100 metrics, scored by Fiddler's Centor Models in the customer's environment.

SourceFiddler, docs.fiddler.ai llms.txt and fiddler.ai llms.txtread 2026-09-21

No browser, desktop or computer control by an agent is documented; Fiddler observes and guards agents.

SourceFiddler, fiddler.ai llms.txtread 2026-09-21