Back to vendors
B

Braintrust

Visit site
Entry priceStarter is free with 1 GB of data and 10K scores. Pro is $249 a month.Full pricing detail

AI observability and evaluation platform: traces, datasets, experiments, online scoring and human review, with hosted prompts and tools, a model gateway, the Loop agent and an MCP server.

Braintrust is an AI observability and evaluation platform for instrumenting, understanding and improving agents. It connects the pieces of an AI development loop: production traces, human feedback, test datasets, experiments, quality scores, prompt iteration, model routing and deployment.

Traces record each request's nested steps, including model calls, tool calls, retrieved context, errors, cost and latency, and teams filter, examine and debug them, with dashboards, alerts to Slack or webhooks, and Topics, which classifies production logs by task, sentiment and issue. Evals run an AI system over versioned datasets and score it with LLM-as-a-judge, autoevals, custom code or human reviewers; experiments are compared side by side, evals run in CI, and online scoring applies the same scorers to live traffic so failures become new test cases.

Braintrust also hosts functions: prompts, tools that deployed prompts can call, scorers and workflows that chain prompts, versioned and tagged by environment, plus a Gateway that routes model requests to providers on the customer's own keys. Loop, its built-in agent, investigates project data, writes scorers and test cases, iterates on prompts and runs on a schedule, pausing for approval before it changes anything in an interactive thread. SDKs cover Python, TypeScript, Go, Java, Ruby and C#, alongside a REST API, the bt CLI and an MCP server for coding agents.

Braintrust lists SOC 2 Type II compliance on every plan, supports SSO and SAML, OIDC, role-based access with custom groups and audit logs, and signs BAAs on Enterprise. It runs as SaaS, as BYOC in the customer's cloud operated by Braintrust, or self-hosted on Enterprise. Starter is free with unlimited users, Pro is $249 a month, both with usage-based overage, and Enterprise is custom.

Vendor details

Canonical URL

https://www.braintrust.dev

Category

Agent infrastructure

Company status

independent

Use cases & customers

Target customers

AI developersproduct teams

Deployment options

SaaSself-hosted

In practice

Your AI feature works in testing but quietly regresses in production and you find out from users. Braintrust scores live traffic with LLM-based judges, surfacing regressions as they happen and turning real failures into new test cases.

A prompt change looks better on one example but you can't tell if it's better overall. Braintrust runs your prompts and models against a dataset, scores them side by side, and captures an immutable experiment you can compare over time.

A bad change keeps slipping into your agent before anyone catches it. Braintrust runs your evals on every pull request and blocks the merge when a change degrades quality, gating releases on measured output quality.

Agentic Index coverage score

10.0 / 14 capabilities · 71%

Integrations & Tool Calling Full

Tools are hosted as functions, general purpose code that deployed prompts can call to perform operations or reach external data. They run serverless in an isolated environment, and any function can be used as a tool. Prompts can also connect to public MCP servers, after OAuth authentication, to reach external tools and data.

SourceBraintrust, braintrust.dev/docs deploy/functions and evaluate/write-prompts (MCP servers)read 2026-09-21

Workflow Orchestration Partial

Workflows are hosted functions that chain two or more prompts and pass outputs between them. Prompts, tools and scorers also compose into applications. Functions are versioned and can be tagged for production, staging or development environments, so each sequence can be versioned and traced. Workflows do not include branching, loops, retries or fallback paths, or deterministic steps mixed with agent steps.

SourceBraintrust, braintrust.dev/docs deploy/functions, deploy/prompts and deploy/environmentsread 2026-09-21

Knowledge Grounding & RAG Not documented

Datasets hold test examples, and traces record the context a customer's retrieval returned. Braintrust's example RAG agent is built from a vector search tool the customer writes and a prompt. Braintrust does not ingest documents or keep an index or retrieval layer over company data.

SourceBraintrust, braintrust.dev/docs deploy/functions and annotate/datasetsread 2026-09-21

Human Oversight & Guardrails Full

Loop, Braintrust's agent, does read only work without interrupting. In an interactive thread it pauses for approval before it creates or edits anything. A scheduled Loop automation has nobody to ask, so each one carries write tool permissions that set what it may change, from nothing (read and report only) to named object types. Human review scores and review queues let people rate the customer's own traces.

SourceBraintrust, braintrust.dev/docs loop, loop/automations and annotate/human-reviewread 2026-09-21

Security, Identity & Governance Full

Sign in supports SSO and SAML with Okta Workforce, Microsoft Entra ID and Google Workspace, and OIDC for custom providers. Role based access control uses built in and custom permission groups set at organization and project level.

API keys are stored as one way hashes and scoped to projects, provider secrets use AES-256 encryption, and all data is encrypted at rest and in transit. Retention policies are configurable, and an audit log records administrative actions such as permission changes and API key creation. Every plan includes SOC 2 Type II compliance, Enterprise adds a BAA for HIPAA, and a trust center sits at trust.braintrust.dev.

SourceBraintrust, braintrust.dev/docs security and admin/audit-logs, and braintrust.dev/pricingread 2026-09-21

Observability & Auditability Full

Traces record each AI request's nested steps, including model calls, tool calls, retrieved context, errors, cost and latency, and teams examine, filter and debug them in the trace view. Logs can be exported, with automatic export to the customer's own S3 bucket on Enterprise. Retention is 14 days on Starter, 30 days on Pro with paid extension, and custom on Enterprise. Administrative actions go to a separate audit log.

SourceBraintrust, braintrust.dev/llms.txt, braintrust.dev/docs observe, admin/data-management/export and admin/audit-logs, and braintrust.dev/pricingread 2026-09-21

Memory & State Persistence Not documented

Loop's threads persist so a person can return to an investigation, and traces, datasets and experiments are kept as records. Agents have no session, workflow or long term memory that they read and write across runs, and there are no controls to review or scope such memory.

SourceBraintrust, braintrust.dev/docs loop and deploy/functionsread 2026-09-21

Deployment & Data Residency Full

Customers can run Braintrust three ways. As SaaS, Braintrust operates both the control plane and the data plane. As BYOC, Braintrust operates the data plane inside the customer's cloud account. Self hosted on Enterprise, the customer deploys and controls the infrastructure that stores its AI data, with the data plane in its own VPC, while Braintrust provides the managed UI and platform. BYOC and self hosting keep logs, traces, datasets and prompts in the customer's account and region.

SourceBraintrust, braintrust.dev/docs admin/self-hosting, admin/deployment/byoc and securityread 2026-09-21

Prebuilt Agents, Templates & Packs Partial

Braintrust ships one ready made agent, Loop, which investigates project data, bootstraps scorers, generates test cases, iterates on prompts and builds charts. The same agent powers Patterns, a preconfigured automation that finds recurring failures, and the trace Debugger. There is no set of ready made agents or templates to choose from.

SourceBraintrust, braintrust.dev/docs loop, loop/automations and observe/patterns, and braintrust.dev/pricingread 2026-09-21

Triggers & Channel Coverage Full

Loop automations run Braintrust's agent on a daily, weekly, interval or cron schedule, with an instruction, a model and set write permissions, and send results to Slack or a webhook. Online scoring runs scorers automatically on incoming production logs. Alerts fire on log, time window and environment conditions.

SourceBraintrust, braintrust.dev/docs loop/automations, evaluate/score-online and observe/alertsread 2026-09-21

Model Flexibility & Routing Full

The Braintrust Gateway gives one API for routing model requests to AI providers. It supports the customer's own provider keys, configured at organization level, workload identity federation for supported providers, custom providers, and Braintrust's built in models billed against monthly credits. Deployed prompts carry their own model configuration, and Loop runs on the model and provider the customer chooses.

SourceBraintrust, braintrust.dev/llms.txt and braintrust.dev/docs deploy/gateway, admin/ai-providers and loopread 2026-09-21

APIs, SDKs & MCP Extensibility Full

Developers get a REST API with an OpenAPI description, SDKs in Python, TypeScript, Go, Java, Ruby and C#, and the bt CLI for running evals, querying logs, syncing data and managing functions. An MCP server, authenticated with OAuth 2.0 and PKCE, lets coding agents reason over project data. Custom tools, scorers and workflows are pushed from code as versioned functions, and evals run in CI.

SourceBraintrust, braintrust.dev/llms.txt and braintrust.dev/docs security, deploy/functions and evaluate/run-in-ciread 2026-09-21

Testing, Debugging & Optimization Full

Evals run an AI system over versioned datasets and score outputs with LLM as a judge, autoevals, custom code or human review scorers. Experiments are compared side by side, and evals run in CI, where configurable checks gate quality before production. Online scoring applies the same scorers to production logs, so quality is scored over time and failures become new dataset examples.

SourceBraintrust, braintrust.dev/docs evaluate, run-in-ci, score-online and annotate/datasetsread 2026-09-21

Browser & Computer Use Not documented

Custom code functions run in isolated serverless environments. No agent controls a browser, a desktop or a computer.

SourceBraintrust, braintrust.dev/docs security and llms-full.txtread 2026-09-21

The Agentic Index coverage score grades every vendor Full, Partial or Not documented against the same 14 buyer facing capabilities, from public evidence only. Each capability links to how all vendors in the index score on it. How this evidence is graded

Product updates

·Observability / auditabilityVerified

Coding agent insights, now in public preview for Braintrust SaaS and BYOC customers and for self hosted customers on data plane 2.13, traces a team's coding agent sessions from Claude Code, Codex, OpenCode, Pi, Antigravity or Grok. A dashboard breaks cost and tokens down by developer, model, tool, branch and API key, and a daily Loop automation surfaces recurring problems across sessions as findings in Patterns.

Bears on: Pricing / packaging

View source
·Observability / auditabilityVerified

Braintrust announced Nitro, a new asynchronous query engine for Brainstore that separates storage reads from compute work to accelerate trace investigations. Nitro is automatically enabled for SaaS and BYOC customers, with support starting at dataplane 2.15 for deployments customers host themselves. Braintrust reports that full text searches ran more than twice as fast in its production comparison.

Bears on: Observability / auditability

View source
·IntegrationsVerified

Braintrust introduced support for tracing Cloudflare Agents. Users can ingest traces by exporting them via OpenTelemetry or by instrumenting their agents directly in JavaScript on the Cloudflare Workers runtime.

Bears on: Integrations

View source
View all 5 Braintrust updates →Tracked since Jun 2026 · Verified from public vendor sources

Pricing

Starter is free with 1 GB of data and 10K scores. Pro is $249 a month.

Usage. A flat monthly platform fee per plan covers set amounts of processed data and scores, use beyond them is metered, and there are no per seat fees.

Free tier

Included quota

Starter includes 1 GB processed data and 10K scores a month, 14 day retention and unlimited seats. Pro includes 5 GB and 50K scores, 30 day retention and Google SSO. Neither plan has per seat fees.

What is public

Braintrust publishes full self serve pricing. Starter is free with 1 GB processed data and 10K scores a month, Pro is $249 a month with 5 GB and 50K scores plus published overage rates, and Enterprise is custom, with BYOC and self hosted deployment. Qualifying early stage startups can get 6 to 12 months of free Pro.

Billing mechanics

Each plan carries a flat monthly platform fee, plus metered on demand usage beyond the included processed data and scores. Pro overage rates are lower than Starter's. Topics and built in model usage draw from a separate monthly credit with uniform overage rates.

Cost watchouts

Usage past the included amounts is billed with no stated cap, at $4/GB and $2.50 per 1K scores on Starter, $3/GB and $1.50 per 1K scores on Pro, token rates for model use past the monthly credits, and $0.50 per GB a month for retention beyond 30 days on Pro. SAML SSO, custom roles, audit logs, the BAA and self hosting are Enterprise only.

Variable cost rationale

Braintrust meters processed data, scores and model use. The platform fee is flat, and past the included amounts each is billed on demand, at $4/GB and $2.50 per 1K scores on Starter, $3/GB and $1.50 per 1K scores on Pro, and token rates for model use past the monthly credits. Cost follows trace volume rather than seats, since users are unlimited. No cap on overage is published, so the included monthly allowances and credits are the only limit. Retention past 30 days on Pro adds $0.50 per GB a month.

Additional watchouts

Overage adds up quickly. A Pro team logging 10 GB and 100K scores a month pays about $339, not $249.

Sales call required

Mixed (some tiers require a call)

Free / trial

Free Starter plan with no card, 1 GB processed data, 10K scores and $10 model credits a month. Qualifying startups get 6 to 12 months of Pro free

Lowest paid plan

Pro at $249 per month with 5 GB processed data and 50K scores

Commercial notes

Unlimited users on every plan, so cost follows processed data, scores and model use rather than seats. Qualifying startups can get 6 to 12 months of Pro free.

Key ambiguities

Enterprise pricing is not published, and model use past the included credits is billed at token rates listed on a separate detailed pricing page.

Support SLA / resale

Starter has community support and Pro priority support. Enterprise adds dedicated support with guaranteed SLAs and an uptime SLA.

Missing data

Enterprise pricing is not published.

Agentic Index verified 2026-09-21

Alternatives to Braintrust

The closest documented capability profiles to Braintrust among agent infrastructure platforms tracked by Agentic Index, ordered by similarity on the same 14 point evidence the rankings use. No vendor pays for placement.

Similarity is computed from each vendor's Agentic Index coverage score evidence, axis by axis, not from the totals. How this evidence is graded

Contact us

Found a vendor we missed? Have feedback on the index? We'd love to hear from you.