Braintrust vs W&B Weave (2026)

Tools are hosted as functions, general purpose code that deployed prompts can call to perform operations or reach external data. They run serverless in an isolated environment, and any function can be used as a tool. Prompts can also connect to public MCP servers, after OAuth authentication, to reach external tools and data.

SourceBraintrust, braintrust.dev/docs deploy/functions and evaluate/write-prompts (MCP servers)read 2026-09-21

Integrations with LLM providers, agent SDKs and harnesses such as the OpenAI Agents SDK, Claude Agent SDK and Google ADK, and frameworks such as LangChain, CrewAI and LlamaIndex let Weave trace calls, and it traces activity between MCP clients and servers, while alerts go out by Slack and email. These bring trace data in and send notifications out, and none connects an agent to a real system to take actions.

SourceWeights & Biases, docs.wandb.ai Weave index (integrations overview, agent integrations, MCP) and wandb.ai/site/pricingread 2026-09-21

Workflows are hosted functions that chain two or more prompts and pass outputs between them. Prompts, tools and scorers also compose into applications. Functions are versioned and can be tagged for production, staging or development environments, so each sequence can be versioned and traced. Workflows do not include branching, loops, retries or fallback paths, or deterministic steps mixed with agent steps.

SourceBraintrust, braintrust.dev/docs deploy/functions, deploy/prompts and deploy/environmentsread 2026-09-21

Weave traces the customer's own multi-step and multi-agent workflows, including sub-agent delegations, and automations trigger actions from monitor metrics and trace activity. Weave itself runs no workflows that sequence, branch or retry steps, or that combine deterministic nodes with agent steps.

SourceWeights & Biases, docs.wandb.ai Weave index (trace sub-agents, automations)read 2026-09-21

Loop automations run Braintrust's agent on a daily, weekly, interval or cron schedule, with an instruction, a model and set write permissions, and send results to Slack or a webhook. Online scoring runs scorers automatically on incoming production logs. Alerts fire on log, time window and environment conditions.

SourceBraintrust, braintrust.dev/docs loop/automations, evaluate/score-online and observe/alertsread 2026-09-21

Monitors passively score production traffic, built-in signals score agent turns and tag quality and safety issues as they arrive, and automations trigger actions from monitor metrics and trace activity, with Slack and email alerts on the Pro plan. Scoring and actions start on incoming traffic without a person asking.

SourceWeights & Biases, docs.wandb.ai Weave index (monitors, signals, automations) and wandb.ai/site/pricingread 2026-09-21

Datasets hold test examples, and traces record the context a customer's retrieval returned. Braintrust's example RAG agent is built from a vector search tool the customer writes and a prompt. Braintrust does not ingest documents or keep an index or retrieval layer over company data.

SourceBraintrust, braintrust.dev/docs deploy/functions and annotate/datasetsread 2026-09-21

RAG applications can be evaluated with LLM judges, and Weave traces the context a customer's retrieval returns. There is no document ingestion, index, retrieval layer or knowledge API that grounds an agent in company data.

SourceWeights & Biases, docs.wandb.ai Weave index (evaluate RAG applications)read 2026-09-21

Loop's threads persist so a person can return to an investigation, and traces, datasets and experiments are kept as records. Agents have no session, workflow or long term memory that they read and write across runs, and there are no controls to review or scope such memory.

SourceBraintrust, braintrust.dev/docs loop and deploy/functionsread 2026-09-21

Conversations, threads and turns of the customer's agent are recorded, and datasets, prompts and models are versioned as tracked objects. There is no session, workflow or long-term memory that an agent reads and writes.

SourceWeights & Biases, docs.wandb.ai Weave index (trace threads, track and version objects)read 2026-09-21

Loop, Braintrust's agent, does read only work without interrupting. In an interactive thread it pauses for approval before it creates or edits anything. A scheduled Loop automation has nobody to ask, so each one carries write tool permissions that set what it may change, from nothing (read and report only) to named object types. Human review scores and review queues let people rate the customer's own traces.

SourceBraintrust, braintrust.dev/docs loop, loop/automations and annotate/human-reviewread 2026-09-21

Guardrails run inline scorers on a user's input or a model's output in real time, before outputs reach users, and can block or modify a response when a score crosses a threshold, with built-in and custom scorers and AWS Bedrock Guardrails as examples. Every guardrail result is stored as a monitor. Annotation queues route traces to domain experts for structured feedback.

SourceWeights & Biases, docs.wandb.ai/weave/guides/evaluation/guardrails and the Weave index (annotation queues)read 2026-09-21

Sign in supports SSO and SAML with Okta Workforce, Microsoft Entra ID and Google Workspace, and OIDC for custom providers. Role based access control uses built in and custom permission groups set at organization and project level.

API keys are stored as one way hashes and scoped to projects, provider secrets use AES-256 encryption, and all data is encrypted at rest and in transit. Retention policies are configurable, and an audit log records administrative actions such as permission changes and API key creation. Every plan includes SOC 2 Type II compliance, Enterprise adds a BAA for HIPAA, and a trust center sits at trust.braintrust.dev.

SourceBraintrust, braintrust.dev/docs security and admin/audit-logs, and braintrust.dev/pricingread 2026-09-21

Users sign in with SSO through Google, GitHub or enterprise providers such as Okta and Azure Active Directory over OIDC, are organized into teams and projects with a restricted visibility scope, get role-based access at team or project level, automate with scoped service accounts, and are managed through the SCIM API. Multi-tenant Cloud and Dedicated Cloud are both SOC 2 Type II compliant with reports on the W&B Security Portal, Dedicated Cloud is HIPAA compliant, and paid plans include audit logs and customizable data retention.

SourceWeights & Biases, docs.wandb.ai/weave/guides/platform and wandb.ai/site/pricingread 2026-09-21

Traces record each AI request's nested steps, including model calls, tool calls, retrieved context, errors, cost and latency, and teams examine, filter and debug them in the trace view. Logs can be exported, with automatic export to the customer's own S3 bucket on Enterprise. Retention is 14 days on Starter, 30 days on Pro with paid extension, and custom on Enterprise. Administrative actions go to a separate audit log.

SourceBraintrust, braintrust.dev/llms.txt, braintrust.dev/docs observe, admin/data-management/export and admin/audit-logs, and braintrust.dev/pricingread 2026-09-21

The Agents view shows conversations, turns, LLM calls, tool calls and sub-agent delegations with their cost, and the trace view follows nested execution paths. Calls can be filtered and exported through the Python SDK, REST API or UI, PII can be redacted from traces, and paid plans include audit logs and customizable data retention.

SourceWeights & Biases, docs.wandb.ai Weave index (agents view, trace tree, query and export calls, redact PII) and wandb.ai/site/weave and pricingread 2026-09-21

Customers can run Braintrust three ways. As SaaS, Braintrust operates both the control plane and the data plane. As BYOC, Braintrust operates the data plane inside the customer's cloud account. Self hosted on Enterprise, the customer deploys and controls the infrastructure that stores its AI data, with the data plane in its own VPC, while Braintrust provides the managed UI and platform. BYOC and self hosting keep logs, traces, datasets and prompts in the customer's account and region.

SourceBraintrust, braintrust.dev/docs admin/self-hosting, admin/deployment/byoc and securityread 2026-09-21

Weave runs on W&B Multi-tenant Cloud in a North America region of W&B's Google Cloud account, or on W&B Dedicated Cloud on AWS, Google Cloud or Azure, with data in a dedicated ClickHouse cluster in the cloud and region of the customer's choice, IP allowlisting and private connectivity. It also runs on Self-Managed instances the customer hosts. W&B fully manages the two cloud options.

SourceWeights & Biases, docs.wandb.ai/weave/guides/platformread 2026-09-21

Braintrust ships one ready made agent, Loop, which investigates project data, bootstraps scorers, generates test cases, iterates on prompts and builds charts. The same agent powers Patterns, a preconfigured automation that finds recurring failures, and the trace Debugger. There is no set of ready made agents or templates to choose from.

SourceBraintrust, braintrust.dev/docs loop, loop/automations and observe/patterns, and braintrust.dev/pricingread 2026-09-21

W&B Skills install into a coding agent to teach it to train models, build agents and analyze experiments on the W&B platform, a packaged, ready-to-install starting point for another agent. Weave has no ready-made agents, packaged employees or deployable workflows for a customer's own work. The Claude Code, Codex and OpenClaw plugins trace sessions and are integrations, not packaged agents.

SourceWeights & Biases, docs.wandb.ai/llms.txt (W&B Skills) and the Weave index, and wandb.ai/site/weaveread 2026-09-21

The Braintrust Gateway gives one API for routing model requests to AI providers. It supports the customer's own provider keys, configured at organization level, workload identity federation for supported providers, custom providers, and Braintrust's built in models billed against monthly credits. Deployed prompts carry their own model configuration, and Loop runs on the model and provider the customer chooses.

SourceBraintrust, braintrust.dev/llms.txt and braintrust.dev/docs deploy/gateway, admin/ai-providers and loopread 2026-09-21

Customers can connect OpenAI-compatible inference endpoints to a project as custom runtimes, managed from the UI or the Python and TypeScript SDKs, compare models in the Playground and Evaluation Playground, and run open source models through W&B Serverless Inference. Customers choose models and bring their own endpoints.

SourceWeights & Biases, docs.wandb.ai Weave index (custom runtimes, Playground, Serverless Inference)read 2026-09-21

Developers get a REST API with an OpenAPI description, SDKs in Python, TypeScript, Go, Java, Ruby and C#, and the bt CLI for running evals, querying logs, syncing data and managing functions. An MCP server, authenticated with OAuth 2.0 and PKCE, lets coding agents reason over project data. Custom tools, scorers and workflows are pushed from code as versioned functions, and evals run in CI.

SourceBraintrust, braintrust.dev/llms.txt and braintrust.dev/docs security, deploy/functions and evaluate/run-in-ciread 2026-09-21

Weave offers Python and TypeScript SDKs, a service REST API with an OpenAPI description, and an OpenTelemetry endpoint that accepts traces from any pipeline without the SDK. The W&B MCP server lets an IDE or agent query W&B data and documentation, W&B Skills teach coding agents to use the platform, and the Pro plan adds CI/CD automations.

SourceWeights & Biases, docs.wandb.ai/llms.txt and the Weave index (OpenTelemetry, query and export calls), and wandb.ai/site/pricingread 2026-09-21

Evals run an AI system over versioned datasets and score outputs with LLM as a judge, autoevals, custom code or human review scorers. Experiments are compared side by side, and evals run in CI, where configurable checks gate quality before production. Online scoring applies the same scorers to production logs, so quality is scored over time and failures become new dataset examples.

SourceBraintrust, braintrust.dev/docs evaluate, run-in-ci, score-online and annotate/datasetsread 2026-09-21

Evaluations run applications against versioned datasets with built-in, local and custom scorers, including LLM judges, for single-turn and multi-turn agents. Evaluations are compared side by side to spot regressions and ranked on leaderboards, monitors passively score production traffic, and the Pro plan adds CI/CD automations.

SourceWeights & Biases, docs.wandb.ai Weave index (evaluations, scorers, compare evaluations, monitors) and wandb.ai/site/weave and pricingread 2026-09-21

Custom code functions run in isolated serverless environments. No agent controls a browser, a desktop or a computer.

SourceBraintrust, braintrust.dev/docs security and llms-full.txtread 2026-09-21

Weave covers tracing, evaluation, monitoring, guardrails, annotation and deployment, and no agent controls a browser, desktop or computer. The CoreWeave Sandboxes listed beside Weave are isolated environments for running agents, and they execute code instead of controlling a browser, desktop or computer.

SourceWeights & Biases, docs.wandb.ai Weave index and wandb.ai/site/weave and pricing; wandb.ai/site/pricingread 2026-09-21