LangWatch

Langy, LangWatch's agent, changes code through bot-authored pull requests opened by a GitHub App, and LangWatch workflows, which can be published as agents or API endpoints, call external systems through HTTP nodes and code blocks that reference encrypted project secrets; the AI Gateway governs which tools and servers a virtual key may reach.

SourceLangWatch, langwatch.ai/docs/llms.txt (Langy pull requests, workflows, secrets, routing policies)read 2026-09-21

A LangWatch workflow is a graph of nodes that LangWatch runs from an Entry point to an End node, mixing LLM prompt nodes with Python code, HTTP and evaluator nodes; versions are saved and restorable, and a workflow can be published as an evaluator, as an agent under test, or as an API the customer's code calls. Loops, branching, retries and fallback paths are not documented.

SourceLangWatch, langwatch.ai/docs/workflows/overview and llms.txt (building a workflow, workflow as agent); langwatch.ai/docs/workflows/overview.mdread 2026-09-21

Built-in RAG evaluators score retrieval, and cookbooks on embedding tuning and vector versus hybrid search cover the customer's own pipeline. No document ingestion, index, retrieval layer or knowledge API that grounds an agent in company data is documented as part of LangWatch.

SourceLangWatch, langwatch.ai/docs/llms.txt (built-in evaluators, cookbooks)read 2026-09-21

Guardrails run evaluators inline to block or modify harmful responses in real time, the AI Gateway enforces budgets, rate limits and routing policies, including which tools and servers a key may reach, before a request goes through, and Langy's code changes arrive as pull requests that a person reviews before merge.

SourceLangWatch, langwatch.ai/docs/llms.txt (guardrails, AI Gateway budgets and routing policies, Langy pull requests)read 2026-09-21

Documented controls include single sign-on with directory provisioning and sign-in policies, SCIM 2.0 provisioning from Okta, Entra ID and others with group-to-role mapping, role-based access with custom roles and role bindings at organization, team and project level, an audit log of every change made through settings, the API and the AI Gateway with who, where and what changed, OCSF export to a SIEM, retention set per organization, team or project, and encrypted write-only secrets; the homepage states ISO 27001 certification and GDPR compliance, with a Vanta-monitored trust center.

SourceLangWatch, langwatch.ai/docs/llms.txt (SSO, SCIM, RBAC, audit log, data retention, secrets, OCSF export), langwatch.ai homepage and pricingread 2026-09-21

LangWatch traces LLM calls, tool executions and retrieval steps through OpenTelemetry-native SDKs and integrations, including coding assistants such as Claude Code and Codex; an audit log records every change through settings, the API and the gateway, governance events export to a SIEM as OCSF records, analytics export into the customer's dashboards, and retention defaults are published per plan and adjustable per organization, team or project.

SourceLangWatch, langwatch.ai/docs/llms.txt (observability, audit log, OCSF export, data retention) and langwatch.ai/docs/pricingread 2026-09-21

Traces, threads and datasets are stored as records of the customer's agent, and each Langy conversation is kept in its own sandboxed worker. No session, workflow or long term memory that an agent reads and writes across runs is documented.

SourceLangWatch, langwatch.ai/docs/llms.txt (concepts, datasets, how Langy works)read 2026-09-21

Customers can run LangWatch as managed multi-tenant SaaS in the EU, US, UK or APAC, self-hosted with Docker, Kubernetes and Helm or in the customer's VPC, or hybrid with the data plane on the customer's infrastructure and the control plane on LangWatch's; the open source repository carries every feature for local use, and an Enterprise license adds longer retention to self-hosted deployments.

SourceLangWatch, langwatch.ai homepage and langwatch.ai/docs/llms.txt (self-hosting overview, editions and licensing, hybrid setup)read 2026-09-21

LangWatch ships Langy, an automated AI engineer that reads traces and evals, writes Scenario tests, optimizes prompts and opens pull requests, and a Skills Directory of installable skills, including skills for product managers and domain experts, that set a coding assistant up to work with LangWatch. That is one packaged agent and a set of setup skills, not a set of ready-made agents or workflow templates a buyer selects among for their own work.

SourceLangWatch, langwatch.ai/docs/llms.txt (Langy, skills directory) and langwatch.ai homepageread 2026-09-21

Online evaluation monitors run an evaluator on every matching trace or thread as it arrives, alerts and automations detect regressions and notify teams, automations add every new trace matching a filter to a dataset, and Langy can set up alerts and automations from what it finds across traces. So scoring and actions start on incoming traffic without a person asking.

SourceLangWatch, langwatch.ai/docs/llms.txt (online evaluation, alerts and automations, datasets from traces, Langy insights)read 2026-09-21

The LangWatch AI Gateway gives one OpenAI- and Anthropic-compatible endpoint across providers including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, xAI, Groq, Cerebras and DeepSeek, with provider credentials held in LangWatch, a virtual key per application, routing policies that set which providers a key routes through, the fallback order and model tiers, and budgets and rate limits enforced before the request; model providers are stored once per organization, team or project for evaluators, workflows and Langy.

SourceLangWatch, langwatch.ai/docs/llms.txt (AI Gateway, routing policies, budgets, providers, model providers)read 2026-09-21

LangWatch publishes a REST API (including the Workflows and Instant Evals APIs), SDKs across Python, TypeScript, Go and Java, the langwatch CLI that drives the platform from the terminal for people and coding assistants, an MCP server that gives coding assistants access to traces, tests and evaluations, installable skills, personal and service API keys for development and CI/CD, and CI integration that gates merges on eval results.

SourceLangWatch, langwatch.ai/docs/llms.txt (CLI, MCP server, API keys, API reference) and langwatch.ai/pricingread 2026-09-21

Experiments run over datasets with built-in, saved and workflow-built evaluators. LangWatch also tests agents with multi-turn scenarios in which a simulated user acts against the agent and a judge scores the conversation, groups scenarios into test suites, compares agents side by side on pass rate, cost and latency, gates merges on eval results in CI, scores production traffic with online monitors, and optimizes prompts and tool contracts with the scenario suite as the quality gate.

SourceLangWatch, langwatch.ai/docs/llms.txt (agent testing, evaluations, improve your agent) and langwatch.ai/pricingread 2026-09-21

Tracing, agent testing, evaluation, the AI Gateway, governance, Langy and self-hosting make up the documented product, and no browser, desktop or computer control by an agent is documented; Langy works in a sandbox through the CLI and code, not by controlling a browser or desktop.

SourceLangWatch, langwatch.ai/docs/llms.txtread 2026-09-21