HoneyHive

More than fifty libraries are instrumented automatically, including LangChain, LangGraph, AWS Strands, Google ADK and the OpenAI Agents SDK, and HoneyHive accepts traces from third-party agent platforms such as ServiceNow, Microsoft Copilot Studio and Salesforce Agentforce and sends alerts by email, Slack or webhook. These bring trace data in and send notifications out; none connects an agent to a real system to take actions.

SourceHoneyHive, honeyhive.ai/llms.txt and pricing, and docs.honeyhive.ai alertsread 2026-09-21

The customer's own agent workflows are drawn as graphs and trajectories from their traces. No workflows that sequence, branch or retry steps, or combine deterministic nodes with agent steps, are documented as something HoneyHive runs.

SourceHoneyHive, docs.honeyhive.ai graph-view and trajectory-viewread 2026-09-21

Evaluator templates score the customer's own retrieval for context relevance and answer faithfulness, and traces record the context that retrieval returned. No index, retrieval layer or knowledge API that grounds an agent's behavior in company data is documented.

SourceHoneyHive, docs.honeyhive.ai evaluator-templates and tracing conceptsread 2026-09-21

Annotation queues organize events for human review against human evaluators that define the fields a team rates, and those judgments become ground truth for automated evaluators; end-user feedback can be attached to traces. That is a human review surface on the agent's outputs; no approval step, consent checkpoint, runtime guardrail or pause and resume control over an agent's actions is documented.

SourceHoneyHive, docs.honeyhive.ai annotation-queues, evaluators/human and setting-user-feedbackread 2026-09-21

HoneyHive holds SOC 2 Type II, audited annually across all hosting options, with reports available through its Drata Trust Center, plus GDPR and HIPAA BAAs on Enterprise. Users sign in with Google, GitHub or Microsoft SSO or SAML 2.0 with Okta, Entra ID, Google Workspace, OneLogin or Ping, with roles provisioned from SAML group claims and MFA for every account; role-based access applies at organization, workspace and project level with custom roles on Enterprise; data is encrypted with AWS KMS at rest, with customer-managed keys on dedicated and self-hosted deployments, and TLS 1.2+ in transit.

SourceHoneyHive, docs.honeyhive.ai setup/security and workspace rolesread 2026-09-21

Each agent run is recorded as a session of OpenTelemetry events (model calls, tool calls, retrieval and custom spans) and shown in tree, timeline, graph, trajectory and thread views, with trace data queryable through the API, data export on every plan, and retention of 30 days on Developer and custom on Enterprise; the platform is built as an audit trail, so individual records cannot be deleted through the API or UI.

SourceHoneyHive, docs.honeyhive.ai tracing concepts, tree-view, graph-view, query-data and self-hosted data-flow, and honeyhive.ai/pricingread 2026-09-21

Sessions, events and datasets persist as observability and test records of what the customer's agent did, and a context retention evaluator scores whether that agent kept information across turns. No session, conversation, workflow or long term memory that an agent reads and writes is documented.

SourceHoneyHive, docs.honeyhive.ai tracing concepts and evaluator-templatesread 2026-09-21

Multi-tenant SaaS runs in AWS US-West-2; Dedicated Cloud keeps the data plane on physically isolated infrastructure in the customer's chosen AWS region, including the EU, with the control plane managed by HoneyHive; hybrid pairs a HoneyHive-managed control plane with a self-hosted data plane; and fully self-hosted deploys both planes in the customer's own account, with PrivateLink for private connectivity.

SourceHoneyHive, docs.honeyhive.ai setup managed, dedicated, self-hosted and security, and honeyhive.ai/pricingread 2026-09-21

HoneyHive publishes three official agent skills (honeyhive-instrument, honeyhive-evaluate and honeyhive-improve), described as reusable HoneyHive workflows a coding agent follows, and on Enterprise, organization templates let platform teams define blueprints of evaluators and monitoring charts that every new project inherits. These are packaged, editable starting points for setting HoneyHive up; no ready-made agents, packaged employees or deployable workflows for a buyer's own work are documented.

SourceHoneyHive, docs.honeyhive.ai ai-coding-agents and workspace/templatesread 2026-09-21

Online evaluations run enabled evaluators automatically and asynchronously on incoming traces that match their filters, at a set sampling rate, and alerts check cost, latency, error and evaluator-score thresholds on a regular cycle and notify by email, Slack or webhook when one is crossed. Evaluation starts on an incoming trace or a monitoring cycle, with no person asking.

SourceHoneyHive, docs.honeyhive.ai monitoring onlineevals, alerts overview and alertsread 2026-09-21

Workspace admins store their own provider credentials for Anthropic, OpenAI, Azure OpenAI, Amazon Bedrock, Gemini and Vertex AI, plus an OAuth gateway on self-hosted deployments, and HoneyHive uses them for LLM evaluators and the Playground, with provider access scoped by workspace; custom model providers are available in prompt management on every plan, and evaluators can run through Portkey.

SourceHoneyHive, docs.honeyhive.ai workspace/provider-keys and evaluators/portkey, and honeyhive.ai/pricingread 2026-09-21

HoneyHive publishes Python and TypeScript SDKs that are OpenTelemetry-native, a documented REST API for events, sessions, datasets, experiments and evaluators, a Control Plane API for workspaces, data planes and alerts, and a CLI for terminal access to resources; other languages send OpenTelemetry traces to its collector, and CI regression detection fits evaluation into a delivery pipeline. It also ships agent skills and an MCP docs search for coding agents.

SourceHoneyHive, docs.honeyhive.ai sdk-reference, API and control-plane references, cli-reference, ai-coding-agents and ci-regression-detectionread 2026-09-21

Experiments run an application over datasets and score it with versioned Python, LLM-as-a-judge and human evaluators, results are compared across runs, and CI regression detection catches drops before a change ships; online evaluations score production traces continuously, and production events can be curated into datasets for the next experiment.

SourceHoneyHive, docs.honeyhive.ai evaluation introduction, comparing evals, ci-regression-detection, onlineevals and dataset-curationread 2026-09-21

The HoneyHive documentation covers tracing, monitoring, evaluation, datasets, prompts, workspace settings and deployment, and no browser, desktop or computer control is documented.

SourceHoneyHive, docs.honeyhive.ai and llms-full.txtread 2026-09-21