AgentFleet (runtime + dashboard)
At a glance
- AgentFleet is the runtime where deployed agents live and execute.
- AgentFleet Dashboard is the observability layer — HITL queue, per-agent SLAs, latency, accuracy, alert feed.
- AgentFleet Builder is the configuration surface for creating and editing agents.
- CS folks live in the Dashboard when monitoring live deployments.
Why this matters
Every agent in Part 4 · Agent catalog runs on AgentFleet. When a customer asks “how is the agent performing?”, you answer from the Dashboard. When you need to spin up a new agent for a POC, you start with the Builder. This is the tool you’ll use daily.
Three paths to build an agent
There are three ways to create agents on the platform, depending on your comfort level and the complexity of the use case:
| Path | What it is | When to use | Who uses it |
|---|---|---|---|
| InternalDashboard permissions | Enable agent capabilities on any existing Shipsy dashboard | Adding AI to an existing product surface | CS + product |
| AgentFleet Builder UI | Visual builder for agent configuration | Standard agent patterns, POCs, demos | CS + solutions eng |
| Claude Code Shipsy Toolkit | Code-level agent creation via the agentfleet skill | Bespoke agents, deep customization | Solutions eng |
Pre-built agents available in the Builder
These agents are already templated and can be configured for a new customer without starting from scratch:
- Rain mode (weather-triggered ops adjustments)
- Address Intelligence
- RFQ Agent (freight quotation)
- POD Intelligence
- Control Tower
Access
| Environment | URL | Use |
|---|---|---|
| Demo org (Builder) | agentfleetdemo.portal.shipsy.io | Build and test agents |
| Demo org (runtime) | agentfleet.demo.shipsy.io | See agent execution |
Restriction: Demo orgs only for now. Production deployment requires AI team sign-off.
The Dashboard
The AgentFleet Dashboard surfaces real-time operational metrics. The first five things to look at when monitoring a live deployment:
- HITL queue — how many conversations are waiting for human review? If this is growing, the agent is under-confident or the escalation threshold needs tuning.
- Escalation rate — % of conversations that required human intervention. Benchmark: < 20% for a mature agent.
- Latency — first-response time and total resolution time. For voice agents, first-token latency matters most.
- Accuracy by intent — are there specific query types where the agent is underperforming?
- Alert feed — SLA breaches, anomalies, system errors.
Demo & sandbox links
These are live environments you can explore:
| Demo | URL | What it shows |
|---|---|---|
| Freight rate intelligence | freight-rates.app.shipsy.ai | RFQ agent in action |
| Carrix port dashboard | carrix.app.shipsy.ai | Port operations control tower |
| POD quality check | floward-pod-qc.vercel.app | Nexa-style POD validation |
| Invoice reconciliation | invoice-recon-australia.vercel.app | Nexa invoice recon |
| Charge intelligence | charge-intelligence.app.shipsy.ai | Charge analysis agent |
AgentFlow vs AgentFleet Builder
| AgentFlow | AgentFleet Builder | |
|---|---|---|
| Standard pattern (CX agent, control tower) | Best fit | Overkill |
| Customer wants to self-serve | Best fit | Too technical |
| Heavy customization / new orchestration | Limited | Best fit |
| Tight integration with Shipsy-internal services | Limited | Best fit |
| Engineering will own the agent long-term | Not ideal | Best fit |
See AgentFlow for the visual workflow builder.
Sources
- Agentic Capabilities — Master deck (75 slides)
- Agent Platform Capabilities slide
- AgentFleet overview (3 slides)
- CPG AgentFleet — Detailed Use Cases
- LSP B2C AgentFleet — Detailed Use Cases
- Slack: #general — Hitesh Sarup (25 May 2026)
Changelog
- 26 May 2026: Enriched with Builder paths, demo URLs, sandbox links, and Dashboard guidance.