Metrics and feedback
What each number on the Metrics page means, and how your thumbs up or down on a reply are used.
The Metrics page, titled How the agent is doing, shows how well the agent works on this platform. Use it to see whether you can trust its replies, and what it costs.
Choose a period#
Choose Last 7 days, Last 30 days or Last 90 days from the menu at the top right. The line under the title says which period you're looking at. Every number counts tickets that arrived in that period, and everything that happened on them since.
Quality and speed#
The numbers sit in small cards. A share also shows as a thin bar under the number.
| Number | What it means |
|---|---|
| Replies sent without edits | The share of sent replies that went out exactly as the agent wrote them. Underneath: how many of how many sent. |
| Root cause confirmed right | The share of thumbs up among all votes on the agent's analysis. Underneath: how many of how many rated. |
| Time to findings | The typical time from a ticket arriving to the agent's first findings, in minutes or hours. It counts every ticket, including ones the log watcher raised, which have no reply. It's the median, so a few slow tickets don't skew it. |
| Cost per ticket | The AI cost per ticket the agent worked on. Underneath: the total for the period, which adds up each ticket's cost as the ticket shows it, so the two always match. Closed tickets count too. |
Volume#
| Number | What it means |
|---|---|
| Tickets | Tickets that arrived in the period. |
| Sent automatically | Replies sent without a person approving them, by automatic replies. |
| Pull requests opened | Code fixes that became a pull request (or merge request). |
| Agent runs that failed | Pieces of agent work that didn't finish. Underneath: out of how many runs. The card turns amber when any failed. When none failed, it shows a green All clear. |
A dash (—) means there isn't enough data yet, for example no replies sent in the period.
Only replies that were actually sent count towards Replies sent without edits. Replies approved on a platform with no ticket system aren't sent, so they don't count.
About cost
Cost is what the AI service reported for each piece of work. If the agent uses a Claude subscription token rather than an API key, the cost is an estimate. See Keys.
Giving feedback#
Was the analysis right? has a thumbs up and a thumbs down. On a ticket, you find it at the side, at the bottom of the Ticket box, once the agent has findings. On a reply's own page (opened from the Overview), it's at the bottom of What the agent found.
- Click thumbs up if the agent found the right root cause.
- Click thumbs down if it didn't.
You can vote before or after you approve, edit or reject. Each person has one vote per reply. Clicking again changes your vote. Once people have voted, the count shows next to each button, so you can see what your colleagues thought.
How feedback is used#
Your votes make up Root cause confirmed right on this page. That's the clearest sign of whether the agent understands your system.
Feedback doesn't change how the agent works by itself. To improve its answers:
- Correct it on the ticket, in Write › To the agent. See Talking to the agent.
- Give it better background in About your system under Settings › Agent, and in each repository's description under Settings › Code. See Agent settings.
- Set rules for all your assistants under Organisation › Rules. See Rules.
Reading the numbers together
High Replies sent without edits and high Root cause confirmed right mean the agent is doing well. If you rarely edit its replies and it's usually right, you may be ready for automatic replies on simple tickets.