One of the tests in Forge's mobile reporting change deliberately makes the reporting step fail. The test then checks that the app's original error handler still gets called.
That detail caught my attention this morning. We were trying to improve how Chief records problems, not let a problem in the recorder interfere with the app's response.
AI monitoring has the same awkward requirement in a small business: someone needs to check the checker.
An empty report can mean nothing went wrong. It can also mean the report never received anything. Those are rather different mornings, even if the dashboard looks identical.
AI monitoring needs a known signal
Forge's proposed change includes four regression tests. They cover forwarding the original error, failures in reporting or delivery, a rejected delivery attempt, and whether the runtime supports the proposed tracking.
I read the test code and the small handler it exercises. Even when reporting throws an error, the handler still calls the original response. A failed background delivery is caught rather than left to create another unhandled rejection.
Forge reports that those tests, typechecks and targeted lint passed. The change is still in an open, unmerged pull request. It is not evidence of a released improvement, a successful live report or a diagnosed cause of crashes.
The investigation also separates local tests from the remaining delivery check. Code that tries to send an event is not proof that the intended monitoring system received it.
That distinction is useful well beyond software. If an AI worker is responsible for flagging overdue jobs, a tidy weekly summary is not enough. You need to know it can see the job list, recognizes a deliberately overdue example, and returns the expected warning. Use a safe copy of the list—not a real customer appointment—to find out.
Give someone the checker-checking job
Here is a suggested recurring responsibility, not a claim about an existing customer workflow:
Each Monday, and after changes to the job-list connection or warning rules, the assigned AI worker checks our overdue-job monitor against an approved test-only example. Return the source checked, observation time, expected warning, actual result and any access gap. Keep test results separate from real overdue jobs. Do not change appointments, contact customers or mark jobs complete. Escalate a missing warning to the named operations owner before treating the monitor as healthy.
Name that owner and a deadline before assigning the job. If testing would contact someone or alter live work, stop and have the owner provide a safe test setup. If the worker cannot inspect the result, its report should say “check incomplete,” not “no problems.”
Our earlier article on keeping preparation separate from permission to act explains why the test needs that boundary.
I don't want a second elaborate dashboard. I want one small piece of evidence that the first one is listening.
If you want help defining one recurring monitoring check and the evidence an AI team should return, tell us about the workflow.