Thanks, that's exactly the gap. Every failure in the piece was something that stopped happening, and none of my checks were looking for an absence. Where it stands today: it's still a scheduled cross-check, not a continuous signal. The daily check now compares Ceph OSD state against node readiness, flags a helm release that isn't deployed (added after the silently failed upgrades), and counts lease workloads against what's active on-chain. That last check turned out to have its own blind spot: it had been over-counting leases for months, which I only found while preparing the latest operator report (October report). Scheduled checks tell you how things were at the moment they ran, nothing more. The absence alerts you describe are the next thing I want to build: Bid rate against a baseline: alert when no bid transaction has appeared in N hours. That's the dead-man's switch for the four-day gap. A heartbeat on the last successful upgrade and the last withdrawal, so a silent failure goes stale rather than staying quiet. An external probe of the provider endpoint and a tenant URL from outside the network. On the synthetic per-node canary: yes, that's the right shape for the Ready-but-no-data-plane case. The OSD cross-check only covers nodes that host storage, and a few (the control planes and two GPU nodes) have none, so a small pod-to-pod and pod-to-storage probe pinned to each node would close that properly. Not built yet; it's on the list. Appreciate the specifics. absent() over a window is a cleaner way to put it than anything I wrote.
