fix(health): one prober, on the path the rules actually use

A node reached only as a chain hop was being measured twice, and the reading
the panel showed was the wrong one. On a router in Russia that is not a cosmetic
difference: a node the chain carries fine behind a WireGuard hop is dead when
dialled straight out of the WAN, so the group card read "0 of 2 alive" while
that very group was carrying every packet.

Two dial paths existed outside the observatory plan. URLTestGroup.PostStart
warmed up every urltest group at box start whether or not any rule reached it,
and the panel's Test button reached URLTest.DialContext, whose first act is
Touch() — arming a ticker that re-swept those groups directly every probe
interval for the next thirty minutes. Both wrote under the BASE node tag, and
both dialled the base outbound, which carries no chain detour at all.

The observatory was never the liar: its plan roots come from the rules, and a
chain hop copy is stored only under its own tag, so no plan job could ever
write under a base tag. The fix is therefore to remove the other two paths, not
to touch the plan.

TestGroups now asks the observatory for an out-of-turn pass and reports what it
measured; a target no enabled rule routes to is not dialled at all and says so.
Unused urltest groups stand down their own self-check via a new SelfCheck option
(nil keeps today's behaviour, so every existing config is unchanged). The one
direct dial left is the exit-address lookup, which has no other possible source
— it now runs only for a target that is both routed and already read alive, so
it travels the routed path and never touches an unused group.

Chain hop wrappers are probed as measurements of their own and surfaced as
chains[].hops[], because "which hop is dead" is the question an operator has and
the chain-level verdict cannot answer it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

option/group.go

@@ -18,6 +18,32 @@ type URLTestOutboundOptions struct {
 	// lx: SPEC 019 v2 — load-balancing.
 	Mode     string                  `json:"mode,omitempty"` // least_test (default) | round_robin
 	Balancer *URLTestBalancerOptions `json:"balancer,omitempty"`
+	// lx: health board §5.C — SelfCheck stands the group's OWN background
+	// health-check up or down. nil/absent == true, so every existing config keeps
+	// today's behaviour.
+	//
+	// Why this exists at all: a urltest group probes its members BY ITSELF — a
+	// warm-up sweep at PostStart and a ticker for as long as traffic keeps
+	// touching it — and it dials the members' outbounds DIRECTLY, from the
+	// router, over whatever the default WAN route is. For a group that traffic
+	// actually flows through, that is exactly right: the probe travels the same
+	// path the connections do. But for a group NO routing rule reaches, that
+	// same probe measures a path nothing uses — and it stores the result under
+	// the members' BASE tags, which every health consumer then reads as "the
+	// node's health". A node that is blocked on the direct WAN and perfectly
+	// alive behind a tunnel therefore reads "dead" the moment such a group
+	// probes it; the reading is not merely stale, it is FALSE, and it poisons
+	// the shared board for everyone (selection, the panel, the observatory's
+	// freshness gate). SelfCheck=false is how the control plane stands such a
+	// group's own schedule down: the shater engine computes which groups the
+	// applied rules actually reach (the observatory's used-set) and disables
+	// the self-check on the rest, so the ONLY prober left is the observatory —
+	// which probes along the real dial paths and nothing else.
+	//
+	// The flag suppresses only the group's own SCHEDULE (the PostStart warm-up
+	// and the Touch ticker). An EXPLICIT CheckOutbounds/URLTest call — the
+	// adapter interface a human or an API invokes on purpose — still works.
+	SelfCheck *bool `json:"self_check,omitempty"`
 }
 
 // URLTestBalancerOptions configures round_robin: a fixed-size pool of live nodes, lazily

protocol/group/urltest.go

@@ -44,6 +44,13 @@ type URLTest struct {
 	group                        *URLTestGroup
 	interruptExternalConnections bool
 	balancer                     *balancer // lx: SPEC 019 — nil for least_test (default)
+	// lx: health board §5.C — true when options.SelfCheck == false: the group's
+	// OWN probing schedule (PostStart warm-up + Touch ticker) is stood down and
+	// the observatory is the only thing that measures its members. Stored
+	// INVERTED so the zero value keeps today's behaviour for every construction
+	// path that does not go through NewURLTest (hand-built groups in tests).
+	// See option.URLTestOutboundOptions.SelfCheck for the full reasoning.
+	selfCheckDisabled bool
 }
 
 func NewURLTest(ctx context.Context, router adapter.Router, logger log.ContextLogger, tag string, options option.URLTestOutboundOptions) (adapter.Outbound, error) {
@@ -71,6 +78,9 @@ func NewURLTest(ctx context.Context, router adapter.Router, logger log.ContextLo
 		idleTimeout:                  time.Duration(options.IdleTimeout),
 		interruptExternalConnections: options.InterruptExistConnections,
 		balancer:                     balancer,
+		// nil/absent means true (self-check on) — the documented default, so a
+		// config written before the flag existed behaves exactly as it always has.
+		selfCheckDisabled: options.SelfCheck != nil && !*options.SelfCheck,
 	}
 	if len(outbound.tags) == 0 {
 		return nil, E.New("missing tags")
@@ -92,6 +102,9 @@ func (s *URLTest) Start() error {
 		return err
 	}
 	group.balancer = s.balancer // lx: SPEC 019 v2 — health-check drives the pool through it
+	// lx: health board §5.C — carry the stand-down flag onto the group the same
+	// way the balancer travels: set after construction, immutable from then on.
+	group.selfCheckDisabled = s.selfCheckDisabled
 	if s.balancer != nil {
 		// lx: health board §5.B — slot liveness reads through the board verdict, so a
 		// death recorded by any prober or a failed dial takes effect on the next pick,
@@ -319,6 +332,12 @@ type URLTestGroup struct {
 	lastActive                   common.TypedValue[time.Time]
 	lastSelected                 common.TypedValue[string] // lx: SPEC 019 — Now() in balanced modes
 	balancer                     *balancer                 // lx: SPEC 019 v2 — round_robin pool; nil for least_test
+	// lx: health board §5.C — mirrors URLTest.selfCheckDisabled (set by Start,
+	// immutable afterwards, zero value = probing on). Guards ONLY the group's
+	// own schedule: the PostStart warm-up sweep and the Touch ticker. An
+	// explicit CheckOutbounds/URLTest call is untouched — the flag stands down
+	// the schedule, not the capability.
+	selfCheckDisabled bool
 }
 
 func NewURLTestGroup(ctx context.Context, outboundManager adapter.OutboundManager, logger log.Logger, outbounds []adapter.Outbound, link string, interval time.Duration, tolerance uint16, idleTimeout time.Duration, interruptExternalConnections bool) (*URLTestGroup, error) {
@@ -362,14 +381,35 @@ func (g *URLTestGroup) PostStart() {
 	g.lastActive.Store(time.Now())
 	// lx: SPEC 019 v2 — seed the pool so round_robin can route from the first connection,
 	// before the first health-check completes (history-warm nodes first, else config order).
+	// The seed only READS the board, so it runs even with the self-check stood down.
 	g.seedPool()
-	go g.CheckOutbounds(false)
+	// lx: health board §5.C — the warm-up sweep is the first half of the group's
+	// own probing schedule, and it fires for EVERY group at box start, including
+	// groups no routing rule reaches. For those, the sweep dials every member
+	// directly from the router — a path nothing uses — and records the outcome
+	// under the members' base tags, forging the board reading the observatory
+	// exists to keep honest. A stood-down group therefore skips it entirely; the
+	// observatory (or nothing, for a truly unused group) is what measures its
+	// members.
+	if !g.selfCheckDisabled {
+		go g.CheckOutbounds(false)
+	}
 }
 
 func (g *URLTestGroup) Touch() {
 	if !g.started {
 		return
 	}
+	// lx: health board §5.C — Touch's only job is to keep the group's OWN
+	// probing ticker alive while traffic flows. With the self-check stood down
+	// there is deliberately no ticker to start or feed: the observatory owns the
+	// schedule, and a stray dial through an unused group (a stale rule cache, a
+	// manual pin) must not arm 30 minutes of direct probing under the members'
+	// base tags. Checked before the lock because the flag is immutable after
+	// Start, exactly like the started fast-path above.
+	if g.selfCheckDisabled {
+		return
+	}
 	g.access.Lock()
 	defer g.access.Unlock()
 	if g.ticker != nil {

protocol/group/urltest_selfcheck_lx_test.go

@@ -0,0 +1,116 @@
+package group
+
+// lx: health board §5.C tests — SelfCheck stands the group's OWN probing
+// schedule down: no PostStart warm-up sweep, no Touch ticker. The explicit
+// CheckOutbounds path stays available, and the nil default keeps probing.
+
+import (
+	"context"
+	"testing"
+	"time"
+
+	"github.com/sagernet/sing-box/common/urltest"
+	"github.com/sagernet/sing-box/log"
+	"github.com/sagernet/sing-box/option"
+)
+
+// waitForHistory polls until the store holds an entry for tag or the deadline
+// passes; reports whether it appeared. PostStart's sweep runs on its own
+// goroutine, so both directions of the assertion need a bounded wait.
+func waitForHistory(hist *urltest.HistoryStorage, tag string, deadline time.Duration) bool {
+	stop := time.Now().Add(deadline)
+	for time.Now().Before(stop) {
+		if hist.LoadURLTestHistory(tag) != nil {
+			return true
+		}
+		time.Sleep(5 * time.Millisecond)
+	}
+	return false
+}
+
+// A group with the self-check stood down writes NOTHING to the history storage
+// on PostStart: the warm-up sweep — which would dial the member directly from
+// the router and mark the failure under its base tag — must not fire. And
+// Touch, the other half of the schedule, must not start a ticker either.
+func TestSelfCheckDisabledPostStartWritesNothing(t *testing.T) {
+	hist := urltest.NewHistoryStorage()
+	a := &healthNode{tag: "a", fail: true}
+	manager := managerOf(a)
+	g := healthTestGroup(hist, manager, a)
+	g.selfCheckDisabled = true
+
+	g.PostStart()
+	// The absence of a write is the assertion, so give the (non-existent) sweep
+	// real time to have happened before declaring victory.
+	if waitForHistory(hist, "a", 150*time.Millisecond) {
+		t.Fatal("a stood-down group's PostStart wrote to the board; the warm-up sweep must not fire")
+	}
+
+	g.Touch()
+	g.access.Lock()
+	ticker := g.ticker
+	g.access.Unlock()
+	if ticker != nil {
+		t.Fatal("Touch armed the probing ticker on a stood-down group")
+	}
+}
+
+// The default (SelfCheck nil, i.e. the zero-value field on a hand-built group)
+// keeps today's behaviour: PostStart's warm-up sweep runs and records the
+// failing member on the board.
+func TestSelfCheckDefaultStillProbesOnPostStart(t *testing.T) {
+	hist := urltest.NewHistoryStorage()
+	a := &healthNode{tag: "a", fail: true}
+	manager := managerOf(a)
+	g := healthTestGroup(hist, manager, a)
+
+	g.PostStart()
+	if !waitForHistory(hist, "a", 5*time.Second) {
+		t.Fatal("default group's PostStart never probed; the self-check must stay on unless stood down")
+	}
+	if v := hist.Verdict("a", 10*time.Minute); v != urltest.VerdictDead {
+		t.Fatalf("verdict(a) = %v, want dead from the warm-up sweep", v)
+	}
+}
+
+// An EXPLICIT CheckOutbounds still probes a stood-down group: the flag
+// suppresses the group's own schedule, never a deliberate request (the adapter
+// interface a human or an API invokes on purpose).
+func TestSelfCheckDisabledExplicitCheckStillProbes(t *testing.T) {
+	hist := urltest.NewHistoryStorage()
+	a := &healthNode{tag: "a", fail: true}
+	manager := managerOf(a)
+	g := healthTestGroup(hist, manager, a)
+	g.selfCheckDisabled = true
+
+	g.CheckOutbounds(true)
+	if hist.LoadURLTestHistory("a") == nil {
+		t.Fatal("an explicit CheckOutbounds(true) did not probe; the flag must only stand down the schedule")
+	}
+}
+
+// The option → outbound plumbing: nil/absent means on, an explicit false means
+// stood down, an explicit true means on. NewURLTest is the only place the
+// option is read, so this is where a plumbing regression would hide.
+func TestSelfCheckOptionPlumbing(t *testing.T) {
+	build := func(selfCheck *bool) *URLTest {
+		t.Helper()
+		opts := option.URLTestOutboundOptions{Outbounds: []string{"a"}}
+		opts.SelfCheck = selfCheck
+		ob, err := NewURLTest(context.Background(), nil, log.NewNOPFactory().Logger(), "t", opts)
+		if err != nil {
+			t.Fatalf("NewURLTest: %v", err)
+		}
+		return ob.(*URLTest)
+	}
+	if build(nil).selfCheckDisabled {
+		t.Fatal("nil SelfCheck must keep the self-check ON (the compatibility default)")
+	}
+	on, off := true, false
+	if build(&on).selfCheckDisabled {
+		t.Fatal("SelfCheck=true must keep the self-check on")
+	}
+	if !build(&off).selfCheckDisabled {
+		t.Fatal("SelfCheck=false must stand the self-check down")
+	}
+}

shater/apply/apply.go

@@ -205,11 +205,15 @@ func (a *Applier) configureObservatory(m *model.Model, opts option.Options) {
 	})
 }
 
-// TestGroups launches the engine's one-shot exit test (delay + exit address, F2)
-// of the named groups/chains, returning started=false when a run is already in
-// flight or the engine is absent. names empty/nil = every group and every chain.
-// It takes NEITHER the apply mutex nor the flock, so kicking off a test never
-// blocks behind an Apply.
+// TestGroups launches the engine's one-shot group/chain test (F2): the engine
+// asks its observatory for an out-of-turn pass and reports what it measured,
+// plus the exit address for alive rule-routed targets — it no longer dials
+// health probes of its own. Returns started=false when a run is already in
+// flight or the engine is absent. names empty/nil = every group and every
+// chain. probeURL is passed through for signature stability and IGNORED by the
+// engine (the probe URL is a global observatory setting now). It takes NEITHER
+// the apply mutex nor the flock, so kicking off a test never blocks behind an
+// Apply.
 func (a *Applier) TestGroups(names []string, probeURL string) (started bool) {
 	if a.eng == nil {
 		return false

shater/engine/engine.go

@@ -193,6 +193,17 @@ func (e *Engine) Apply(opts option.Options) (changed bool, err error) {
 }
 
 func (e *Engine) applyLocked(opts option.Options) (bool, error) {
+	// Stand down the self-check of urltest groups no rule reaches (see
+	// selfcheck.go for the whole argument). This MUTATES opts in place — the
+	// option structs are pointers behind an `any` — and it must run BEFORE the
+	// hash below, so the hash describes the config that is really built: a rule
+	// change that flips a group used<->unused is then a real change that
+	// triggers a swap, and an unchanged config hashes identically on every
+	// reconcile because the stand-down is deterministic.
+	if stood := standDownUnusedSelfCheck(opts); stood > 0 && e.log != nil {
+		e.log.Info("apply: stood down self-check on ", stood, " unused urltest group(s); the observatory is their only prober")
+	}
+
 	newHash, err := e.hashOptions(opts)
 	if err != nil {
 		return false, E.Cause(err, "hash options")

shater/engine/grouphealth.go

@@ -1,6 +1,7 @@
 package engine
 
 import (
+	"sort"
 	"strings"
 
 	"github.com/sagernet/sing-box/adapter"
@@ -320,11 +321,15 @@ func parseGroupCopyTag(group, tag string) (member string, ok bool) {
 	return member, true
 }
 
-// ChainHealth is one configured chain's reachability, mirroring GroupHealth.Used
-// for the chain card (plan §5.E): a chain no enabled rule routes through is never
-// probed — the observatory walks only reachable paths — and the panel renders it
-// "unused" rather than as a health problem. A chain has no membership counters: it
-// is a fixed path, and its end-to-end health is the exit test's job, not a roll-up.
+// ChainHealth is one configured chain's reachability plus its PER-HOP health,
+// mirroring GroupHealth.Used for the chain card (plan §5.E): a chain no enabled
+// rule routes through is never probed — the observatory walks only reachable
+// paths — and the panel renders it "unused" rather than as a health problem.
+// A chain has no membership counters of its own: it is a fixed path, and its
+// end-to-end health is the exit probe's job. What it DOES have is hops, and
+// since the observatory now probes every hop wrapper (probeplan.go walkDetour),
+// each hop's health is on the board and is projected here so an operator can
+// see WHICH hop died instead of only that the chain did.
 type ChainHealth struct {
 	// Name is the chain's model name (config chain "chain:<name>"), what the Targets
 	// page lists and what a rule targets.
@@ -336,34 +341,218 @@ type ChainHealth struct {
 	// when the observatory is disabled or not yet configured: no badge is better
 	// than a wrong one (the same rule as GroupHealth.Used).
 	Used bool `json:"used"`
+	// Hops is the per-hop health readout, L1..Ln in wire order (see ChainHopHealth).
+	// It is EMPTY for a chain the running box never materialised: a chain no rule
+	// references is resolved lazily and never built, and a 1-hop chain without an
+	// egress entry resolves straight to its target with no wrapper — in both cases
+	// there are no "chain-<name>-h…" outbounds to project. An absent "hops" key
+	// therefore means "nothing materialised to report on", NEVER "this chain has
+	// no hops" — the model, not this projection, knows how many hops were
+	// configured.
+	Hops []ChainHopHealth `json:"hops,omitempty"`
 }
 
-// ChainHealth reports the reachability (used/unused) of every named chain, one row
-// per name, in the order given. It is the chain analogue of GroupHealth.Used: a
-// chain the observatory's used-set does not cover is reported Used=false so the
-// panel can mark it "unused" instead of running an exit test against a path nothing
-// routes through.
+// ChainHopHealth is one hop of one materialised chain, as the health board saw
+// it — a projection, like everything in this file: nothing here dials.
+//
+// A NODE hop is a single measurement: the observatory dials the hop wrapper
+// "chain-<name>-h<i>", which pulls exactly the path prefix up to and including
+// this hop, so Total=1, the counters follow the hop's own state, DelayMs and
+// AgeSeconds are its own observation, and Selected is "" (a fixed hop selects
+// nothing).
+//
+// A GROUP hop rolls up its member copies "chain-<name>-h<i>-<member>", each of
+// which the observatory probes through its own prefix of the chain. The
+// counters obey the same invariants as GroupHealth — Tested == Alive+Dead and
+// Alive+Dead+Untested == Total — so the panel needs no arithmetic of its own.
+// State summarises them: "alive" when at least one member is alive (the hop can
+// carry traffic), "dead" when at least one was tested and none is alive (a
+// positive finding of a dead hop), "untested" when nothing was tested. Selected
+// is the node NAME the wrapper currently picks; DelayMs/AgeSeconds are the
+// SELECTED member's observation, or the freshest ALIVE member's when the
+// selection has no measurement of its own — the number shown must always be a
+// measurement somebody took, never an average nobody did.
+type ChainHopHealth struct {
+	Index      int    `json:"index"`       // 1-based position on the wire, L1..Ln
+	Tag        string `json:"tag"`         // "chain-<name>-h<i>" — the wrapper actually dialled
+	Kind       string `json:"kind"`        // "node" | "group"
+	Exit       bool   `json:"exit"`        // the LAST hop: where traffic leaves to the internet
+	State      string `json:"state"`       // "alive" | "dead" | "untested"
+	DelayMs    int    `json:"delay_ms"`
+	AgeSeconds int64  `json:"age_seconds"` // -1 when unknown
+	Selected   string `json:"selected"`    // group hop: the node NAME it currently selects; "" otherwise
+	Total      int    `json:"total"`
+	Tested     int    `json:"tested"`
+	Alive      int    `json:"alive"`
+	Dead       int    `json:"dead"`
+	Untested   int    `json:"untested"`
+}
+
+// ChainHealth reports the reachability (used/unused) of every named chain plus
+// the per-hop health of each one the running box materialised, one row per
+// name, in the order given. The Used half is the chain analogue of
+// GroupHealth.Used: a chain the observatory's used-set does not cover is
+// reported Used=false so the panel can mark it "unused" instead of a health
+// readout against a path nothing routes through.
 //
 // names come from the desired-state model, NOT the running box: a chain no rule
 // references is never materialised (generate/chain.go resolveChain is lazy), so it
 // is invisible to a box-only enumeration — yet the panel lists it from the config
-// and must be able to badge it. The engine supplies the only fact a box read can
-// add here, the observatory's published used-set. Pure apart from that read; nil
-// names or a stopped engine (nil used-set) yield an empty/used-everything result.
+// and must be able to badge it. The engine supplies what only it can: the
+// observatory's published used-set, and the running box's outbound/endpoint
+// pool the hop projection reads. Still no dialling anywhere; nil names or a
+// stopped engine (nil used-set, empty pool) yield an empty/used-everything
+// result with no hops.
 func (e *Engine) ChainHealth(names []string) []ChainHealth {
 	out := make([]ChainHealth, 0, len(names))
 	used := e.observatoryUsed()
 	usedChains := usedChainNames(used)
+	pool := e.runningPool()
+	view := e.HealthView()
 	for _, name := range names {
 		name = strings.TrimSpace(name)
 		if name == "" {
 			continue
 		}
-		out = append(out, ChainHealth{Name: name, Used: used == nil || usedChains[name]})
+		out = append(out, ChainHealth{
+			Name: name,
+			Used: used == nil || usedChains[name],
+			Hops: chainHopHealthOf(pool, name, view),
+		})
 	}
 	return out
 }
 
+// runningPool snapshots the running box's outbounds AND endpoints into one
+// list. The endpoints matter: a chain hop rebuilt from a wireguard/AmneziaWG
+// node is an ENDPOINT copy, invisible in Outbounds() — the same trap
+// groupTargets documents — and a hop projection that missed it would silently
+// drop the very hop this feature exists to localise (the production chain's
+// first hop IS an AWG endpoint). Empty (never nil-unsafe) on a stopped engine.
+func (e *Engine) runningPool() []adapter.Outbound {
+	var pool []adapter.Outbound
+	inst := e.Instance()
+	if inst == nil {
+		return pool
+	}
+	if om := inst.Outbound(); om != nil {
+		pool = append(pool, om.Outbounds()...)
+	}
+	if em := inst.Endpoint(); em != nil {
+		for _, ep := range em.Endpoints() {
+			pool = append(pool, ep)
+		}
+	}
+	return pool
+}
+
+// chainHopHealthOf projects one chain's hop wrappers out of an outbound pool
+// against one health view. Pure apart from the view reads — the same
+// unit-testing contract as groupHealthOf: hand it a fake pool and a hand-built
+// store and every branch is reachable without a box.
+//
+// A pool entry belongs to chain <name> when its tag is exactly
+// "chain-<name>-h<digits>" — the hop WRAPPER the observatory dials. Member
+// copies ("chain-<name>-h<i>-<member>") are not hops themselves; they are
+// reached through the wrapper's own member list (adapter.OutboundGroup.All), so
+// the roll-up sees exactly what the wrapper can select, in its order.
+func chainHopHealthOf(pool []adapter.Outbound, name string, view HealthView) []ChainHopHealth {
+	prefix := "chain-" + name + "-h"
+	var hops []ChainHopHealth
+	for _, ob := range pool {
+		tag := ob.Tag()
+		rest, ok := strings.CutPrefix(tag, prefix)
+		if !ok {
+			continue
+		}
+		idx, ok := parseAllDigits(rest)
+		if !ok {
+			continue // a member copy, or another chain sharing the prefix
+		}
+		if g, isGroup := ob.(adapter.OutboundGroup); isGroup {
+			hops = append(hops, chainGroupHop(g, name, idx, view))
+		} else {
+			hops = append(hops, chainNodeHop(tag, idx, view))
+		}
+	}
+	sort.Slice(hops, func(i, j int) bool { return hops[i].Index < hops[j].Index })
+	if len(hops) > 0 {
+		// The largest index is the exit — the wrapper whose probe leaves to the
+		// internet. Marked after sorting so the flag cannot depend on pool order.
+		hops[len(hops)-1].Exit = true
+	}
+	return hops
+}
+
+// chainNodeHop is the one-measurement hop: the wrapper itself was dialled by
+// the observatory, so its own board state IS the hop's health and the counters
+// degenerate to whichever bucket that state fills.
+func chainNodeHop(tag string, idx int, view HealthView) ChainHopHealth {
+	hop := ChainHopHealth{Index: idx, Tag: tag, Kind: "node", Total: 1}
+	hop.State, hop.DelayMs, hop.AgeSeconds = view.State(tag)
+	switch hop.State {
+	case HealthAlive:
+		hop.Alive = 1
+	case HealthDead:
+		hop.Dead = 1
+	default:
+		hop.Untested = 1
+	}
+	hop.Tested = hop.Alive + hop.Dead
+	return hop
+}
+
+// chainGroupHop rolls a group hop up over its member copies. The counters carry
+// the GroupHealth invariants; the summary State answers the only question a hop
+// row asks — "can this hop carry the chain": alive while anything answers,
+// dead only on a positive all-tested-dead finding, untested when nothing is
+// known (never dead-by-absence, the same honesty rule as everywhere else).
+func chainGroupHop(g adapter.OutboundGroup, chain string, idx int, view HealthView) ChainHopHealth {
+	hop := ChainHopHealth{Index: idx, Tag: g.Tag(), Kind: "group", AgeSeconds: -1}
+	selected := g.Now()
+	if selected != "" {
+		hop.Selected = chainMemberName(selected, chain)
+	}
+
+	// The number a hop row shows must be a real observation: the selected
+	// member's when it has one, else the freshest alive member's.
+	freshDelay, freshAge := 0, int64(-1)
+	selDelay, selAge := 0, int64(-1)
+	for _, tag := range g.All() {
+		state, delayMs, age := view.State(tag)
+		switch state {
+		case HealthAlive:
+			hop.Alive++
+			if age >= 0 && (freshAge < 0 || age < freshAge) {
+				freshDelay, freshAge = delayMs, age
+			}
+		case HealthDead:
+			hop.Dead++
+		default:
+			hop.Untested++
+		}
+		hop.Total++
+		if tag == selected && age >= 0 {
+			selDelay, selAge = delayMs, age
+		}
+	}
+	hop.Tested = hop.Alive + hop.Dead
+	switch {
+	case hop.Alive > 0:
+		hop.State = HealthAlive
+	case hop.Tested > 0:
+		hop.State = HealthDead
+	default:
+		hop.State = HealthUntested
+	}
+	if selAge >= 0 {
+		hop.DelayMs, hop.AgeSeconds = selDelay, selAge
+	} else {
+		hop.DelayMs, hop.AgeSeconds = freshDelay, freshAge
+	}
+	return hop
+}
+
 // usedChainNames recovers the set of chain NAMES the observatory's used-set covers.
 //
 // The used-set is keyed by outbound TAG (the generator's schema, materialised in

shater/engine/grouphealth_test.go

@@ -309,8 +309,145 @@ func TestChainHealthNilUsedSet(t *testing.T) {
 		t.Fatalf("ChainHealth = %+v, want %+v (nil used-set ⇒ all used, blanks dropped)", got, want)
 	}
 	for i, w := range want {
-		if got[i] != w {
+		if got[i].Name != w.Name || got[i].Used != w.Used {
 			t.Errorf("ChainHealth[%d] = %+v, want %+v", i, got[i], w)
 		}
+		// A stopped engine materialised nothing: hops must be empty, and the
+		// contract says empty means "nothing materialised", not "no hops".
+		if len(got[i].Hops) != 0 {
+			t.Errorf("ChainHealth[%d].Hops = %+v, want empty on a stopped engine", i, got[i].Hops)
+		}
+	}
+}
+
+// TestChainHopHealthProjection drives the per-hop readout over a 3-hop chain —
+// node hop L1, group hop L2, group hop L3 (the exit) — with exactly ONE hop's
+// members dead. The dead state must land on THAT hop's index and nowhere else,
+// the counters must obey the GroupHealth invariants on every hop, and the exit
+// flag must sit on the largest index. This is the "which hop died" question the
+// whole hop surface exists to answer.
+func TestChainHopHealthProjection(t *testing.T) {
+	hist := urltest.NewHistoryStorage()
+	now := time.Now()
+
+	// L1 (node hop wrapper): alive — the prefix up to hop 1 works.
+	hist.StoreURLTestHistory("chain-c-h1", &adapter.URLTestHistory{LastOK: now.Add(-5 * time.Second), Delay: 40})
+	// L2 (group hop): BOTH member copies dead — this is the hop that died.
+	hist.StoreURLTestHistory("chain-c-h2-x", &adapter.URLTestHistory{LastFail: now.Add(-3 * time.Second)})
+	hist.StoreURLTestHistory("chain-c-h2-y", &adapter.URLTestHistory{LastFail: now.Add(-2 * time.Second)})
+	// L3 (group hop, exit): one member alive, one never measured.
+	hist.StoreURLTestHistory("chain-c-h3-a", &adapter.URLTestHistory{LastOK: now.Add(-7 * time.Second), Delay: 200})
+	// chain-c-h3-b: nothing at all.
+
+	pool := []adapter.Outbound{
+		&depOutbound{failingOutbound{tag: "chain-c-h1"}, nil},
+		&depGroup{fakeGroup{
+			tag: "chain-c-h2", kind: C.TypeSelector,
+			all: []string{"chain-c-h2-x", "chain-c-h2-y"},
+			now: "chain-c-h2-x",
+		}, []string{"chain-c-h2-x", "chain-c-h2-y"}},
+		&depGroup{fakeGroup{
+			tag: "chain-c-h3", kind: C.TypeSelector,
+			all: []string{"chain-c-h3-a", "chain-c-h3-b"},
+			now: "chain-c-h3-a",
+		}, []string{"chain-c-h3-a", "chain-c-h3-b"}},
+		// Noise the projection must ignore: a member copy is not a hop, another
+		// chain's wrapper is not this chain's.
+		&depOutbound{failingOutbound{tag: "chain-c-h2-x"}, nil},
+		&depOutbound{failingOutbound{tag: "chain-other-h1"}, nil},
+	}
+
+	hops := chainHopHealthOf(pool, "c", newHealthView(hist, healthTTLFloor, now))
+	if len(hops) != 3 {
+		t.Fatalf("got %d hops, want 3: %+v", len(hops), hops)
+	}
+
+	// Ordered by index, exit on the largest.
+	for i, wantIdx := range []int{1, 2, 3} {
+		if hops[i].Index != wantIdx {
+			t.Fatalf("hops out of order: %+v", hops)
+		}
+		if got, want := hops[i].Exit, wantIdx == 3; got != want {
+			t.Errorf("hop %d Exit = %v, want %v", wantIdx, got, want)
+		}
+	}
+
+	h1, h2, h3 := hops[0], hops[1], hops[2]
+
+	// L1: one measurement, its own numbers, no selection.
+	if h1.Kind != "node" || h1.State != HealthAlive || h1.Total != 1 || h1.Alive != 1 ||
+		h1.DelayMs != 40 || h1.AgeSeconds != 5 || h1.Selected != "" {
+		t.Errorf("h1 = %+v, want an alive node hop with its own 40ms/5s and no selection", h1)
+	}
+
+	// L2: the dead hop. Both members tested, none alive => a POSITIVE dead
+	// finding on exactly this index. The selected member (x) is dead, and a
+	// dead observation carries delay 0 with the failure's age.
+	if h2.Kind != "group" || h2.State != HealthDead {
+		t.Fatalf("h2 = %+v, want the DEAD group hop — this is the answer to 'which hop died'", h2)
+	}
+	if h2.Total != 2 || h2.Tested != 2 || h2.Alive != 0 || h2.Dead != 2 || h2.Untested != 0 {
+		t.Errorf("h2 counters = %+v, want 2 tested / 2 dead", h2)
+	}
+	if h2.Selected != "x" {
+		t.Errorf("h2 Selected = %q, want the node NAME x", h2.Selected)
+	}
+	if h2.DelayMs != 0 || h2.AgeSeconds != 3 {
+		t.Errorf("h2 delay/age = %d/%d, want 0/3 (the selected member's failure observation)", h2.DelayMs, h2.AgeSeconds)
+	}
+
+	// L3: alive (one member answers), one member honestly untested — never
+	// folded into dead. The selected member is the alive one, so its numbers show.
+	if h3.Kind != "group" || h3.State != HealthAlive {
+		t.Fatalf("h3 = %+v, want an alive exit hop", h3)
+	}
+	if h3.Total != 2 || h3.Tested != 1 || h3.Alive != 1 || h3.Dead != 0 || h3.Untested != 1 {
+		t.Errorf("h3 counters = %+v, want 1 alive / 1 untested", h3)
+	}
+	if h3.Selected != "a" || h3.DelayMs != 200 || h3.AgeSeconds != 7 {
+		t.Errorf("h3 = %+v, want selection a with its 200ms/7s", h3)
+	}
+
+	// The invariants, on every hop, exactly as GroupHealth promises.
+	for _, h := range hops {
+		if h.Tested != h.Alive+h.Dead {
+			t.Errorf("hop %d: Tested = %d, want alive+dead = %d", h.Index, h.Tested, h.Alive+h.Dead)
+		}
+		if h.Alive+h.Dead+h.Untested != h.Total {
+			t.Errorf("hop %d: alive+dead+untested = %d, want total = %d",
+				h.Index, h.Alive+h.Dead+h.Untested, h.Total)
+		}
+	}
+}
+
+// A group hop whose SELECTION has no measurement shows the freshest ALIVE
+// member's numbers instead — the shown number must always be an observation
+// somebody took.
+func TestChainHopSelectionWithoutMeasurementFallsBack(t *testing.T) {
+	hist := urltest.NewHistoryStorage()
+	now := time.Now()
+	hist.StoreURLTestHistory("chain-c-h1-a", &adapter.URLTestHistory{LastOK: now.Add(-30 * time.Second), Delay: 90})
+	hist.StoreURLTestHistory("chain-c-h1-b", &adapter.URLTestHistory{LastOK: now.Add(-4 * time.Second), Delay: 150})
+	// The selection points at c, which has no observation at all.
+	pool := []adapter.Outbound{
+		&depGroup{fakeGroup{
+			tag: "chain-c-h1", kind: C.TypeSelector,
+			all: []string{"chain-c-h1-a", "chain-c-h1-b", "chain-c-h1-c"},
+			now: "chain-c-h1-c",
+		}, nil},
+	}
+	hops := chainHopHealthOf(pool, "c", newHealthView(hist, healthTTLFloor, now))
+	if len(hops) != 1 {
+		t.Fatalf("got %d hops, want 1", len(hops))
+	}
+	h := hops[0]
+	if h.Selected != "c" {
+		t.Errorf("Selected = %q, want c (the selection is reported even unmeasured)", h.Selected)
+	}
+	if h.DelayMs != 150 || h.AgeSeconds != 4 {
+		t.Errorf("delay/age = %d/%d, want the freshest ALIVE member's 150/4", h.DelayMs, h.AgeSeconds)
+	}
+	if h.State != HealthAlive || h.Alive != 2 || h.Untested != 1 {
+		t.Errorf("hop = %+v, want alive with 2 alive / 1 untested", h)
 	}
 }

shater/engine/grouptest.go

@@ -12,33 +12,46 @@ import (
 	"time"
 
 	"github.com/sagernet/sing-box/adapter"
-	"github.com/sagernet/sing-box/common/urltest"
 	C "github.com/sagernet/sing-box/constant"
 )
 
-// Exit test — "what am I actually going out through, and how fast" (F2, plan §5.E).
+// Group/chain test — "ask the observatory to refresh, then report what it
+// measured, plus the exit address" (F2, plan §5.E; reworked under the one-probe
+// rule).
 //
-// # Why this is not just another node probe
+// # This file no longer measures latency. On purpose.
 //
-// The observatory (observatory.go) answers "which nodes are alive". It cannot
-// answer the question an operator actually asks after switching a group: *through
-// which address am I leaving the country right now*. A group is an indirection —
-// a selector or urltest over many members — so the delay of the group and the
-// public address it exits from are properties of the CURRENT selection, not of
-// any node the panel can point at. A CHAIN is the same question over a longer
-// path: its exit wrapper "chain-<name>-hN" tunnels through every hop, so dialling
-// it measures the whole L1..Ln path end to end.
+// It used to: one urltest.URLTest per target, dialled right here. That was the
+// defect, not a feature. The observatory (observatory.go) probes every path the
+// routing rules actually use — per-chain member copies, egress-bound group
+// copies, chain exits — and it is the ONLY thing allowed to dial for health.
+// A second dial path from this file measured the WRONG thing: pressing "Test"
+// dialled the base groups directly from the router over the default WAN, a path
+// no rule routes through, and (worse) URLTest's DialContext touched the group,
+// arming its own 30-minute probing ticker, which kept writing those false
+// direct measurements under the members' base tags. For a node that is blocked
+// on the direct WAN and alive only behind a tunnel, that reading is not stale —
+// it is FALSE, and it poisoned selection and the panel alike.
 //
-// So one test per target (group or chain) measures two things:
+// So a manual test is now a READ with a refresh request in front of it:
 //
-//	delay_ms  — reusing urltest.URLTest, the SAME primitive the observatory uses
-//	            for a node. There is deliberately no second latency mechanism
-//	            here: the target is just an outbound, and probing it exercises
-//	            exactly the path traffic will take.
-//	exit_ip   — a real HTTP request THROUGH the target to a service that echoes
-//	            the client address back. Nothing else can produce this number: the
-//	            router cannot know its own public address, and the proxy protocol
-//	            does not report it.
+//	delay_ms / ok — the observatory's OWN measurement of the target's dial
+//	                path. TestGroups rewinds the observatory (RefreshObservatory)
+//	                so the numbers are fresh — taken AFTER the button press —
+//	                then each target waits for an observation newer than the
+//	                run's start instant and reports it verbatim. tested_unix is
+//	                the instant that observation was taken, not "now".
+//	exit_ip       — a real HTTP request THROUGH the target to a service that
+//	                echoes the client address back. This is the ONE connection
+//	                this file still opens, and it stays because no probe can
+//	                answer it: the router cannot know its own public address,
+//	                and the proxy protocol does not report it. It is NOT a
+//	                second health probe — it travels the target's own routed
+//	                path (the same outbound object the rules dial), and it is
+//	                only ever issued for a target that (a) the observatory's
+//	                used-set covers and (b) just resolved ALIVE. A target
+//	                outside the rules, or one whose path is down, gets an empty
+//	                exit_ip — never a connection.
 //
 // # The direct-egress trap
 //
@@ -51,16 +64,15 @@ import (
 // exit_ip instead of somebody else's number.
 
 const (
-	// groupTestDelayTimeout bounds the latency probe of one group.
-	groupTestDelayTimeout = 5 * time.Second
 	// groupTestExitTimeout bounds the whole exit-address lookup for one group
 	// (connect through the tunnel + TLS + response). Short on purpose: this runs
 	// while a human waits on a panel button, and a slow answer is worth less than a
 	// prompt "could not determine".
 	groupTestExitTimeout = 6 * time.Second
-	// groupTestConcurrency bounds how many groups are tested in parallel. Small:
-	// every one of these opens a real tunnelled connection, and a router with a
-	// handful of groups is the normal case.
+	// groupTestConcurrency bounds how many EXIT-ADDRESS lookups run in parallel.
+	// Small: each one opens a real tunnelled connection, and a router with a
+	// handful of groups is the normal case. The board polling itself is not
+	// bounded by this — it is sleep-and-read, no I/O.
 	groupTestConcurrency = 4
 	// exitBodyLimit caps what is read from the exit-address service. These responses
 	// are a few hundred bytes; anything larger is a hijacked/captive-portal answer
@@ -68,19 +80,58 @@ const (
 	exitBodyLimit = 4 << 10
 )
 
+// groupTestWaitDeadline / groupTestPollEvery pace the wait for the observatory:
+// each covered target polls the health board once a second until its measured
+// tag carries an observation newer than the run's start, giving up after the
+// deadline. 120s covers a forced pass of a large plan (batches chain
+// back-to-back during a forced pass, see observatoryTickOnce) with room for the
+// probe timeouts of a mostly-dead population. Package variables, not constants,
+// for exactly one reason: the timeout tests must not take two minutes.
+var (
+	groupTestWaitDeadline = 120 * time.Second
+	groupTestPollEvery    = time.Second
+)
+
+// The fixed result texts for the targets that are never (or not yet) measured.
+// They are contract, not decoration — the panel shows them verbatim.
+const (
+	// groupTestErrNotRouted: the target is outside the observatory's used-set,
+	// so no rule routes through it and nothing measures it. Dialling it anyway
+	// would recreate the false direct measurement this rework removed.
+	groupTestErrNotRouted = "not routed by any enabled rule, so nothing measures it — the observatory only probes paths the rules use"
+	// groupTestErrNotReached: the deadline passed without a fresh observation.
+	groupTestErrNotReached = "the observatory has not reached this target yet — it refreshes on the global probe interval"
+	// groupTestErrProbingOff: the observatory is disabled (the GroupHealth
+	// master switch), so a refresh request has nothing to wake and waiting for
+	// the deadline would just delay the same answer by two minutes.
+	groupTestErrProbingOff = "background probing is disabled, so there is nothing to measure this target with"
+	// groupTestErrPathDead: the fresh observation exists and it is a FAILURE —
+	// the observatory probed the target's path after the button press and the
+	// path did not answer. An honest negative, not a missing measurement.
+	groupTestErrPathDead = "the observatory's probe through this path failed"
+)
+
 // GroupTestResult is one target's test outcome — a group's or a chain's (Group
 // then carries the chain's model name). The JSON tags are the panel contract —
-// see the shater API docs for /api/groups/test.
+// see the shater API docs for /api/groups/test. The field names and types are
+// FROZEN; what changed in the rework is where the numbers come from.
+//
+// DelayMs and OK are a READ of the observatory's measurement of the target's
+// dial path — not a fresh dial performed by this file. OK is true exactly when
+// the health board's state for the measured tag is alive; DelayMs is that
+// observation's RTT; TestedUnix is the instant the OBSERVATION was taken (now
+// minus its age), so a result honestly says how old its number is instead of
+// stamping the poll time over it.
 //
 // Selected is the group's current pick (OutboundGroup.Now()); for a chain it is
 // the node NAME the chain's last group hop currently selects, "" when the chain
 // has no group hop (a fixed path selects nothing).
 //
-// OK reports whether the LATENCY measurement succeeded, which is the test's primary
-// question. A failed exit-address lookup deliberately does NOT clear it: knowing the
-// target is up and fast is useful on its own, and a probe service being unreachable
-// says nothing about the tunnel. In that case OK stays true and ExitIP is empty —
-// "not determined", never a guess and never somebody else's address.
+// A failed exit-address lookup deliberately does NOT clear OK: knowing the
+// target is up and fast is useful on its own, and a probe service being
+// unreachable says nothing about the tunnel. In that case OK stays true and
+// ExitIP is empty — "not determined", never a guess and never somebody else's
+// address.
 type GroupTestResult struct {
 	Group       string `json:"group"`
 	Selected    string `json:"selected"`
@@ -259,19 +310,30 @@ func parseAllDigits(s string) (int, bool) {
 // and chains (empty/nil = every group and every chain in the running box),
 // returning started=false when a run is already in flight.
 //
-// It is a SINGLETON: a second request while a run is in flight is refused rather
-// than queued or run in parallel, because these runs open real tunnelled
-// connections and a panel that double-fires a button must not multiply the load
-// on the uplink. The observatory checks the same guard and skips its tick while
-// a run is in flight (observatory.go), so a manual test never competes with
-// background probing for the uplink.
+// It does NOT probe. It records the run's start instant, asks the observatory
+// for an out-of-turn full pass (RefreshObservatory), and then each target waits
+// for the health board to carry an observation NEWER than that instant on the
+// tag the observatory actually measures for it — see testOneTarget. The only
+// connection a run may still open is the exit-address lookup, and only for a
+// target that resolved alive on a rule-routed path.
 //
-// probeURL is the latency-probe URL; "" falls back to urltest's gstatic default.
+// It is a SINGLETON: a second request while a run is in flight is refused
+// rather than queued, because two overlapping runs would each rewind the
+// observatory's cursor and neither pass would ever complete — and the panel's
+// progress contract assumes one run's counters at a time anyway.
 //
-// Apply-swap safety: the target outbounds are snapshotted up front, so a config swap
-// mid-run cannot change what is being tested. A target torn down mid-run simply fails
-// its probe and is reported not-ok.
+// probeURL is accepted and IGNORED. The probe URL is a global observatory
+// setting now (ObservatoryConfig.ProbeURL, set at apply time); a per-run URL
+// would mean this run measures something different from what the board holds,
+// which is exactly the two-instruments split the rework removed. The parameter
+// stays so the callers (shater/apply, shater/panel) keep compiling and the
+// control-plane API shape does not churn.
+//
+// Apply-swap safety: the target outbounds are snapshotted up front, so a config
+// swap mid-run cannot change what is being tested. A target torn down mid-run
+// simply never receives a fresh observation and resolves on the deadline.
 func (e *Engine) TestGroups(names []string, probeURL string) (started bool) {
+	_ = probeURL // ignored — see the doc comment above
 	if !e.groupTestRunning.CompareAndSwap(false, true) {
 		return false
 	}
@@ -285,6 +347,13 @@ func (e *Engine) TestGroups(names []string, probeURL string) (started bool) {
 	// GroupTestStatus for why the scope is a set of names.
 	e.setGroupTestScope(scopeOf(targets, missing))
 
+	// The freshness watermark: only an observation taken AFTER this instant may
+	// answer this run. Recorded BEFORE the refresh request so a probe that lands
+	// between the two can never be missed, only double-counted as fresh — the
+	// harmless direction.
+	t0 := time.Now()
+	e.RefreshObservatory()
+
 	go func() {
 		defer e.groupTestRunning.Store(false)
 
@@ -303,15 +372,22 @@ func (e *Engine) TestGroups(names []string, probeURL string) (started bool) {
 		e.setGroupTestResults(results)
 		e.groupTestDone.Add(int64(len(missing)))
 
+		// The used-set and the enabled bit are snapshotted once for the whole
+		// run: they only change on an apply, and a run that straddles an apply
+		// is already best-effort (see the swap-safety note above).
+		used := e.observatoryUsed()
+		obsEnabled, _, _ := e.ObservatoryStatus()
+
+		// One goroutine per target: they spend their life sleeping on the board
+		// poll, so there is nothing to bound — the semaphore below bounds the
+		// exit-address lookups, the only real connections left in a run.
 		sem := make(chan struct{}, groupTestConcurrency)
 		var wg sync.WaitGroup
 		for i, tgt := range targets {
 			wg.Add(1)
-			sem <- struct{}{}
 			go func(i int, tgt groupTestTarget) {
 				defer wg.Done()
-				defer func() { <-sem }()
-				res := e.testOneTarget(tgt, probeURL)
+				res := e.testOneTarget(tgt, used, obsEnabled, t0, sem)
 				e.storeGroupTestResult(i, res)
 				e.groupTestDone.Add(1)
 			}(i, tgt)
@@ -393,29 +469,122 @@ func (e *Engine) groupTargets(names []string) (targets []groupTestTarget, missin
 	return targets, missing
 }
 
-// testOneTarget measures one target: what it currently selects, the latency
-// through it, and the public address it exits from. For a chain the dialled
-// outbound is the exit wrapper, so the delay and the exit address are end-to-end
-// properties of the whole L1..Ln path.
-func (e *Engine) testOneTarget(t groupTestTarget, probeURL string) GroupTestResult {
+// testOneTarget resolves one target WITHOUT probing it: it decides whether the
+// observatory measures this target at all, and if so waits for a fresh
+// observation and reports it. The three ways out, in order:
+//
+//  1. the used-set does not cover the target — no enabled rule routes through
+//     it, so nothing measures it and nothing SHOULD: resolved immediately with
+//     groupTestErrNotRouted, never dialled, empty exit address. A nil used-set
+//     means "unknown" (observatory not yet configured) and is treated as
+//     covered — no refusal is better than a wrong one;
+//  2. the observatory is disabled — the refresh request went nowhere, so the
+//     wait below could only ever end on its deadline: resolved immediately
+//     with groupTestErrProbingOff instead of stalling the panel for two
+//     minutes to say the same thing;
+//  3. covered and enabled — poll the health board once a second until the
+//     MEASURED TAG carries an observation newer than since, then report that
+//     observation verbatim (readFreshObservation). On the deadline:
+//     groupTestErrNotReached.
+//
+// The exit-address lookup runs ONLY on the alive path of (3) — a rule-routed
+// target whose path just answered a probe — and through sem, so a run never
+// opens more than groupTestConcurrency tunnelled connections at once.
+func (e *Engine) testOneTarget(t groupTestTarget, used map[string]bool, obsEnabled bool, since time.Time, sem chan struct{}) GroupTestResult {
 	res := GroupTestResult{Group: t.name, TestedUnix: time.Now().Unix()}
 	if t.sel != nil {
 		res.Selected = t.sel()
 	}
 
-	ctx, cancel := context.WithTimeout(context.Background(), groupTestDelayTimeout)
-	delay, err := urltest.URLTest(ctx, probeURL, t.ob)
-	cancel()
-	if err != nil {
-		res.Error = err.Error()
+	if used != nil && !used[t.ob.Tag()] {
+		res.Error = groupTestErrNotRouted
+		return res
+	}
+	if !obsEnabled {
+		res.Error = groupTestErrProbingOff
 		return res
 	}
-	res.OK = true
-	res.DelayMs = int(delay)
 
-	// The exit address is best-effort by design: see GroupTestResult.OK.
-	res.ExitIP, res.ExitCountry = e.exitAddress(t.ob)
-	return res
+	deadline := time.Now().Add(groupTestWaitDeadline)
+	for {
+		if e.readFreshObservation(&res, t, since) {
+			if res.OK {
+				// The exit address is best-effort by design: see GroupTestResult.
+				sem <- struct{}{}
+				res.ExitIP, res.ExitCountry = e.exitAddress(t.ob)
+				<-sem
+			}
+			return res
+		}
+		if !time.Now().Before(deadline) {
+			res.Error = groupTestErrNotReached
+			return res
+		}
+		time.Sleep(groupTestPollEvery)
+	}
+}
+
+// measuredTagOf is the tag the observatory actually probes for this target's
+// dial path — the tag whose board entry answers "how is this target doing":
+//
+//   - a GROUP target dials whatever it currently selects, so the group's health
+//     IS its selection's health: OutboundGroup.Now(). For an egress-bound group
+//     that is a per-group copy tag, which is exactly what the plan probes;
+//   - a CHAIN whose exit wrapper is a PLAIN outbound is probed end-to-end under
+//     that wrapper tag (probeplan.go: a chain exit is its own measurement);
+//   - a CHAIN whose exit wrapper is a GROUP (the last hop is a group) has its
+//     member copies probed instead of the wrapper, so the wrapper's Now() — the
+//     member copy the chain currently dials through — is the measured tag. The
+//     wrapper here IS the last group hop, so this is chainLastGroupHop's Now()
+//     without a second lookup;
+//   - fallback: the target's own tag, for a group that has not selected yet
+//     (cold selector mid-swap). Its board entry is almost certainly empty, and
+//     the caller then honestly reports "not reached" rather than inventing one.
+func measuredTagOf(t groupTestTarget) string {
+	if g, ok := t.ob.(adapter.OutboundGroup); ok {
+		if now := g.Now(); now != "" {
+			return now
+		}
+	}
+	return t.ob.Tag()
+}
+
+// readFreshObservation reads the board once: if the target's measured tag holds
+// an observation taken at or after since, it is written into res (state, delay,
+// the observation's own timestamp, and the current selection so Selected and
+// the measurement describe the same pick) and true is returned. Otherwise res
+// is left for the next poll.
+//
+// Precision note: the board reports ages in whole seconds, so "at or after
+// since" is accurate to one second — an observation taken up to a second
+// BEFORE the refresh can slip through as fresh. That is the acceptable
+// direction: it is still a real measurement of the same path, at most a second
+// older than requested; the strict direction (discarding genuinely fresh
+// observations) would make every run one probe interval slower for nothing.
+func (e *Engine) readFreshObservation(res *GroupTestResult, t groupTestTarget, since time.Time) bool {
+	// Re-read the selection at every poll: a forced observatory pass is exactly
+	// the kind of event that makes a urltest group switch members, and the
+	// measurement below is taken against the CURRENT pick.
+	if t.sel != nil {
+		res.Selected = t.sel()
+	}
+	tag := measuredTagOf(t)
+	now := time.Now()
+	state, delayMs, age := e.HealthView().State(tag)
+	if age < 0 {
+		return false // no observation at all (untested)
+	}
+	observedAt := now.Add(-time.Duration(age) * time.Second)
+	if observedAt.Before(since) {
+		return false // an old reading; the refresh has not reached this tag yet
+	}
+	res.OK = state == HealthAlive
+	res.DelayMs = delayMs
+	res.TestedUnix = observedAt.Unix()
+	if !res.OK {
+		res.Error = groupTestErrPathDead
+	}
+	return true
 }
 
 // exitProbe is one exit-address service: a URL and the parser for its body.

shater/engine/grouptest_test.go

@@ -3,6 +3,7 @@ package engine
 import (
 	"context"
 	"crypto/tls"
+	"fmt"
 	"net"
 	"net/http"
 	"net/http/httptest"
@@ -143,10 +144,10 @@ func TestChainMemberName(t *testing.T) {
 	}
 }
 
-// dialableOutbound routes every dial to a fixed local address — a stand-in for a
-// chain exit whose whole path is up. urltest.URLTest dials the probe URL's host
-// through the outbound, so pointing every dial at a local HTTP server makes the
-// latency probe succeed without any network.
+// dialableOutbound routes every dial to a fixed local address — the sink the
+// exit-address lookup lands in during tests. Since the rework nothing else in
+// this file dials at all; the local server refuses to be an exit-address
+// service (wrong status, no TLS), so the lookup honestly comes back empty.
 type dialableOutbound struct {
 	failingOutbound
 	addr string
@@ -158,20 +159,42 @@ func (d *dialableOutbound) DialContext(ctx context.Context, network string, _ M.
 	return (&net.Dialer{}).DialContext(ctx, network, d.addr)
 }
 
-// TestChainExitTestMeasuresEndToEnd is the §6-S4 acceptance path for chains: the
-// exit test dials the chain's EXIT TAG, returns a measured delay, carries the
-// chain's model name (not the wrapper tag) as the result's Group, and reports the
-// last group hop's pick as Selected. The exit address is measured through the
-// same outbound (unreachable from a test => empty, never a guess).
-func TestChainExitTestMeasuresEndToEnd(t *testing.T) {
-	probe := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+// countingOutbound counts every DialContext/ListenPacket. It is the tripwire of
+// the rework: a target the observatory does not cover must NEVER be dialled,
+// and the counter is the proof.
+type countingOutbound struct {
+	failingOutbound
+	dials int
+}
+
+func (c *countingOutbound) DialContext(ctx context.Context, network string, dst M.Socksaddr) (net.Conn, error) {
+	c.dials++
+	return nil, fmt.Errorf("dial refused (test)")
+}
+
+func (c *countingOutbound) ListenPacket(ctx context.Context, dst M.Socksaddr) (net.PacketConn, error) {
+	c.dials++
+	return nil, fmt.Errorf("dial refused (test)")
+}
+
+// groupTestSem is a fresh exit-lookup semaphore for direct testOneTarget calls.
+func groupTestSem() chan struct{} { return make(chan struct{}, groupTestConcurrency) }
+
+// TestChainTestReportsObservatoryMeasurement is the §6-S4 acceptance path for
+// chains under the one-probe rule: the manual test does NOT dial the exit — it
+// reads the observatory's board entry for the exit tag, reports its delay and
+// ITS timestamp, carries the chain's model name as Group and the last group
+// hop's pick as Selected. The exit-address lookup still travels the exit
+// outbound (unreachable from a test => empty, never a guess).
+func TestChainTestReportsObservatoryMeasurement(t *testing.T) {
+	sink := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		w.WriteHeader(http.StatusNoContent)
 	}))
-	defer probe.Close()
+	defer sink.Close()
 
 	exit := &dialableOutbound{
 		failingOutbound: failingOutbound{tag: "chain-x-h2"},
-		addr:            probe.Listener.Addr().String(),
+		addr:            sink.Listener.Addr().String(),
 		deps:            []string{"chain-x-h1"},
 	}
 	pool := []adapter.Outbound{
@@ -187,14 +210,31 @@ func TestChainExitTestMeasuresEndToEnd(t *testing.T) {
 	if len(targets) != 1 || targets[0].ob.Tag() != "chain-x-h2" {
 		t.Fatalf("targets = %+v, want the chain dialled via its exit tag", targets)
 	}
+	if got := measuredTagOf(targets[0]); got != "chain-x-h2" {
+		t.Fatalf("measuredTagOf = %q, want the plain exit wrapper itself", got)
+	}
 
 	e := New()
-	res := e.testOneTarget(targets[0], probe.URL)
+	// The observatory measured the exit 10 seconds ago, AFTER the run's start
+	// instant below: that observation — delay and timestamp both — is the answer.
+	observedAt := time.Now().Add(-10 * time.Second)
+	e.URLTestHistory().StoreURLTestHistory("chain-x-h2", &adapter.URLTestHistory{LastOK: observedAt, Delay: 77})
+	since := time.Now().Add(-time.Minute)
+
+	res := e.testOneTarget(targets[0], map[string]bool{"chain-x-h2": true}, true, since, groupTestSem())
 	if res.Group != "x" {
 		t.Errorf("Group = %q, want the chain's model name x", res.Group)
 	}
 	if !res.OK || res.Error != "" {
-		t.Fatalf("result = %+v, want a successful measurement through the exit tag (a local roundtrip may legitimately read 0ms)", res)
+		t.Fatalf("result = %+v, want the board's alive observation reported", res)
+	}
+	if res.DelayMs != 77 {
+		t.Errorf("DelayMs = %d, want the observatory's 77 — this file measures nothing itself", res.DelayMs)
+	}
+	// tested_unix is the OBSERVATION's instant (whole-second precision), never
+	// the poll's.
+	if got, want := res.TestedUnix, observedAt.Unix(); got < want-1 || got > want+1 {
+		t.Errorf("TestedUnix = %d, want the observation's instant ~%d", got, want)
 	}
 	if res.Selected != "relay" {
 		t.Errorf("Selected = %q, want the last group hop's pick relay", res.Selected)
@@ -204,6 +244,93 @@ func TestChainExitTestMeasuresEndToEnd(t *testing.T) {
 	}
 }
 
+// TestTestOneTargetUnroutedNeverDialled is the rework's core promise: a target
+// outside the observatory's used-set is resolved immediately with the
+// documented explanation and ZERO dials — no latency probe, no exit-address
+// lookup, nothing. Dialling it would manufacture exactly the false direct
+// measurement the rework removed.
+func TestTestOneTargetUnroutedNeverDialled(t *testing.T) {
+	e := New()
+	ob := &countingOutbound{failingOutbound: failingOutbound{tag: "idle"}}
+	tgt := groupTestTarget{name: "idle", ob: ob}
+
+	res := e.testOneTarget(tgt, map[string]bool{"something-else": true}, true, time.Now(), groupTestSem())
+	if ob.dials != 0 {
+		t.Fatalf("an unrouted target was dialled %d time(s); it must never be", ob.dials)
+	}
+	if res.OK {
+		t.Fatal("an unrouted target reported ok=true")
+	}
+	if res.Error != groupTestErrNotRouted {
+		t.Fatalf("Error = %q, want the not-routed explanation", res.Error)
+	}
+	if res.ExitIP != "" || res.ExitCountry != "" {
+		t.Fatalf("an unrouted target must carry no exit address: %+v", res)
+	}
+}
+
+// A disabled observatory resolves covered targets immediately too: the refresh
+// request went nowhere, so waiting out the deadline would only delay the same
+// honest answer — and still, nothing is dialled.
+func TestTestOneTargetProbingOffNeverDialled(t *testing.T) {
+	e := New()
+	ob := &countingOutbound{failingOutbound: failingOutbound{tag: "auto"}}
+	tgt := groupTestTarget{name: "auto", ob: ob}
+
+	res := e.testOneTarget(tgt, nil, false, time.Now(), groupTestSem())
+	if ob.dials != 0 {
+		t.Fatalf("target dialled %d time(s) with probing off; want 0", ob.dials)
+	}
+	if res.OK || res.Error != groupTestErrProbingOff {
+		t.Fatalf("result = %+v, want ok=false with the probing-off explanation", res)
+	}
+}
+
+// The deadline path: a covered target whose measured tag never receives a fresh
+// observation resolves with the not-reached explanation — and, again, without a
+// single dial of its own.
+func TestTestOneTargetDeadlineWithoutObservation(t *testing.T) {
+	// Shrink the wait so the test answers in milliseconds; restore afterwards.
+	oldDeadline, oldPoll := groupTestWaitDeadline, groupTestPollEvery
+	groupTestWaitDeadline, groupTestPollEvery = 30*time.Millisecond, 5*time.Millisecond
+	defer func() { groupTestWaitDeadline, groupTestPollEvery = oldDeadline, oldPoll }()
+
+	e := New()
+	ob := &countingOutbound{failingOutbound: failingOutbound{tag: "auto"}}
+	tgt := groupTestTarget{name: "auto", ob: ob}
+
+	res := e.testOneTarget(tgt, map[string]bool{"auto": true}, true, time.Now(), groupTestSem())
+	if ob.dials != 0 {
+		t.Fatalf("target dialled %d time(s) while waiting on the board; want 0", ob.dials)
+	}
+	if res.OK || res.Error != groupTestErrNotReached {
+		t.Fatalf("result = %+v, want ok=false with the not-reached explanation", res)
+	}
+}
+
+// A fresh DEAD observation is an answer, not a timeout: ok=false with the
+// path-dead explanation, delay 0, and no exit-address connection for a path
+// that just failed its probe.
+func TestTestOneTargetReportsFreshDeath(t *testing.T) {
+	e := New()
+	ob := &countingOutbound{failingOutbound: failingOutbound{tag: "auto"}}
+	tgt := groupTestTarget{name: "auto", ob: ob}
+
+	since := time.Now().Add(-time.Minute)
+	e.URLTestHistory().MarkFailed("auto")
+
+	res := e.testOneTarget(tgt, map[string]bool{"auto": true}, true, since, groupTestSem())
+	if ob.dials != 0 {
+		t.Fatalf("a dead target was dialled %d time(s); the exit lookup is for alive paths only", ob.dials)
+	}
+	if res.OK || res.Error != groupTestErrPathDead {
+		t.Fatalf("result = %+v, want ok=false with the path-dead explanation", res)
+	}
+	if res.DelayMs != 0 || res.ExitIP != "" {
+		t.Fatalf("a dead path must carry no delay and no exit address: %+v", res)
+	}
+}
+
 // TestGroupTestSingleton is the "no parallel runs" invariant: while a run holds the
 // guard, a second TestGroups must be refused rather than starting a concurrent run
 // (each of these opens real tunnelled connections).

shater/engine/observatory.go

@@ -100,6 +100,14 @@ type observatoryState struct {
 	cycles uint64
 	// busy guards against a slow tick overlapping the next one.
 	busy bool
+	// force is the ONE-SHOT "check now" flag raised by RefreshObservatory: while
+	// it is up the freshness gate is bypassed, so a manual refresh really
+	// RE-MEASURES everything on the plan instead of skipping whatever happens to
+	// be fresh, and each finished batch immediately nudges the next one so the
+	// pass runs back-to-back rather than on the 10s tick. It is cleared the
+	// moment the forced pass schedules its last batch (cursor reaches the end of
+	// the plan) — one full walk, then back to the polite steady state.
+	force bool
 }
 
 // ConfigureObservatory installs (or re-tunes, or stops) the observatory. It is
@@ -120,6 +128,7 @@ func (e *Engine) ConfigureObservatory(cfg ObservatoryConfig) {
 
 	if !cfg.Enabled {
 		e.obs.plan, e.obs.used, e.obs.cursor = nil, nil, 0
+		e.obs.force = false
 		if e.obs.stop != nil {
 			close(e.obs.stop)
 			e.obs.stop, e.obs.done, e.obs.nudge = nil, nil, nil
@@ -163,6 +172,7 @@ func (e *Engine) StopObservatory() {
 	e.obs.stop, e.obs.done, e.obs.nudge = nil, nil, nil
 	e.obs.enabled = false
 	e.obs.plan, e.obs.used, e.obs.cursor = nil, nil, 0
+	e.obs.force = false
 	if stop != nil {
 		close(stop)
 	}
@@ -172,6 +182,38 @@ func (e *Engine) StopObservatory() {
 	}
 }
 
+// RefreshObservatory asks for an OUT-OF-TURN full pass of the probe plan: the
+// cursor is rewound to the top, the one-shot force flag is raised so the pass
+// bypasses the freshness gate (a "check now" that skipped everything fresh
+// would measure nothing and answer with yesterday's numbers), and the loop is
+// nudged so the first batch starts immediately instead of on the next 10s tick.
+//
+// This is the ONLY way the panel's "Test" button touches the network now: the
+// manual group test (grouptest.go) no longer dials anything itself — it calls
+// this, then watches the health board for observations newer than its start
+// instant. One prober, one set of dial paths, one truth.
+//
+// No-op when the observatory is disabled or has no plan: there is nothing to
+// walk, and raising force with no loop to consume it would only arm a stale
+// flag for a future enable. The caller (TestGroups) reports that situation
+// through its own results; this method stays silent about it on purpose —
+// it is a request, not a query.
+func (e *Engine) RefreshObservatory() {
+	e.obsMu.Lock()
+	defer e.obsMu.Unlock()
+	if !e.obs.enabled || len(e.obs.plan) == 0 {
+		return
+	}
+	e.obs.cursor = 0
+	e.obs.force = true
+	if e.obs.nudge != nil {
+		select {
+		case e.obs.nudge <- struct{}{}:
+		default:
+		}
+	}
+}
+
 // ObservatoryStatus reports whether the observatory is on, how many measurements
 // the current plan holds, and how many full passes have completed. Cheap; used by
 // tests and available for diagnostics.
@@ -240,17 +282,18 @@ func (e *Engine) observatoryLoop(stop <-chan struct{}, done chan<- struct{}, nud
 // observatoryTickOnce runs one batch. It is deliberately conservative about when
 // it does nothing:
 //
-//   - a manual exit test (grouptest.go) is in flight => SKIP. That run is the
-//     human's explicit request; the observatory defers rather than competing for
-//     the uplink. It checks the guard but never CLAIMS it, so a pressed Test
-//     button can never be answered "already running" because of background work;
 //   - the previous tick has not finished => SKIP, so slow probes never stack;
 //   - the engine is stopped => jobs no longer resolve and are skipped (probeJob).
+//
+// There is deliberately NO deferral to a manual group test any more. The guard
+// that used to sit here ("skip the tick while e.groupTestRunning is up") made
+// sense when the manual run dialled the uplink itself and the two would have
+// competed for it; that run no longer probes ANYTHING — it asks for a forced
+// pass via RefreshObservatory and then reads the board. Keeping the guard would
+// therefore be worse than pointless: a manual run now DEPENDS on the
+// observatory ticking, and skipping ticks for its whole duration would deadlock
+// the very refresh it is waiting on (up to its full 120s deadline).
 func (e *Engine) observatoryTickOnce() {
-	if e.groupTestRunning.Load() {
-		return
-	}
-
 	e.obsMu.Lock()
 	if e.obs.busy || !e.obs.enabled || len(e.obs.plan) == 0 {
 		e.obsMu.Unlock()
@@ -258,7 +301,8 @@ func (e *Engine) observatoryTickOnce() {
 	}
 	if e.obs.cursor >= len(e.obs.plan) {
 		// Cycle complete: count it and start the next pass from the top. This is
-		// the ONE place the cursor is deliberately rewound.
+		// the ONE place the cursor is rewound on the observatory's own schedule
+		// (RefreshObservatory rewinds it too, but that is the caller's request).
 		e.obs.cycles++
 		e.obs.cursor = 0
 	}
@@ -270,12 +314,32 @@ func (e *Engine) observatoryTickOnce() {
 	batch := append([]ProbeJob(nil), e.obs.plan[start:end]...)
 	e.obs.cursor = end
 	interval := e.obs.interval
+	// A forced pass (RefreshObservatory) bypasses the freshness gate for its one
+	// walk of the plan. The flag is captured for THIS batch and cleared the
+	// moment the walk's last batch is scheduled: the remaining jobs of this very
+	// batch still run unfiltered off the captured copy, and the next pass is an
+	// ordinary polite one again.
+	force := e.obs.force
+	if force && end >= len(e.obs.plan) {
+		e.obs.force = false
+	}
 	e.obs.busy = true
 	e.obsMu.Unlock()
 
 	defer func() {
 		e.obsMu.Lock()
 		e.obs.busy = false
+		// Mid-forced-pass: chain straight into the next batch instead of waiting
+		// out the 10s tick. A human pressed "check now"; walking a subscription-
+		// sized plan at one batch per tick would stretch the answer past the
+		// manual run's deadline for no reason — the batch size still bounds how
+		// much is in flight at once, which is the limit that actually matters.
+		if e.obs.force && e.obs.nudge != nil {
+			select {
+			case e.obs.nudge <- struct{}{}:
+			default:
+			}
+		}
 		e.obsMu.Unlock()
 	}()
 
@@ -287,7 +351,7 @@ func (e *Engine) observatoryTickOnce() {
 	sem := make(chan struct{}, observatoryConcurrency)
 	var wg sync.WaitGroup
 	for _, j := range batch {
-		if !observatoryShouldProbe(hist, j, interval, now) {
+		if !force && !observatoryShouldProbe(hist, j, interval, now) {
 			continue
 		}
 		wg.Add(1)

shater/engine/observatory_test.go

@@ -100,28 +100,89 @@ func TestObservatoryTickStoppedEngine(t *testing.T) {
 	}
 }
 
-// The exit-test gate: while a manual group/chain test is in flight the tick is a
-// no-op — the observatory checks the guard but never claims it, so a pressed
-// Test button is never refused because of background work.
-func TestObservatoryDefersToExitTest(t *testing.T) {
+// The INVERSE of the old exit-test gate, pinned so it cannot come back: the
+// tick must RUN while a manual group test is in flight. The manual run no
+// longer probes anything — it asks for a forced pass and then waits on the
+// board — so a tick that deferred to it would deadlock the very refresh the
+// run is polling for, until its 120s deadline reported "not reached" about
+// paths nobody attempted.
+func TestObservatoryTicksDuringManualRun(t *testing.T) {
 	e := New()
 	defer e.StopObservatory()
 	e.ConfigureObservatory(ObservatoryConfig{Enabled: true, Options: obsFixture(), ProbeInterval: time.Minute})
 
 	e.groupTestRunning.Store(true)
 	e.observatoryTickOnce()
-	if _, cursor, _ := obsState(e); cursor != 0 {
-		t.Fatal("tick ran while an exit test was in flight")
+	if _, cursor, _ := obsState(e); cursor == 0 {
+		t.Fatal("tick deferred to an in-flight manual run; the run depends on the tick now")
 	}
 	e.groupTestRunning.Store(false)
 
-	// And the guard is free: a manual run can start immediately.
+	// The observatory never claims the manual-run guard either way round: a
+	// manual run can still start immediately.
 	if !e.TestGroups(nil, "") {
-		t.Fatal("a manual exit test was refused; the observatory must never hold groupTestRunning")
+		t.Fatal("a manual test was refused; the observatory must never hold groupTestRunning")
 	}
 	waitGroupTestIdle(t, e)
 }
 
+// RefreshObservatory is the manual run's whole probing story: it rewinds the
+// cursor, raises the one-shot force flag (bypassing the freshness gate for one
+// full walk), and the flag clears itself when the forced pass schedules its
+// last batch. Disabled or plan-less observatories ignore it entirely.
+func TestRefreshObservatoryForcesOnePass(t *testing.T) {
+	e := New()
+	defer e.StopObservatory()
+
+	// Disabled: a refresh request is a documented no-op.
+	e.RefreshObservatory()
+	e.obsMu.Lock()
+	force := e.obs.force
+	e.obsMu.Unlock()
+	if force {
+		t.Fatal("RefreshObservatory raised force on a disabled observatory")
+	}
+
+	e.ConfigureObservatory(ObservatoryConfig{Enabled: true, Options: obsFixture(), ProbeInterval: time.Minute})
+
+	// Fresh observations on every planned tag: the polite gate would skip them
+	// all, which is exactly what a forced pass must NOT do.
+	hist := e.URLTestHistory()
+	now := time.Now()
+	hist.StoreURLTestHistory("n1", &adapter.URLTestHistory{LastOK: now, Delay: 5})
+	hist.StoreURLTestHistory("n2", &adapter.URLTestHistory{LastOK: now, Delay: 5})
+
+	// Pretend a walk was mid-plan; the refresh must rewind it.
+	e.obsMu.Lock()
+	e.obs.cursor = 1
+	e.obsMu.Unlock()
+	e.RefreshObservatory()
+	e.obsMu.Lock()
+	cursor, force := e.obs.cursor, e.obs.force
+	e.obsMu.Unlock()
+	if cursor != 0 || !force {
+		t.Fatalf("after refresh: cursor=%d force=%v, want 0/true", cursor, force)
+	}
+
+	// One tick covers the whole 2-job plan (batch is 24), so the forced pass
+	// completes and the flag clears itself — one full walk, not a permanent mode.
+	e.observatoryTickOnce()
+	e.obsMu.Lock()
+	cursor, force = e.obs.cursor, e.obs.force
+	e.obsMu.Unlock()
+	if force {
+		t.Fatal("force flag survived the forced pass; it must be one-shot")
+	}
+	if cursor != 2 {
+		t.Fatalf("cursor after forced tick = %d, want 2 (the pass really walked the plan)", cursor)
+	}
+	// The stopped engine resolves no outbounds, so the probes were skipped —
+	// but the fresh history proves the GATE was bypassed only if the jobs were
+	// attempted; attempt-tracking lives in probeJob, which needs a box. What is
+	// pinned here is the flag lifecycle and the rewind, the two halves
+	// TestGroups depends on.
+}
+
 // The freshness gate: a job whose every covered tag has an observation younger
 // than the global probe interval is skipped — that set is exactly what an ACTIVE
 // group is measuring itself, and the observatory must not duplicate or suppress

shater/engine/probeplan.go

@@ -27,8 +27,12 @@ import (
 //   - a CHAIN entry tag "chain-<n>-hN" (the exit wrapper, generate/chain.go) is
 //     probed ITSELF: dialling the copy pulls the whole L1→…→Ln path through its
 //     Detour links, so one probe is the chain's end-to-end health. Intermediate
-//     NODE hops are not probed separately — their death is visible in the exit
-//     probe and no selection depends on them;
+//     NODE hop wrappers ("chain-<n>-h<i>" that are plain outbounds) are probed
+//     TOO — not because selection needs them (it does not), but because an
+//     operator does: dialling "chain-<n>-h1" measures exactly the prefix of the
+//     path up to and including hop 1, so when the exit probe dies the per-hop
+//     probes say WHICH hop died instead of only that the chain did (the
+//     ChainHealth hops surface, grouphealth.go);
 //   - a GROUP hop inside a chain ("chain-<n>-h<i>", a wrapper selector reached by
 //     walking the exit's Detour links) additionally has its member copies
 //     "chain-<n>-h<i>-<member>" probed: they are the wrapper's selection
@@ -291,8 +295,20 @@ func (w *planWalk) visitMember(group, member string) {
 // walkDetour follows a probed tag's Detour links toward the router. A group met
 // on the way is a chain's GROUP HOP wrapper: its member copies are selection
 // candidates and are probed, each through its own prefix of the path. A plain
-// outbound met on the way is an intermediate node hop — marked used, never probed
-// on its own — and the walk continues through it.
+// outbound met on the way falls in two halves:
+//
+//   - a chain NODE HOP wrapper ("chain-<n>-h<i>", parseChainExitTag) is probed
+//     as a measurement of its own. Dialling it pulls exactly the prefix of the
+//     chain up to and including hop i through the Detour links, so its verdict
+//     LOCALISES a failure: the exit probe says the chain died, the hop probes
+//     say where. It used to be skipped here on the argument that its death
+//     shows up in the exit probe anyway — true for end-to-end health, useless
+//     for an operator staring at a dead 4-hop chain. The measurement is stored
+//     under the wrapper tag alone: a chain prefix is nobody else's dial path,
+//     so no alias may borrow its verdict;
+//   - anything else on the detour path — an "egress-…" interface outbound, an
+//     ordinary node's egress detour — keeps today's behaviour: marked used,
+//     never probed on its own, and the walk continues through it.
 func (w *planWalk) walkDetour(tag string) {
 	if tag == "" || w.walked[tag] {
 		return
@@ -303,25 +319,28 @@ func (w *planWalk) walkDetour(tag string) {
 		return
 	}
 	w.used[tag] = true
-	if ent.group {
-		next := ""
-		for _, m := range ent.members {
-			ment, ok := w.byTag[m]
-			if !ok {
-				continue
-			}
-			w.used[m] = true
-			if !ment.group && probeablePlanTag(ment.typ, m) {
-				w.addTarget(dialKeyTag(m), m)
-			}
-			if next == "" {
-				next = ment.detour // every member detours into the same previous hop
-			}
+	if !ent.group {
+		if _, isHopWrapper := parseChainExitTag(tag); isHopWrapper && probeablePlanTag(ent.typ, tag) {
+			w.addTarget(dialKeyTag(tag), tag)
 		}
-		w.walkDetour(next)
+		w.walkDetour(ent.detour)
 		return
 	}
-	w.walkDetour(ent.detour)
+	next := ""
+	for _, m := range ent.members {
+		ment, ok := w.byTag[m]
+		if !ok {
+			continue
+		}
+		w.used[m] = true
+		if !ment.group && probeablePlanTag(ment.typ, m) {
+			w.addTarget(dialKeyTag(m), m)
+		}
+		if next == "" {
+			next = ment.detour // every member detours into the same previous hop
+		}
+	}
+	w.walkDetour(next)
 }
 
 // dialKeyTag / dialKeyCopy build the dedup key for one measurement target.

shater/engine/probeplan_test.go

@@ -218,6 +218,64 @@ func TestObservatoryPlanChain(t *testing.T) {
 	}
 }
 
+// A chain with a plain NODE hop wrapper on the way to the exit: the wrapper IS
+// a measurement of its own now (walkDetour) — dialling "chain-<c>-h1" measures
+// exactly the path prefix up to hop 1, which is what localises a dead hop. And
+// the other half of the honesty contract: a base node that appears ONLY as a
+// chain group-hop member is never measured under its BASE tag — the member
+// copy's observation stays on the copy, because the copy's dial path (through
+// the chain prefix) is not the base outbound's dial path, and no job may write
+// a direct-looking verdict onto a tag it did not dial.
+func TestObservatoryPlanChainHopWrappersProbed(t *testing.T) {
+	opts := option.Options{
+		Outbounds: []option.Outbound{
+			// L1: a plain node hop wrapper.
+			fixNode("chain-c2-h1", ""),
+			// L2: a group hop over one member copy of base node N.
+			fixNode("chain-c2-h2-N", "chain-c2-h1"),
+			fixGroup(C.TypeSelector, "chain-c2-h2", "chain-c2-h2-N"),
+			// L3: the exit, a plain node copy.
+			fixNode("chain-c2-h3", "chain-c2-h2"),
+			// The base node behind the L2 member copy: emitted, referenced by
+			// NOTHING but the copy's name.
+			fixNode("N", ""),
+		},
+		Route: fixRoute("", "chain-c2-h3"),
+	}
+	byDial, used := planOf(t, opts)
+
+	// The node hop wrapper is a job of its own, stored only under itself.
+	j := assertPlanHas(t, byDial, "chain-c2-h1")
+	if len(j.Store) != 1 || j.Store[0] != "chain-c2-h1" {
+		t.Errorf("node hop wrapper store = %v, want only itself (a chain prefix is nobody else's path)", j.Store)
+	}
+	// The exit and the group-hop member copy are jobs, as before.
+	assertPlanHas(t, byDial, "chain-c2-h3")
+	assertPlanHas(t, byDial, "chain-c2-h2-N")
+
+	// NO job may record anything under the base tag N: not as a dial, not as a
+	// store alias. A direct measurement can never land on it from this config.
+	if _, ok := byDial["N"]; ok {
+		t.Error("plan dials the base node N, which no rule reaches")
+	}
+	for dial, job := range byDial {
+		for _, tag := range job.Store {
+			if tag == "N" {
+				t.Errorf("job %q stores under the base tag N; a chain member copy must never alias its base", dial)
+			}
+		}
+	}
+	if used["N"] {
+		t.Error("used-set claims the base node N; only the chain copies are on the path")
+	}
+	// The wrappers themselves are used (they are the path).
+	for _, u := range []string{"chain-c2-h1", "chain-c2-h2", "chain-c2-h3", "chain-c2-h2-N"} {
+		if !used[u] {
+			t.Errorf("used-set is missing %q", u)
+		}
+	}
+}
+
 // DNS server detours and route.Final are roots too — a group referenced only as a
 // resolver's detour is still a used, probed path.
 func TestObservatoryPlanDNSDetourRoot(t *testing.T) {

shater/engine/selfcheck.go

@@ -0,0 +1,70 @@
+package engine
+
+import (
+	C "github.com/sagernet/sing-box/constant"
+	"github.com/sagernet/sing-box/option"
+)
+
+// Standing down the self-check of unused urltest groups (health board §5.C).
+//
+// # Why the engine does this, and not the generator
+//
+// A urltest group probes its own members: a warm-up sweep at PostStart and a
+// ticker for as long as traffic touches it (protocol/group/urltest.go). For a
+// group the routing rules actually use, that probing travels the same path the
+// traffic does and is welcome. For a group NO rule reaches, it dials the
+// members' base outbounds directly from the router — a path nothing uses — and
+// stores the results under the base tags, where every health consumer reads
+// them as "the node's health". A node that only works behind a tunnel then
+// reads dead on the shared board because an idle group measured it over the
+// blocked direct WAN. The observatory's plan (probeplan.go) is the single
+// statement of what gets probed and along which paths; this file makes the
+// config that reaches box.New SAY so, by flipping SelfCheck off on every
+// urltest group outside the plan's used-set.
+//
+// The generator cannot do this: whether a group is used is a property of the
+// EMITTED rules and DNS detours as a whole, and BuildObservatoryPlan is the one
+// place that reachability is computed. Recomputing it in the generator would be
+// a second copy of the same walk, and second copies drift.
+
+// standDownUnusedSelfCheck flips SelfCheck to false on every urltest group in
+// opts that the observatory's used-set does not cover, and returns how many it
+// stood down (for the apply log and the tests). Selector groups are left alone
+// — they have no self-check to stand down — and a group any rule, route.Final
+// or DNS detour reaches keeps its default (nil == on).
+//
+// It MUTATES the option structs the caller handed in: opts.Outbounds carries
+// the generator's option values as pointers behind an `any` (the generator
+// always emits *option.URLTestOutboundOptions), so writing through them changes
+// the caller's config. That is deliberate, not an accident to guard against:
+// the plan is the single source of what gets probed, and the config that
+// reaches box.New must already say so — a copy-on-write here would build a box
+// whose groups probe paths the hash and the plan claim nobody probes. It runs
+// BEFORE hashOptions in applyLocked for the same reason: the hash must describe
+// what is really built, so a rule change that flips a group used<->unused is a
+// real config change and triggers a swap.
+//
+// A urltest outbound whose Options is not the expected pointer type (a value,
+// or something foreign) is skipped rather than guessed at: it did not come from
+// our generator, and silently rebuilding somebody else's option struct is worse
+// than leaving one group's self-check up.
+func standDownUnusedSelfCheck(opts option.Options) int {
+	_, used := BuildObservatoryPlan(opts, "")
+	stood := 0
+	for i := range opts.Outbounds {
+		ob := &opts.Outbounds[i]
+		if ob.Type != C.TypeURLTest || used[ob.Tag] {
+			continue
+		}
+		utOpts, ok := ob.Options.(*option.URLTestOutboundOptions)
+		if !ok {
+			continue
+		}
+		// One fresh pointer per group, so no two option structs alias a shared
+		// bool that a later caller could flip for both at once.
+		off := false
+		utOpts.SelfCheck = &off
+		stood++
+	}
+	return stood
+}

shater/engine/selfcheck_test.go

@@ -0,0 +1,204 @@
+package engine
+
+import (
+	"testing"
+
+	C "github.com/sagernet/sing-box/constant"
+	"github.com/sagernet/sing-box/option"
+)
+
+// standDownUnusedSelfCheck is the enforcement half of the one-probe rule: a
+// urltest group no rule reaches gets SelfCheck=false written into its option
+// struct (in place — the struct the box will be built from), a rule-reachable
+// one keeps its nil default, and selectors are never touched at all (they have
+// no self-check to stand down). The count it returns is what applyLocked logs.
+func TestStandDownUnusedSelfCheck(t *testing.T) {
+	opts := option.Options{
+		Outbounds: []option.Outbound{
+			fixNode("n1", ""),
+			fixNode("idle1", ""),
+			fixNode("pin1", ""),
+			// Reached by a rule: must keep probing itself.
+			fixGroup(C.TypeURLTest, "auto", "n1"),
+			// Reached by nothing: its self-check dials a path nobody uses and
+			// must be stood down.
+			fixGroup(C.TypeURLTest, "idle", "idle1"),
+			// A selector reached by nothing: left alone — no self-check exists.
+			fixGroup(C.TypeSelector, "pins", "pin1"),
+		},
+		Route: fixRoute("auto"),
+	}
+
+	stood := standDownUnusedSelfCheck(opts)
+	if stood != 1 {
+		t.Fatalf("stood down %d group(s), want exactly 1 (idle)", stood)
+	}
+
+	find := func(tag string) option.Outbound {
+		t.Helper()
+		for _, ob := range opts.Outbounds {
+			if ob.Tag == tag {
+				return ob
+			}
+		}
+		t.Fatalf("outbound %q missing from opts", tag)
+		return option.Outbound{}
+	}
+
+	// The unused urltest group: SelfCheck is now an explicit false in the very
+	// struct the caller handed in (the mutation is the point — the config that
+	// reaches box.New must say what the plan says).
+	idle := find("idle").Options.(*option.URLTestOutboundOptions)
+	if idle.SelfCheck == nil || *idle.SelfCheck {
+		t.Fatalf("idle group SelfCheck = %v, want an explicit false", idle.SelfCheck)
+	}
+
+	// The used urltest group keeps the nil default (self-check on): its probing
+	// travels the path traffic actually takes and is welcome.
+	auto := find("auto").Options.(*option.URLTestOutboundOptions)
+	if auto.SelfCheck != nil {
+		t.Fatalf("used group SelfCheck = %v, want nil (untouched default)", *auto.SelfCheck)
+	}
+
+	// The selector's option struct has no SelfCheck field at all; what is
+	// pinned here is that stand-down neither counted it nor mangled its type.
+	if _, ok := find("pins").Options.(*option.SelectorOutboundOptions); !ok {
+		t.Fatal("selector options were rebuilt; stand-down must leave selectors alone")
+	}
+
+	// Idempotence: a second pass finds the same one group (already-false is
+	// still counted — the function reports plan membership, not novelty) and
+	// changes nothing further. This is what keeps the apply hash stable across
+	// the every-minute reconcile.
+	if again := standDownUnusedSelfCheck(opts); again != 1 {
+		t.Fatalf("second pass stood down %d, want 1 (deterministic on identical opts)", again)
+	}
+	if idle.SelfCheck == nil || *idle.SelfCheck {
+		t.Fatal("second pass flipped the idle group's SelfCheck back")
+	}
+}
+
+// TestStandDownNeverTouchesChainHopWrappers pins the property the production
+// config lives or dies by, DIRECTLY rather than transitively through the plan
+// tests: a CHAIN HOP WRAPPER group ("chain-<name>-h<i>", type urltest) must
+// NEVER be stood down.
+//
+// The failure mode this guards against is subtle and silent. The owner's live
+// rule routes through egress:ewan -> node:awgout -> group:sub0 -> group:sub1
+// -> group:sub2 — a chain whose group hops are rebuilt as urltest wrappers
+// over per-chain member copies. Those wrappers are the ONLY probing that
+// travels the real path: their own schedule is the on-path measurement, and
+// the selection inside the chain (which member each hop dials) depends on the
+// verdicts it writes. Reachability of a wrapper is established by walkDetour
+// (probeplan.go) walking the exit's Detour links — a DIFFERENT code path from
+// the root/member expansion the other tests exercise. If a future change to
+// that walk dropped the wrappers from the used-set, standDownUnusedSelfCheck
+// would obediently flip their SelfCheck off, the chain would stop measuring
+// itself, and nothing else would ever measure it: we would have replaced the
+// false reading this rework removed with NO reading at all. Both existing
+// tests could stay green while that happened — the plan test asserts the
+// used-set, not the stand-down; the stand-down test above never mentions
+// chains. This one closes that gap by asserting the stand-down's OUTPUT on
+// the production topology.
+//
+// The fixture mirrors what the generator actually emits for that config, not
+// a toy: an exit wrapper chain-p-h4 (urltest over member copies) whose copies
+// detour into chain-p-h3 (urltest), whose copies detour into chain-p-h2
+// (urltest), whose copies detour into chain-p-h1 (the plain AmneziaWG hop
+// copy), which detours into egress-ewan — the detour lives on the member
+// COPIES, not on the wrappers, exactly as generate/chain.go builds it. The
+// base groups sub0/sub1/sub2 are ALSO emitted, over the base node tags,
+// reached by nothing — which is precisely what buildGroups does today and
+// precisely the false direct prober the stand-down exists to silence.
+func TestStandDownNeverTouchesChainHopWrappers(t *testing.T) {
+	opts := option.Options{
+		Outbounds: []option.Outbound{
+			// The egress the whole chain leaves through.
+			fixUtility(C.TypeDirect, "egress-ewan"),
+			// L1: the plain AWG hop copy (in production an endpoint; a plain
+			// outbound here exercises the same walk — walkDetour treats both as
+			// non-group entries).
+			fixNode("chain-p-h1", "egress-ewan"),
+			// L2..L4: urltest group hop wrappers over per-chain member copies;
+			// each COPY detours into the previous hop.
+			fixNode("chain-p-h2-a", "chain-p-h1"),
+			fixNode("chain-p-h2-b", "chain-p-h1"),
+			fixGroup(C.TypeURLTest, "chain-p-h2", "chain-p-h2-a", "chain-p-h2-b"),
+			fixNode("chain-p-h3-a", "chain-p-h2"),
+			fixNode("chain-p-h3-b", "chain-p-h2"),
+			fixGroup(C.TypeURLTest, "chain-p-h3", "chain-p-h3-a", "chain-p-h3-b"),
+			fixNode("chain-p-h4-a", "chain-p-h3"),
+			fixNode("chain-p-h4-b", "chain-p-h3"),
+			fixGroup(C.TypeURLTest, "chain-p-h4", "chain-p-h4-a", "chain-p-h4-b"),
+			// The base nodes and the base groups over them: emitted alongside
+			// the chain, reached by no rule — the generator keeps emitting them
+			// today, and they are exactly the direct-WAN probers that poisoned
+			// the board.
+			fixNode("a", ""),
+			fixNode("b", ""),
+			fixGroup(C.TypeURLTest, "sub0", "a", "b"),
+			fixGroup(C.TypeURLTest, "sub1", "a", "b"),
+			fixGroup(C.TypeURLTest, "sub2", "a", "b"),
+		},
+		// One rule, targeting the chain's exit wrapper — the production shape.
+		Route: fixRoute("", "chain-p-h4"),
+	}
+
+	stood := standDownUnusedSelfCheck(opts)
+	if stood != 3 {
+		t.Fatalf("stood down %d group(s), want exactly the 3 base groups sub0/sub1/sub2", stood)
+	}
+
+	utOptions := func(tag string) *option.URLTestOutboundOptions {
+		t.Helper()
+		for _, ob := range opts.Outbounds {
+			if ob.Tag == tag {
+				o, ok := ob.Options.(*option.URLTestOutboundOptions)
+				if !ok {
+					t.Fatalf("outbound %q is not a urltest group", tag)
+				}
+				return o
+			}
+		}
+		t.Fatalf("outbound %q missing from opts", tag)
+		return nil
+	}
+
+	// Every hop wrapper keeps its self-check: nil, the untouched default. An
+	// explicit false on ANY of these is the chain going blind.
+	for _, wrapper := range []string{"chain-p-h2", "chain-p-h3", "chain-p-h4"} {
+		if sc := utOptions(wrapper).SelfCheck; sc != nil {
+			t.Errorf("hop wrapper %q SelfCheck = %v, want nil — its own schedule IS the on-path measurement", wrapper, *sc)
+		}
+	}
+	// Every base group is stood down: nothing routes through them, and their
+	// probing would dial the base nodes over the direct WAN.
+	for _, base := range []string{"sub0", "sub1", "sub2"} {
+		if sc := utOptions(base).SelfCheck; sc == nil || *sc {
+			t.Errorf("base group %q SelfCheck = %v, want an explicit false", base, sc)
+		}
+	}
+
+	// And the reason the stand-down of sub0/sub1/sub2 is SAFE in this config:
+	// with the base groups silenced, the base node tags a/b have no measurement
+	// path at all — no job dials them and no job stores under them (the chain's
+	// member copies record only under themselves). Their board entries simply
+	// stop being written, honestly untested, instead of carrying a false
+	// direct-WAN verdict.
+	jobs, used := BuildObservatoryPlan(opts, "")
+	for _, baseTag := range []string{"a", "b"} {
+		if used[baseTag] {
+			t.Errorf("used-set claims base node %q; only the chain copies are on the path", baseTag)
+		}
+		for _, j := range jobs {
+			if j.Dial == baseTag {
+				t.Errorf("plan dials base node %q, which no rule reaches", baseTag)
+			}
+			for _, s := range j.Store {
+				if s == baseTag {
+					t.Errorf("job %q stores under base node %q; a chain copy must never alias its base", j.Dial, baseTag)
+				}
+			}
+		}
+	}
+}

shater/generate/observatory_reach_integration_test.go

@@ -162,14 +162,23 @@ func TestObservatoryPlanFromGeneratedConfig(t *testing.T) {
 			t.Errorf("chain exit %q store = %v, want only itself (a chain prefix is nobody else's path)", chainExit, store)
 		}
 	}
-	// The chain's intermediate node hop n1 is used (the exit detours through it) but
-	// NOT probed separately — its death is visible in the exit probe, and no
-	// selection depends on it. n1 IS probed via the plain "auto" group, so the
-	// assertion is "no SEPARATE chain-hop-h1 job", not "n1 never dialled".
-	if jobDials(jobs, "chain-hop-h1") {
-		t.Errorf("plan dials intermediate chain hop chain-hop-h1 — only the exit is probed end-to-end")
+	// The chain's intermediate NODE hop wrapper IS probed now, as a measurement
+	// of its own: dialling chain-hop-h1 measures exactly the prefix of the path
+	// up to and including hop 1, which is what lets an operator see WHICH hop
+	// died instead of only that the chain did (engine/probeplan.go walkDetour,
+	// surfaced through ChainHealth.Hops). Its store is only itself — a chain
+	// prefix is nobody else's dial path, so the base node n1 must never inherit
+	// a verdict measured through the chain.
+	chainHop := "chain-hop-h1"
+	if !jobDials(jobs, chainHop) {
+		t.Errorf("plan does not probe intermediate chain hop %q; jobs=%v", chainHop, dialsOfJobs(jobs))
+	} else {
+		store := jobStore(jobs, chainHop)
+		if len(store) != 1 || store[0] != chainHop {
+			t.Errorf("chain hop %q store = %v, want only itself (no base alias for a chain prefix)", chainHop, store)
+		}
 	}
-	if !used[chainExit] || !used["chain-hop-h1"] {
+	if !used[chainExit] || !used[chainHop] {
 		t.Errorf("used-set is missing chain hop tags; used=%v", used)
 	}
 	// The used groups themselves are in the used-set but never dialled (a group is

shater/panel/api.go

@@ -1359,9 +1359,24 @@ type groupTestStartResponse struct {
 	Reason  string `json:"reason,omitempty"`
 }
 
-// handleGroupsTest → /api/groups/test: measure each target's (group's or chain's)
-// latency and the public address it currently exits from. A chain is dialled via
-// its exit wrapper, so the numbers are end-to-end properties of the whole path.
+// handleGroupsTest → /api/groups/test: report each target's (group's or chain's)
+// health as the OBSERVATORY just measured it, plus the public address it
+// currently exits from.
+//
+// POST no longer dials anything. It asks the engine's observatory for an
+// out-of-turn pass of its probe plan and then reports what that pass measured:
+// each target resolves off the health board the moment its measured path
+// carries an observation newer than the button press. The consequences the
+// frontend should expect (the response SHAPE is unchanged):
+//   - delay_ms/ok/tested_unix describe the observatory's measurement of the
+//     target's real routed path — tested_unix is when that observation was
+//     taken, which may be a second or two before the poll that delivered it;
+//   - a target no enabled rule routes through is never measured and comes back
+//     ok=false with an error saying so, immediately — the observatory only
+//     probes paths the rules use, and the panel should render that as a
+//     routing fact, not a failure;
+//   - a run can take up to the engine's wait deadline (120s) when the
+//     observatory has a large plan to walk; poll GET for progress as before.
 //
 // Contract for the frontend:
 //   - POST {"name":"auto"} → test that group or chain; an empty/absent name tests
@@ -1395,9 +1410,11 @@ func (s *Server) handleGroupsTest(w http.ResponseWriter, r *http.Request) {
 		if n := strings.TrimSpace(req.Name); n != "" {
 			names = []string{n}
 		}
-		// Probe URL from the model (best-effort) — the global one, the only probe
-		// instrument left; a UCI read failure falls back to the engine's built-in
-		// default.
+		// Probe URL from the model (best-effort), passed through for signature
+		// stability only: the engine IGNORES it now. The probe URL is a global
+		// observatory setting installed at apply time, and the manual test reads
+		// the observatory's measurements rather than dialling with a URL of its
+		// own — see engine.TestGroups.
 		var probeURL string
 		if m, err := model.ReadUCI(); err == nil {
 			probeURL = strings.TrimSpace(m.Globals.ProbeURL)
@@ -1469,6 +1486,31 @@ type groupHealthResponse struct {
 //     was measured THROUGH the group's egress, so it is not comparable with the global
 //     per-node numbers in /api/stats node_health. That is the entire point of this
 //     endpoint — the same node in two groups with two egresses has two health states.
+//
+// chains[] additionally carries the PER-HOP readout for every chain the running
+// box materialised:
+//
+//	chains[].hops = [{index,tag,kind,exit,state,delay_ms,age_seconds,selected,
+//	                  total,tested,alive,dead,untested}...]
+//
+//   - hops is L1..Ln in wire order (index is 1-based); the entry with exit=true
+//     is the last hop, where traffic leaves to the internet. Each hop's numbers
+//     measure the chain PREFIX up to and including that hop — the observatory
+//     dials the hop wrappers — so a dead hop N with alive hops 1..N-1 localises
+//     the failure to hop N's own leg.
+//   - kind is "node" (one measurement: total=1, selected empty) or "group" (a
+//     roll-up of the hop's member copies, with the same counter invariants as a
+//     group: tested == alive+dead, alive+dead+untested == total; selected is
+//     the node NAME the hop currently picks; delay_ms/age_seconds are the
+//     selected member's observation, or the freshest alive member's when the
+//     selection has none).
+//   - state follows the same closed set and the same honesty rule as groups:
+//     "dead" only on a positive finding, "untested" for no fresh data — and for
+//     a group hop, "alive" as long as ANY member answers.
+//   - an ABSENT/empty hops key means the running box never materialised the
+//     chain (unused, or a 1-hop chain that resolves straight to its target) —
+//     it must NOT be read as "this chain has no hops"; the config, not this
+//     endpoint, knows the configured hop count.
 func (s *Server) handleGroupsHealth(w http.ResponseWriter, r *http.Request) {
 	if r.Method != http.MethodGet {
 		writeError(w, http.StatusMethodNotAllowed, "method not allowed")