feat(dns,fetch): resolvers and list fetches follow the uplink; the node cache survives the reboot it exists for
test / go + panel tests (push) Successful in 1m42s
release / test gate (push) Successful in 1m40s
release / apk aarch64_cortex-a53 (push) Successful in 2m55s
release / apk x86_64 (push) Successful in 2m56s
release / release apk (push) Successful in 9s

The carrier behind this router's SIM refuses TCP/443 to 9.9.9.9 and 1.1.1.1 while
carrying everything else — measured with a positive control (ya.ru:443 and
77.88.8.8:53 connect, every sim-bypass node connects, those two are refused). The
configured resolvers go out DIRECT, not through the tunnel, so on that uplink DNS
resolved nothing: the vless server names did not resolve, the hop in front of
awgout never came up, and the whole chain died with it. One pair of global scalars
cannot be right for two uplinks; the object that knows which uplink is live is the
profile.

  * config profile gains resolver_default, resolver_fallback and fetch_detour
    beside endpoint_resolver. Empty = inherit, PER FIELD.
  * globals.fetch_detour replaces `const filterFetchDetour = tagDirect`. Behind a
    carrier whitelist `direct` is not the safe path, it is the path where the
    source is refused forever and the list never loads.
  * A subscription's fetch_via becomes an OVERRIDE, which gives it a third state.
    ReadUCI used to parse an absent option as the literal "direct", so "chose
    clear-text" and "never touched this row" were the same value. migrate2to3
    performs the reinterpretation ONCE, in the open. Schema 2 -> 3.
  * An unusable override falls back (resolvers to globals, fetch_detour to direct)
    and says so at critical, naming profile, field, value and what is in force.

The panel was displaying globals while the engine used the profile's value; the
owner caught it. The field now keeps the STORED value with a separate line naming
what is in force, and the rule that answers "what is in force" moved to the daemon
(GET /api/config/effective) so it stops existing in two languages.

Cold start, by owner's requirement: rule-sets are read from the cache when the
source is unreachable instead of being dropped, and the subscription cache reader
is fixed. Its first fix was wrong and only Linux said so — mtime ties to the digit
because the kernel caches the stamp per tick, and this board has no RTC, so the
ordering can invert across a reboot. Replaced by a generation counter in the file.

Woke and closed a LAN-dark defect: wgdedup read only the deprecated, always-empty
DownloadDetour, never HTTPClient.Detour, so fetch_detour=node:<awg> made a node
used, the dedup pass did not know, merged it away, and left the rule-set pointing
at a tag box.Start could not resolve. Reproduced through a real box.New.

Also: ValidateProfiles had no caller; "applied from the cache" graded critical
though the list is in force; the auth matrix never walked /api/log or
/api/rules/reachability; the CLI and daemon disagreed about where a subscription
is fetched.

NOT fixed, stated rather than implied: the R5 preflight still probes direct, so a
list never yet fetched cannot bootstrap over the detour alone; the router's own
DNS on the SIM stays dead (dnscrypt-proxy bootstraps via blocked addresses).

Gate: bash scripts/run-tests.sh green, 7/7, privileged tests really ran.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-28 14:40:58 +03:00
co-authored by Claude Opus 5
parent 625942834b
commit 8c0ea55054
55 changed files with 8328 additions and 627 deletions
+194
View File
@@ -1490,3 +1490,197 @@ still fails against an ISP that fake/disorder/oob/autottl defeats, the argument
in D13 for choosing ByeDPI over zapret is still the right argument and this
decision is the one to revisit — with a measurement of the FIXED presets first,
which is the step that was skipped last time.
## D30 — `fetch_via` has THREE states, and the third one had to be bought with a schema bump
Decided 2026-07-28. Adds `fetch_detour` to `config globals` and `config profile`,
adds `resolver_default` / `resolver_fallback` to `config profile`, and turns a
subscription's `fetch_via` from *the* statement about where that feed is fetched
into an **override** of the general one. `CurrentSchemaVersion` goes 2 → 3.
**What forced it, measured on the production router (BPI-R3, SIM uplink).** The
operator's mobile carrier refuses TCP/443 to `9.9.9.9` and `1.1.1.1` while
carrying everything else — verified with a positive control (`ya.ru:443` and
`77.88.8.8:53` connect; the two resolvers are refused). The resolvers configured
in `globals` go out **direct**, not through the tunnel (`dnsDetour` returns `""`,
which is `dialer.NewDefault`, an ordinary system socket), so on that uplink DNS
resolved nothing: the vless server names did not resolve, the hop in front of
`awgout` never came up, and the whole `swan-bypass-wg-subs` chain died with it.
The ethernet uplink has the opposite problem — it can reach those resolvers
perfectly well and there was no reason to give them up. One pair of global
scalars cannot be right for both uplinks, and the object that already knows which
uplink is live is the profile.
**Inheritance is PER FIELD.** A profile that sets only `resolver_fallback` keeps
globals' `resolver_default`. The alternative — "a profile that names any resolver
owns all of them" — silently drags a field the operator never wrote into the
override, which is the same defect class as an open `default:`.
**The empty string is "not set", and that is not a convention we invented.** UCI
draws exactly this line already: `option` present vs absent. The renderer's
`strOpt` omits an empty value, so a model field of `""` round-trips as an absent
option with no new machinery, and the accepted vocabulary of each option is a
closed positive set that `""` is not a member of. A `*string` would have changed
the JSON the panel exchanges over `/api/config` and added a nil-deref class to a
stdlib-only leaf package; a companion `FetchViaSet bool` would have allowed the
contradictory state *(not set, "proxy")* that nothing could reject.
**WHAT THIS BREAKS, stated plainly.** Until now `ReadUCI` parsed an absent
`option fetch_via` as the literal string `direct`
(`s.optOr("fetch_via", "direct")`). So "the operator chose a clear-text fetch"
and "the operator never touched this row" were **the same value** in the model,
and they now have to land in different places. Every subscription with a blank
`fetch_via` changes meaning: it used to mean *direct*, it now means *inherit the
general `fetch_detour`*.
**So the reinterpretation is performed ONCE, by a migration, in the open.**
`migrate2to3` writes `fetch_via='direct'` into every subscription whose value is
absent, blank, or not one of `proxy`/`direct`. After it runs, every existing feed
fetches exactly where it fetched before, the change is visible in
`/etc/config/shater` to `uci show`, and the pre-migration file is sitting beside
it (`migrateWith` takes `config.pre-v2.bak` before any step runs). Simply dropping
the parser default without the migration would have been one line and would have
been *worse*: the reinterpretation would then happen continuously and invisibly,
at every read, and the first operator to set
`globals.fetch_detour='group:sim-bypass'` would find every subscription that had
never mentioned `fetch_via` silently moved into a tunnel.
**An unrecognised value is normalised, not preserved.** `fetch_via='yes'` meant
`direct` under v2 — every reader in the tree asks `EqualFold(v, "proxy")` and
treats everything else as direct — so writing `direct` records what the config
already did rather than changing it. Each such rewrite is announced through
`migrateNotef`; none is silent. At runtime the same value would now be
`FetchViaUnknown`, which `ClassifyFetchVia` refuses to fold into a real mode:
there is no safe silent answer, because choosing `direct` discloses the feed URL
and this router's real address to the provider and the ISP (the exact thing
`fetch_via=proxy` is set to prevent) and choosing the general detour puts a feed
through a tunnel nobody asked for. `ValidateSubscriptions` names it instead.
**WHY THE SCHEMA VERSION IS BUMPED, when the last two changes deliberately did
not bump it.** Both precedents turn on the same test and both passed it:
`RetiredEgressTypes` (D29) — "no stored field changes meaning, nothing is
migrated, and a bump would only make configs written by this build unreadable to
an older daemon for no gain" — and `Device.Blocklists` — "purely additive, no
existing field is reinterpreted". This change fails that test on both counts: the
**absence** of `fetch_via` changes meaning, and there **is** something to migrate.
The bump is the mechanism that guarantees the rewrite happens before any reader
can act on the new meaning; without it a v2 config is simply handed to a v3
parser, and there is no second chance to tell the two cases apart afterwards. The
cost is the usual one — a config written by this build is refused by an older
daemon — and it is the correct signal, because an older daemon does not
understand `globals.fetch_detour` at all and would quietly fetch everything
direct while the panel showed otherwise.
**The panel now has to write `fetch_via` explicitly.** `panel/src/subEdit.ts`
wrote an explicitly-chosen "Direct" out as *no option at all*, which was
indistinguishable from "unset" and is now a different setting. Until it emits the
value, a panel save silently converts an explicit direct fetch back into
"inherit" — and that is a real change of destination the moment a general
`fetch_detour` exists.
## D31 — The subscription cache is the DEFAULT source; the newest copy wins, not the persistent one
Decided 2026-07-28. Two findings about `shater/model/subcache.go`, one of them a
live defect.
**"Cache first, refresh later" is not a fallback — it is the only path.** The
daemon's start sequence is `Migrate()` → `ReadUCI()` → apply, and `ReadUCI` is
`ParseUCIExport` + `MergeSubCaches`. **No fetch happens on that path at all**;
the refresh is a separate cron job (`/etc/init.d/shater-cron`). So a boot with no
network is the ordinary case and the cache is what the node inventory *is*, not
what it degrades to. That was already true and is now pinned by a test that boots
a config plus cache files with no network and asserts the whole inventory —
including each node's hand-set `Enabled` flag, because a node parked by hand that
comes back enabled after a reboot puts traffic on it again.
**A failed fetch cannot cost the inventory, and that is layered.**
`subscribe.Fetch` returns an error for a non-2xx status, an empty body and a body
read that ends early (a connection torn mid-stream);
`subscribe.UpdateSubscription` refuses a body that yields zero nodes and leaves
the model untouched; and every caller reaches `SaveSubCache` only after both have
succeeded. `SaveSubCache` itself writes temp-file-then-rename, so a write that
fails at any step leaves the previous file exactly as it was. A refresh that
cannot be persisted therefore costs a **stale** inventory, never an empty one.
**THE DEFECT: the stale copy was shadowing the fresh one.** `SaveSubCache`
degrades to tmpfs precisely when the persistent home *refused* the write — a full
or read-only `/overlay`, which is a routine OpenWrt state. At that instant the
persistent copy is by definition the stale one. But the load rule was "the
earlier (higher-priority) dir wins", so `sub update` fetched successfully,
reported "N nodes cached", wrote them to `/tmp/shater-subs` — and every reader
went on serving the **old** set out of `/etc/shater/subs`, with nothing anywhere
saying the refresh had not taken effect. An instrument that reads identically in
the working and the broken case.
**THE FIRST FIX WAS WRONG, AND IT IS WORTH RECORDING WHY.** The rule became "the
file with the newer modification time wins". It passed on the Windows dev host 5
runs out of 5 and **failed on Linux 5 runs out of 5** — that is, it did not fix
the defect at all on the only platform that matters, and the green run was taken
on the wrong one. Two independent reasons, either fatal on its own:
- **Resolution.** Two writes inside one operation fall in the same kernel timer
tick and receive *bit-identical* timestamps. Measured in the CI container: the
persistent and the tmpfs copy came out at the same `UnixNano` **to the digit**,
so `fresh.After(stale)` was `false` and the first-visited (persistent, stale)
copy won. This is not a rare tie — for two small consecutive writes it is the
normal outcome.
- **No RTC.** This router boots with its clock at the epoch until NTP syncs, over
the very SIM uplink that drops. A cache written before a sync carries a 1970
stamp and loses to an older file written after a previous sync, so mtime
ordering can *invert* across a reboot. The codebase already refuses to act on
an unsynced clock elsewhere (`alert/expiry.go`); the cache must not depend on
one either.
**The rule is now "the highest generation wins," and the generation is carried
inside the file** (`"seq"` in `subs/*.json`). Every write takes the highest
generation found in any candidate directory and adds one, so "this supersedes
that" is a fact about the data, not about the filesystem. A tie — which now means
only "both files predate the counter", i.e. both are 0 — keeps the persistent
copy, exactly the behaviour those files had before. A reboot needs no special
case: tmpfs is empty then, so the persistent copy is the only candidate and the
cold-start-on-cache guarantee is unchanged.
**The cache format version is NOT bumped.** A new field is transparent in both
directions — `decodeSubCache` uses a plain `json.Unmarshal`, so an older daemon
ignores `seq` and a newer one reads `0` — whereas bumping it would make every
cache file already on the router unreadable, which is precisely the cold-start
wipe this file exists to prevent.
**One case now costs a write that used to be skipped**, deliberately: when the
node set is unchanged *but the copy in force is the tmpfs one*, the persistent
home is rewritten anyway with a higher generation, so authority moves back to the
home that survives a reboot. Skipping it is how a router that filled its overlay
once would stay permanently one reboot away from serving a stale set.
**A superseded copy is deleted best-effort only.** The generation counter has
already decided the outcome, so a cleanup that fails costs a few KB and nothing
else — which is the property mtime did not have, where a failed cleanup was
indistinguishable from a fresh write.
**`writeFileAtomic` became a seam** (`var`, defaulting to the real
implementation), and `subCacheDirPersistent`/`subCacheDirFallback` became `var`s,
for the reason `migrate.go` already gives for `statBackup`/`writeBackupFile`: the
degraded two-directory state is exactly what `SHATER_SUBS_DIR` cannot produce (it
collapses the chain to one directory) and what a temp directory cannot produce on
demand. Each test built on that seam carries its own positive control — the same
instrument, unbroken, is shown to give the other answer — because "no damage
found" by an instrument that could not have seen damage is not a result.
**`TestAuthorityIgnoresTheClock` is the guard that makes the mtime mistake
unrepeatable.** It hands the *stale* copy every advantage a clock could give it —
mtime in the future for the stale file, the epoch for the fresh one — asserts
first that the inversion really is in place, and then requires the fresh set to
be returned anyway. Any implementation that consults the clock, coarse or fine,
strict or not, fails it on every platform. **Every run of this package is now
taken in Linux under Docker**; a Windows-only green run is not evidence for
anything here, and that is the concrete lesson this entry paid for.
**Known and deliberately NOT changed: `SyncSubCaches` writes what it is given.**
It makes the cache files equal to the `FromSub` nodes carried in the model,
*including* emptying one whose nodes are all gone — which is what makes "delete
the last node" stick in the panel. The consequence is that a caller which builds
a model without `MergeSubCaches` (`ParseUCIExport` alone carries no `FromSub`
nodes) and hands it to `SyncSubCaches` empties every cache the router has,
silently and totally. The panel does not do this — it PUTs back the merged model
it GETs — but nothing in the package enforces it. It is pinned by a test that
states the hazard rather than fixed by a heuristic, because the only available
heuristic ("nobody would delete every node of every subscription at once") is a
guess about intent.
+152
View File
@@ -857,13 +857,27 @@ export interface Globals {
FwmarkBase: number
TableBase: number
ConfirmTimeout: number
/**
* The `config resolver` every DNS query falls to when no DNS rule matches.
*
* STORED, NOT NECESSARILY EFFECTIVE: the active profile may override it
* (Profile.ResolverDefault). Read the live answer through
* {@link getEffectiveConfig} — a page that renders this string as "the
* resolver in use" is the defect that endpoint exists to fix.
*/
ResolverDefault: string
/** The resolver tried when the default fails. Same profile caveat as
* ResolverDefault (Profile.ResolverFallback). */
ResolverFallback: string
/**
* `config resolver` used to resolve the SERVER DOMAINS of proxy configs
* (vless/awg endpoints) — sing-box route.default_domain_resolver, a bootstrap
* resolver that is always direct. "" ⇒ not set (model.Globals.EndpointResolver,
* UCI `endpoint_resolver`).
*
* Same profile caveat as ResolverDefault, and this is the field it was caught
* on: the DNS page printed `local` from here while the router, under the
* active `mobile-uplink` profile, was resolving endpoints through `yandex`.
*/
EndpointResolver: string
ProbeURL: string
@@ -933,6 +947,33 @@ export interface Globals {
* is in sole charge.
*/
UntunnelableEgress?: string
/**
* The route the daemon FETCHES LIST DATA over — every blocklist, allowlist,
* rule-set and geoip/geosite `.srs` (model.Globals.FetchDetour, UCI
* `fetch_detour`). Same target grammar the routing rules use, so the same
* picker reads it: `direct` | `block` | `node:<n>` | `group:<n>` |
* `egress:<n>` | `chain:<n>`.
*
* "" ⇒ `direct`, which is what every build before this field did with no way
* to say otherwise (a `filterFetchDetour = tagDirect` constant in
* generate/dnsfilter.go). It stops being a detail on an uplink that blocks the
* hosts the lists live on: a list that cannot be fetched is not applied, and
* the engine reports it BY NAME ("…was omitted and matches NOTHING until it
* loads") rather than failing the apply — so the router keeps running with a
* filter that filters nothing.
*
* Two things the panel must not conflate with it. The active profile can
* override it (Profile.FetchDetour), so this string is the STORED value, not
* necessarily the effective one — read {@link getEffectiveConfig}. And
* {@link Subscription.FetchDetour} is a different, narrower setting: a
* subscription that names its own fetch route keeps it.
*
* CIRCULARITY IS POSSIBLE AND IS NOT DETECTED HERE: pointing this at a group
* or chain whose own routing depends on a list fetched through it is a knot
* the daemon has to untie, not the picker. The panel offers the targets; it
* does not promise they will work.
*/
FetchDetour?: string
/**
* Where a `source=geosite` / `source=geoip` list (a {@link Ruleset}, a
* {@link Blocklist} or an {@link Allowlist}) fetches its data from
@@ -1274,8 +1315,42 @@ export interface Profile {
// either key fails the whole write with 400.
EnableRules?: string[] | null
DisableRules?: string[] | null
/**
* Per-profile override of {@link Globals.ResolverDefault} — the resolver
* unmatched queries fall to while this profile is active
* (model.Profile.ResolverDefault, UCI `resolver_default`).
*
* "" ⇒ NO OVERRIDE, and inheritance is PER FIELD: a profile may set only the
* fallback and keep the global default. So "empty" and "empty on purpose" are
* the same thing here, and the editor offers no way to force a global's value
* — leaving the override off already does that.
*
* Why it exists: a mobile uplink that refuses :443 to 9.9.9.9 and 1.1.1.1
* (measured on this router) kills DoH outright, and with it the resolution of
* every proxy server domain. The profile that matches that uplink switches the
* resolvers to ones the carrier does route.
*/
ResolverDefault?: string
/** Per-profile override of {@link Globals.ResolverFallback}. Same per-field,
* ""-means-inherit contract as {@link ResolverDefault}
* (model.Profile.ResolverFallback, UCI `resolver_fallback`). */
ResolverFallback?: string
/** Per-profile override of the endpoint resolver (keyed by active WAN: SIM→yandex, WiFi→DoH). "" ⇒ no override (model.Profile.EndpointResolver, UCI `endpoint_resolver`). */
EndpointResolver?: string
/**
* Per-profile override of {@link Globals.FetchDetour} — where list data is
* fetched from while this profile is active (model.Profile.FetchDetour, UCI
* `fetch_detour`). Same target grammar, same ""-means-inherit contract.
*
* Note the asymmetry with the resolvers: for a detour `""` and `direct` are
* NOT the same input. `""` inherits whatever globals says; an explicit
* `direct` forces the plain WAN even when globals routes fetches through a
* tunnel. The panel must keep those two apart, and the daemon does it for
* them: {@link getEffectiveConfig} compares an unset detour and an explicit
* `direct` as ONE route, so a profile spelling out `direct` over empty globals
* correctly reports nothing overridden.
*/
FetchDetour?: string
}
/**
@@ -1995,6 +2070,83 @@ export function getRulesReachability(): Promise<RulesReachability> {
return MOCK ? mock().getRulesReachability() : req<RulesReachability>('api/rules/reachability')
}
/**
* One profile-overridable scalar, resolved by the DAEMON.
*
* The redundancy between `Profile` and `Overridden` is deliberate and pinned by
* a server-side test: `Profile !== ''` is exactly `Overridden`. So a caller that
* renders on either one behaves identically, and the permanent noise line this
* shape exists to prevent cannot come back through a careless reading.
*/
export interface EffectiveOverride {
/** What the engine takes. `''` is a legitimate answer, not an error. */
Value: string
/**
* The value in `config globals` that this verdict was computed against —
* i.e. what was ON DISK when the daemon answered.
*
* Not the same thing as the value in the input on screen: the page holds an
* optimistic copy and may carry an edit that has not been PUT yet. Comparing
* the two is the only way a row can notice it is showing a verdict about a
* different configuration than the one it is drawing.
*/
Stored: string
/** The active profile responsible for the DIFFERENCE, else `''`. Bare name —
* no `profile:` prefix to strip. */
Profile: string
/** True only when `Value` differs from `Stored`. The whole condition for
* saying anything at all. */
Overridden: boolean
}
/**
* GET /api/config/effective — the four profile-overridable scalars as the ENGINE
* reads them.
*
* WHY A SEPARATE REQUEST AND NOT A FIELD ON /api/config: the panel PUTs that
* body back verbatim and `handleConfigPut` decodes with `DisallowUnknownFields`,
* so a derived field there answers 400 — the daemon's author proved it by
* mutation, planting a harmless derived key and getting
* `decode model: json: unknown field "ActiveProfileName"`. The same reasoning is
* already written down for /api/rules/reachability.
*
* WHY THE PANEL DOES NOT COMPUTE THIS: it used to (panel/src/effective.ts), and
* that was a second copy of "which profile is active" and "whose value wins",
* written in another language and shipped on its own schedule. Two
* implementations of one rule drift, and the drift IS the defect this answers.
*
* CONSEQUENCE WORTH KNOWING: the verdict is computed from the config ON DISK, so
* typing a new value into a globals field does NOT move it. The line keeps
* naming the profile's value until the edit is saved — which is correct, because
* until then the engine is still using the profile's value. Do not "fix" that
* with a client-side recompute; that is the drift coming back.
*/
export interface EffectiveConfig {
/**
* The profile whose overrides apply, or `''` when none does. Reported
* INDEPENDENTLY of whether it overrode any of the four: "a profile is active
* and changed none of these" and "no profile is active" are different facts.
*/
ActiveProfile: string
ResolverDefault: EffectiveOverride
ResolverFallback: EffectiveOverride
EndpointResolver: EffectiveOverride
FetchDetour: EffectiveOverride
}
/**
* GET /api/config/effective — what the engine takes for the four scalars a
* profile can override.
*
* Rejects as an ApiError on a daemon that predates the endpoint (404). Callers
* must treat that as "no reading", never as "nothing is overridden": the whole
* point is that a stored value and an effective one can differ, and a panel that
* answers a question it could not measure is the defect, louder.
*/
export function getEffectiveConfig(): Promise<EffectiveConfig> {
return MOCK ? mock().getEffectiveConfig() : req<EffectiveConfig>('api/config/effective')
}
/** GET /api/ruleset/status — remote rule-set / blocklist freshness + rule counts. */
export function getRulesetStatus(): Promise<RulesetStatus[]> {
return MOCK ? mock().getRulesetStatus() : req<RulesetStatus[]>('api/ruleset/status')
+108
View File
@@ -0,0 +1,108 @@
import type { DetourCatalog } from '../detour'
/**
* The detour picker: `Direct` plus every group / chain / interface-egress / node
* the Model currently defines.
*
* One <select>, shared by every page that pins traffic to a route — a resolver's
* DNS path, an alert's delivery, a subscription's fetch, the list-fetch route.
* It was three byte-identical copies (DNS.tsx, Nodes.tsx, Alerts.tsx) before the
* fourth consumer arrived; the option labels are the words the operator learns
* the vocabulary from, so they have to be the SAME words on every page.
*
* `className` and `directLabel` are the only things a page varies. `directLabel`
* matters: under a proxy-fetch `Direct` is not "no preference", it is the plain
* WAN with the router's real address, and the page that means that says so.
*
* A stored value the catalog no longer knows is kept as a trailing
* `<option>… (missing)</option>` rather than silently reselecting the first
* entry — a picker that quietly rewrites a stale setting to `direct` would turn
* a visible misconfiguration into an invisible leak.
*/
export interface DetourSelectProps {
/** Canonical value — run the stored string through `canonDetour` first. */
value: string
catalog: DetourCatalog
/** `detourValues(catalog)` — what counts as still-resolvable. */
valid: Set<string>
disabled?: boolean
/** A save/apply is in flight; folded into `disabled`. */
busy?: boolean
ariaLabel: string
onChange: (v: string) => void
className?: string
directLabel?: string
/**
* Turns the picker into an OVERRIDE picker: adds a leading `<option value="">`
* with this label, meaning "not set here — inherit". Only for fields where
* empty and `direct` are different inputs (a profile override: `''` inherits
* whatever globals says, `direct` forces the plain WAN over a globals setting
* that tunnels). Leave it off and `''` is not a selectable state.
*/
inheritLabel?: string
}
export function DetourSelect({
value,
catalog,
valid,
disabled = false,
busy = false,
ariaLabel,
onChange,
className = 'fp-input',
directLabel = 'Direct (no proxy)',
inheritLabel,
}: DetourSelectProps) {
const missing = value !== '' && value !== 'direct' && !valid.has(value)
return (
<select
className={className}
value={value}
onChange={(e) => onChange(e.target.value)}
disabled={busy || disabled}
aria-label={ariaLabel}
>
{inheritLabel !== undefined && <option value="">{inheritLabel}</option>}
<option value="direct">{directLabel}</option>
{catalog.groups.length > 0 && (
<optgroup label="Groups">
{catalog.groups.map((g) => (
<option key={g} value={`group:${g}`}>
Group {g} (balancer)
</option>
))}
</optgroup>
)}
{catalog.chains.length > 0 && (
<optgroup label="Chains">
{catalog.chains.map((c) => (
<option key={c} value={`chain:${c}`}>
Chain {c}
</option>
))}
</optgroup>
)}
{catalog.egresses.length > 0 && (
<optgroup label="Interfaces / egresses">
{catalog.egresses.map((e) => (
<option key={e.name} value={`egress:${e.name}`}>
Interface/egress {e.name}
{e.type ? ` (${e.type})` : ''}
</option>
))}
</optgroup>
)}
{catalog.nodes.length > 0 && (
<optgroup label="Nodes">
{catalog.nodes.map((n) => (
<option key={n} value={`node:${n}`}>
Node {n}
</option>
))}
</optgroup>
)}
{missing && <option value={value}>{value} (missing)</option>}
</select>
)
}
+2
View File
@@ -17,6 +17,8 @@ export { Button } from './Button'
export type { ButtonProps } from './Button'
export { Select } from './Select'
export type { SelectProps, SelectOption } from './Select'
export { DetourSelect } from './DetourSelect'
export type { DetourSelectProps } from './DetourSelect'
export { ConfirmDialog, ConfirmProvider, useConfirm } from './ConfirmDialog'
export type { ConfirmDialogProps, ConfirmOptions, ConfirmTone } from './ConfirmDialog'
export { Clock } from './Clock'
+106
View File
@@ -0,0 +1,106 @@
// The detour vocabulary, in one place.
//
// A "detour" is the panel's word for a model TARGET STRING — `direct`,
// `node:<n>`, `group:<n>`, `egress:<n>`, `chain:<n>` — the same grammar
// generate.resolveTarget parses for rule targets, device targets, DNS-resolver
// detours, alert delivery, subscription fetches and (now) list fetches. One
// grammar, so one reader.
//
// These four helpers used to be COPIED into DNS.tsx, Nodes.tsx and Alerts.tsx,
// character for character. The copy in Alerts.tsx carried the standing
// instruction that made this file: "If a third consumer ever appears, promote
// them then." The third and fourth appeared (Globals.FetchDetour on the DNS
// page, Profile.FetchDetour on the Profiles page), so they are promoted. The
// duplication was never harmless: `canonDetour` decides whether a stored value
// is drawn as live or as `(missing)`, and three copies free to drift are three
// chances for one page to call a working route broken.
//
// This module is deliberately PURE (no JSX, no imports from ./api): it is what
// `node --test` can execute directly. The <select> that renders these options
// lives in components/DetourSelect.tsx.
/** The live targets a detour can point at, harvested from the Model. */
export interface DetourCatalog {
groups: string[]
chains: string[]
egresses: { name: string; type: string }[]
nodes: string[]
}
/** An empty catalog — every picker still offers `direct`. */
export const EMPTY_CATALOG: DetourCatalog = { groups: [], chains: [], egresses: [], nodes: [] }
/**
* Normalise a stored detour to a picker option value. Empty/`direct` ⇒
* `direct`; already-prefixed values (`group:`/`chain:`/`egress:`/`node:`) pass
* through; a bare legacy name is resolved against the catalog so a still-valid
* setup isn't mislabelled; anything unresolved is kept verbatim (shown stale).
*/
export function canonDetour(raw: string | undefined, cat: DetourCatalog): string {
const d = (raw ?? '').trim()
if (!d || d.toLowerCase() === 'direct') return 'direct'
if (/^(node|group|chain|egress):/i.test(d)) return d
if (cat.egresses.some((e) => e.name === d)) return `egress:${d}`
if (cat.groups.includes(d)) return `group:${d}`
if (cat.chains.includes(d)) return `chain:${d}`
if (cat.nodes.includes(d)) return `node:${d}`
return d
}
/** Every valid option value for a catalog, including `direct`. */
export function detourValues(cat: DetourCatalog): Set<string> {
const s = new Set<string>(['direct'])
for (const g of cat.groups) s.add(`group:${g}`)
for (const c of cat.chains) s.add(`chain:${c}`)
for (const e of cat.egresses) s.add(`egress:${e.name}`)
for (const n of cat.nodes) s.add(`node:${n}`)
return s
}
/** Describe a canonical detour value for a row readout. */
export function describeDetour(
canon: string,
cat: DetourCatalog,
valid: Set<string>,
): { direct: boolean; prefix: string; name: string; missing: boolean } {
if (canon === 'direct') return { direct: true, prefix: '', name: '', missing: false }
const i = canon.indexOf(':')
const kind = i === -1 ? '' : canon.slice(0, i)
const name = i === -1 ? canon : canon.slice(i + 1)
const missing = !valid.has(canon)
let prefix = 'via'
if (kind === 'group') prefix = 'via group'
else if (kind === 'chain') prefix = 'via chain'
else if (kind === 'node') prefix = 'via node'
else if (kind === 'egress') {
const eg = cat.egresses.find((e) => e.name === name)
prefix = eg?.type === 'interface' ? 'via interface' : 'via egress'
}
return { direct: false, prefix, name, missing }
}
/**
* One human-readable phrase for a canonical detour — "direct", "via group auto".
* Used where there is room for a sentence but not for a whole readout row.
*/
export function detourLabel(canon: string, cat: DetourCatalog, valid: Set<string>): string {
const d = describeDetour(canon, cat, valid)
if (d.direct) return 'direct'
return `${d.prefix} ${d.name}${d.missing ? ' (missing)' : ''}`
}
/** Harvest a catalog from the Model slices, in the shape every page passes. */
export function catalogOf(m: {
Groups?: { Name: string }[] | null
Chains?: { Name: string }[] | null
Egresses?: { Name: string; Type: string }[] | null
Nodes?: { Name: string }[] | null
}): DetourCatalog {
const arr = <T,>(a: T[] | null | undefined): T[] => (a ? a : [])
return {
groups: arr(m.Groups).map((g) => g.Name),
chains: arr(m.Chains).map((c) => c.Name),
egresses: arr(m.Egresses).map((e) => ({ name: e.Name, type: e.Type })),
nodes: arr(m.Nodes).map((n) => n.Name),
}
}
+160
View File
@@ -0,0 +1,160 @@
// Reading the daemon's effective-config answer.
//
// Run with `npm test`. Plain module, no React, no DOM.
//
// This file replaces effective.test.ts, which tested a RULE the panel no longer
// owns — which profile is active, whose value wins. That rule is
// model.ResolveActiveProfile + model.Override, answered by
// GET /api/config/effective, and pinned by Go tests next to the code that
// implements it. What is left to test here is the reading, and it has its own
// ways to be wrong.
//
// WHAT THESE PROTECT, in the order they matter:
//
// 1. NO READING IS NOT "NOTHING IS OVERRIDDEN". A daemon that predates the
// endpoint 404s, and a request can fail. Both are easy to write as `null ⇒
// draw nothing`, which is indistinguishable on screen from "the stored
// value is what runs" — the original defect, restored, in the exact
// situation where the panel has least right to an opinion. `unknown` is a
// third state and stays one.
// 2. `Overridden` IS THE WHOLE CONDITION. Rendering whenever a profile is
// merely active brings back the permanent "in effect now: <same thing>"
// line under every field. The server pins `Profile !== '' ⟺ Overridden`
// precisely so both readings behave alike; this checks the reading the
// panel actually uses.
// 3. THE VERDICT KNOWS WHICH STORED VALUE IT JUDGED. `Stored` is what was on
// disk when the daemon answered. When the input on screen has moved on, the
// row would otherwise pair a fresh field with an old verdict and let it
// read as one statement.
// 4. …BUT NOT WHILE THE ANSWER IS IN FLIGHT. Every save moves the on-screen
// value first and the verdict a moment later, so an unsuppressed stale mark
// would fire on every single edit — and a warning that appears routinely is
// one nobody reads on the day it means something.
// 5. CONTROL: A REAL OVERRIDE IS STILL REPORTED, with the production values
// that started all of this. A reader that answered `none` to everything
// would satisfy 1–4 read carelessly.
import { test } from 'node:test'
import assert from 'node:assert/strict'
import type { EffectiveConfig, EffectiveOverride } from './api.ts'
import { activeProfileName, effectiveLine } from './effectiveView.ts'
const ov = (o: Partial<EffectiveOverride> = {}): EffectiveOverride => ({
Value: '',
Stored: '',
Profile: '',
Overridden: false,
...o,
})
/** The production shape from the bug report: globals say `local`, the engine
* uses `yandex` because `mobile-uplink` is pinned. */
const PROD: EffectiveConfig = {
ActiveProfile: 'mobile-uplink',
ResolverDefault: ov({ Value: 'yandex', Stored: 'quad9', Profile: 'mobile-uplink', Overridden: true }),
ResolverFallback: ov({ Value: 'cloudflare', Stored: 'cloudflare' }),
EndpointResolver: ov({ Value: 'yandex', Stored: 'local', Profile: 'mobile-uplink', Overridden: true }),
FetchDetour: ov({ Value: 'group:sim-bypass', Stored: 'direct', Profile: 'mobile-uplink', Overridden: true }),
}
// ---- 1. no reading is its own answer ---------------------------------------
test('with no reading every field reports unknown, never "not overridden"', () => {
for (const f of ['ResolverDefault', 'ResolverFallback', 'EndpointResolver', 'FetchDetour'] as const) {
assert.deepEqual(effectiveLine(null, f, 'quad9'), { kind: 'unknown' }, f)
}
assert.equal(activeProfileName(null), null, 'and "which profile" is unknown too, not ""')
})
test('a response missing a field is unknown for that field, not silence', () => {
// A daemon that grew the endpoint before it grew the fourth scalar.
const partial = { ActiveProfile: 'mobile-uplink', ResolverDefault: PROD.ResolverDefault } as unknown as EffectiveConfig
assert.deepEqual(effectiveLine(partial, 'FetchDetour', 'direct'), { kind: 'unknown' })
assert.equal(effectiveLine(partial, 'ResolverDefault', 'quad9').kind, 'in-effect')
})
// ---- 2. Overridden is the whole condition -----------------------------------
test('an active profile that changed nothing produces no lines at all', () => {
const quiet: EffectiveConfig = {
ActiveProfile: 'evening-direct',
ResolverDefault: ov({ Value: 'quad9', Stored: 'quad9' }),
ResolverFallback: ov({ Value: 'cloudflare', Stored: 'cloudflare' }),
EndpointResolver: ov({ Value: '', Stored: '' }),
FetchDetour: ov({ Value: 'direct', Stored: 'direct' }),
}
for (const f of ['ResolverDefault', 'ResolverFallback', 'EndpointResolver', 'FetchDetour'] as const) {
assert.deepEqual(effectiveLine(quiet, f, quiet[f].Stored), { kind: 'none' }, f)
}
// …and the profile is still NAMED. "Active and changed nothing" and "none
// active" are different facts and the page says which.
assert.equal(activeProfileName(quiet), 'evening-direct')
})
test('no active profile is "" — an answer, distinct from no reading', () => {
const none: EffectiveConfig = { ...PROD, ActiveProfile: '' }
assert.equal(activeProfileName(none), '')
assert.notEqual(activeProfileName(none), activeProfileName(null))
})
// ---- 3. the verdict names the stored value it judged ------------------------
test('a field edited past the judged value is marked, not silently paired', () => {
// The daemon judged `local`; the operator has since typed `router-local` and
// not saved. The verdict is still true about disk, and says which disk.
const line = effectiveLine(PROD, 'EndpointResolver', 'router-local')
assert.deepEqual(line, {
kind: 'in-effect',
value: 'yandex',
profile: 'mobile-uplink',
stale: true,
})
})
test('whitespace is not a difference — trimming matches the daemon', () => {
const line = effectiveLine(PROD, 'EndpointResolver', ' local ')
assert.equal(line.kind === 'in-effect' && line.stale, false)
})
// ---- 4. …but not while the answer is in flight ------------------------------
test('the stale mark is suppressed while a refetch is pending', () => {
const line = effectiveLine(PROD, 'EndpointResolver', 'router-local', true)
assert.deepEqual(line, {
kind: 'in-effect',
value: 'yandex',
profile: 'mobile-uplink',
stale: false,
})
})
// ---- 5. control: a real override is still reported --------------------------
test('control — the defect that started this still reports, with both names', () => {
assert.deepEqual(effectiveLine(PROD, 'EndpointResolver', 'local'), {
kind: 'in-effect',
value: 'yandex',
profile: 'mobile-uplink',
stale: false,
})
assert.deepEqual(effectiveLine(PROD, 'FetchDetour', 'direct'), {
kind: 'in-effect',
value: 'group:sim-bypass',
profile: 'mobile-uplink',
stale: false,
})
// The one field the profile left alone stays quiet in the same response.
assert.deepEqual(effectiveLine(PROD, 'ResolverFallback', 'cloudflare'), { kind: 'none' })
})
test('an effective value of "" is a value, and is reported as one', () => {
// globals name a resolver, the profile clears it back to the engine default.
const cleared: EffectiveConfig = {
...PROD,
EndpointResolver: ov({ Value: '', Stored: 'local', Profile: 'mobile-uplink', Overridden: true }),
}
const line = effectiveLine(cleared, 'EndpointResolver', 'local')
assert.equal(line.kind, 'in-effect')
assert.equal(line.kind === 'in-effect' && line.value, '')
})
+81
View File
@@ -0,0 +1,81 @@
// How a page READS the daemon's effective-config answer. No rule, just a reading.
//
// The rule itself — which profile is active, whose value wins — used to live in
// this directory (panel/src/effective.ts) as a second implementation in a second
// language on a second release schedule. It is gone: GET /api/config/effective
// answers it now, from model.ResolveActiveProfile, the same call generate and
// netplane make. What is left here is the part that is genuinely the client's:
// deciding what to DRAW from an answer that may not have arrived.
//
// The one thing this module exists to stop is the panel answering a question it
// could not measure. A daemon that predates the endpoint 404s; a request can
// fail. Neither is "nothing is overridden" — the whole premise is that stored
// and effective can differ, so with no reading the honest output is a stated
// absence, not a confident silence.
import type { EffectiveConfig, EffectiveOverride } from './api'
/** The four fields the endpoint resolves, spelled as it spells them. */
export type EffectiveField =
| 'ResolverDefault'
| 'ResolverFallback'
| 'EndpointResolver'
| 'FetchDetour'
/**
* What to render under one globals field. A CLOSED set — the caller switches on
* `kind` and there is no fall-through.
*
* none — nothing to say: the value in the field is the value in force.
* unknown — no reading was obtained. The field may or may not be overridden
* and this panel cannot tell; say so rather than implying `none`.
* in-effect — a profile makes the engine use something else. `stale` marks the
* case where the verdict was computed against a DIFFERENT stored
* value than the one on screen, so the row is not quietly pairing
* a fresh input with an old verdict.
*/
export type EffectiveLine =
| { kind: 'none' }
| { kind: 'unknown' }
| { kind: 'in-effect'; value: string; profile: string; stale: boolean }
/**
* Decide the line for one field.
*
* `eff` is null when no reading was obtained (never fetched, or the request
* failed). `onScreenStored` is the value the input is currently showing, which
* is the panel's optimistic copy and can legitimately run ahead of disk.
*
* `pending` suppresses the stale mark while a refetch is in flight. Every save
* moves the on-screen value first and the verdict a moment later, so without it
* the mark would flash on every keystroke-and-save — and a warning that appears
* routinely is one nobody reads on the day it means something.
*/
export function effectiveLine(
eff: EffectiveConfig | null,
field: EffectiveField,
onScreenStored: string | undefined,
pending = false,
): EffectiveLine {
if (!eff) return { kind: 'unknown' }
const ov: EffectiveOverride | undefined = eff[field]
// A daemon that answers with the endpoint but not this field is the same
// situation as no answer: unmeasured, not unoverridden.
if (!ov) return { kind: 'unknown' }
if (!ov.Overridden) return { kind: 'none' }
return {
kind: 'in-effect',
value: ov.Value,
profile: ov.Profile,
stale: !pending && (onScreenStored ?? '').trim() !== ov.Stored,
}
}
/**
* The active profile, or `''` when none is — reported even when it overrode
* nothing, because the endpoint reports it that way and the two facts differ.
* `null` (no reading) is a third answer and stays distinguishable.
*/
export function activeProfileName(eff: EffectiveConfig | null): string | null {
return eff ? eff.ActiveProfile : null
}
+122 -15
View File
@@ -10,7 +10,7 @@ import { killSwitchClosed } from './planeState'
// The SAME field lists and matcher the page documents and the daemon implements —
// so a search in `?mock` cannot quietly be more (or less) generous than the real one.
import { connSearchFields, logSearchFields, rowMatches } from './logRoute'
import type { ApplyResult, ChainHealth, ChainHopHealth, ConnLogEntry, DiscoveredDevice, GroupHealth, GroupMemberHealth, GroupsHealth, GroupTestResult, GroupTestStart, GroupTestStatus, Interface, Model, Profile, QueryLogEntry, RuleReach, RulesReachability, RulesetCategories, RulesetCheck, RulesetStatus, Stats, StatsLogPage, StatsLogQuery, Status, StatusWarning, TestKind, Traffic } from './api'
import type { ApplyResult, ChainHealth, ChainHopHealth, ConnLogEntry, DiscoveredDevice, EffectiveConfig, EffectiveOverride, GroupHealth, GroupMemberHealth, GroupsHealth, GroupTestResult, GroupTestStart, GroupTestStatus, Interface, Model, Profile, QueryLogEntry, RuleReach, RulesReachability, RulesetCategories, RulesetCheck, RulesetStatus, Stats, StatsLogPage, StatsLogQuery, Status, StatusWarning, TestKind, Traffic } from './api'
/** One URL knob, safe to read before `location` exists (SSR-less builds/tests). */
function mockParam(name: string): string | null {
@@ -53,9 +53,19 @@ const CONFIG: Model = {
FwmarkBase: 0x2000,
TableBase: 0x2000,
ConfirmTimeout: 90,
// Three resolver slots and the list-fetch route, ALL of them overridable by
// the active profile below — which is the point of these particular values.
// `mobile-uplink` (pinned as ActiveProfile) repoints the default, the
// endpoint resolver and the fetch route, and says NOTHING about the
// fallback. So `?mock` renders the DNS card in both states at once: three
// rows carrying an "in effect now" line, one row quiet because nothing
// overrode it. A fixture where every row is overridden would hide the bug
// that a permanently-visible line is no better than no line at all.
ResolverDefault: 'cloudflare-doh',
ResolverFallback: 'router-local',
EndpointResolver: '',
// '' is `direct` — what every build did before the field existed.
FetchDetour: '',
ProbeURL: 'https://www.gstatic.com/generate_204',
ProbeInterval: '60s',
SchemaVersion: 2,
@@ -157,6 +167,13 @@ const CONFIG: Model = {
// Provider reported NOTHING (all zeros) ⇒ the honest "not reported" rendering,
// never a 0-of-0 bar.
{ Name: 'backup', Enabled: false, URL: 'https://sub.example.org/link', Format: 'clash', UpdateInterval: '24h', FetchVia: 'proxy', FetchDetour: 'group:auto' },
// The three `fetch_via` states are all present in this fixture on purpose:
// `backup` is an explicit `proxy`, `legacy` below is an explicit `direct`
// (what the v2→v3 migration stamped on every existing subscription, so their
// behaviour did not change under it), and `primary` above leaves the option
// ABSENT — the new third state, where the general fetch route decides. A
// fixture missing the absent one would let the two-state reading come back
// unnoticed, which is exactly how it survived this long.
// ALREADY EXPIRED, and on an unlimited/unreported plan (UserTotal 0 with real
// traffic counters). A valid state, not an error — exercises both edge cases.
{
@@ -164,6 +181,9 @@ const CONFIG: Model = {
Enabled: false,
URL: 'https://sub.example.com/old',
Format: 'auto',
// Explicitly pinned in the clear — stays direct even once the general
// fetch route sends everything else through a tunnel.
FetchVia: 'direct',
UserUpload: 1_073_741_824,
UserDownload: 15_032_385_536,
UserTotal: 0,
@@ -278,6 +298,11 @@ const CONFIG: Model = {
{ Name: 'cloudflare-doh', Type: 'doh', Address: 'https://1.1.1.1/dns-query', Detour: 'tunnel' },
{ Name: 'router-local', Type: 'local' },
{ Name: 'fakeip-pool', Type: 'fakeip', Pool: '198.18.0.0/15' },
// The one a mobile profile switches to. Modelled on the measurement that
// started this: the carrier refuses :443 to 1.1.1.1 and 9.9.9.9 but routes
// 77.88.8.8:53 fine, so plain DNS to an address it does carry is the only
// resolver that works on that uplink.
{ Name: 'carrier-plain', Type: 'plain', Address: '77.88.8.8' },
],
// Which resolver answers what, first match wins by ascending Order. Covers all
// three matcher shapes: domains only, source only, and both together.
@@ -364,8 +389,23 @@ const CONFIG: Model = {
MatchIface: ['wan1'],
EnableRules: ['default-tunnel'],
DisableRules: ['block-ads', 'ru-bypass', 'private-direct'],
// The scalar overrides, chosen to exercise PER-FIELD inheritance rather
// than to look tidy: default + endpoint + fetch route are repointed, the
// FALLBACK IS DELIBERATELY ABSENT and keeps the globals value
// (`router-local`). A fixture that set all four would pass a panel that
// wrongly treats "profile has any override" as "profile overrides
// everything" — the exact mistake this contract invites.
ResolverDefault: 'carrier-plain',
EndpointResolver: 'carrier-plain',
// A route, not a resolver: the lists are pulled through the tunnel because
// the uplink cannot be trusted to reach the hosts they live on.
FetchDetour: 'group:stealth',
},
{
// No scalar overrides at all — the control. Switch the override on the
// Profiles page to this one in `?mock` and every "in effect now" line on
// the DNS page must DISAPPEAR. If one survives, the page is drawing the
// line from "a profile is active" rather than from a real difference.
Name: 'evening-direct',
Enabled: true,
Priority: 10,
@@ -455,6 +495,86 @@ const RULESET_STATUS: RulesetStatus[] = [
{ tag: 'al-work-allow', name: 'work-allow', category: '', kind: 'allowlist', remote: true, last_updated: '', interval_seconds: 21_600, rule_count: 0 },
]
/**
* model.ResolveActiveProfile, as the daemon runs it: a pin naming an existing,
* ENABLED profile wins outright; otherwise auto-select the highest Priority among
* enabled profiles, ties by Name, SKIPPING any with an iface condition — those
* need live network state, and the WAN watcher expresses its verdict as the pin.
*
* One copy here, used by both endpoints that depend on it. Two would be the same
* mistake the panel just deleted from its own source tree, only in the fixtures.
*/
function resolveActiveProfile(): Profile | null {
const profiles = CONFIG.Profiles ?? []
const pinned = String(CONFIG.Globals?.ActiveProfile ?? '').trim()
const hit = profiles.find((p) => p.Enabled && p.Name === pinned)
if (hit) return hit
let best: Profile | null = null
for (const p of profiles) {
if (!p.Enabled || (p.MatchIface ?? []).length > 0) continue
const pp = p.Priority ?? 0
const bp = best?.Priority ?? 0
if (!best || pp > bp || (pp === bp && p.Name < best.Name)) best = p
}
return best
}
/**
* GET /api/config/effective. Mirrors shater/panel/effective.go exactly, including
* the two things it is easy to get almost right:
*
* - `Overridden` is `Value !== Stored`, NOT "the profile set something". A
* profile pinning the value globals already carries has changed nothing, and
* must not produce a permanent "in effect now: <same>" line.
* - the fetch detour compares through `canonFetchDetour`: unset and an explicit
* `direct` are ONE route. Resolver names compare verbatim (the generator
* matches them case-sensitively, so folding case here would call `Quad9` and
* `quad9` the same setting when the engine will not).
*/
export async function getEffectiveConfig(): Promise<EffectiveConfig> {
await wait(60)
// `?mock&noeffective=1` — the daemon that predates this endpoint, or a read
// that fails. Reachable on purpose: "no reading" must NOT look like "nothing
// is overridden", and the only way to check that on screen is to be able to
// produce it. Without a knob the branch ships untested and its whole point is
// that it is indistinguishable from the bug when it is wrong.
if (mockParam('noeffective') === '1') {
throw new ApiErrorLike(404, 'not found')
}
const prof = resolveActiveProfile()
const g = CONFIG.Globals
const canonDetour = (v: string): string => {
const s = (v ?? '').trim()
return !s || s.toLowerCase() === 'direct' ? 'direct' : s
}
const verbatim = (v: string): string => (v ?? '').trim()
const one = (
stored: string | undefined,
override: string | undefined,
canon: (v: string) => string,
): EffectiveOverride => {
const base = (stored ?? '').trim()
const raw = (override ?? '').trim()
const value = prof && raw !== '' ? raw : base
const differs = prof !== null && raw !== '' && canon(value) !== canon(base)
return {
Value: value,
Stored: base,
Profile: differs && prof ? prof.Name : '',
Overridden: differs,
}
}
return {
ActiveProfile: prof ? prof.Name : '',
ResolverDefault: one(g.ResolverDefault, prof?.ResolverDefault, verbatim),
ResolverFallback: one(g.ResolverFallback, prof?.ResolverFallback, verbatim),
EndpointResolver: one(g.EndpointResolver, prof?.EndpointResolver, verbatim),
FetchDetour: one(g.FetchDetour, prof?.FetchDetour, canonDetour),
}
}
/** GET /api/rules/reachability. Mirrors the daemon's analysis over CONFIG.Rules:
* a rule with no conditions is the router's default, and the LAST such rule by
* Order wins — every earlier one can never apply. It reads the live CONFIG so
@@ -468,20 +588,7 @@ export async function getRulesReachability(): Promise<RulesReachability> {
await wait(60)
const rules = CONFIG.Rules ?? []
// A pin naming an existing, ENABLED profile wins outright. Otherwise auto-select:
// highest Priority among enabled profiles, ties by Name, skipping any with an
// iface condition (the WAN watcher owns those and expresses its verdict as the pin).
const profiles = CONFIG.Profiles ?? []
const pinned = String(CONFIG.Globals?.ActiveProfile ?? '').trim()
let prof: Profile | null = profiles.find((p) => p.Enabled && p.Name === pinned) ?? null
if (!prof) {
for (const p of profiles) {
if (!p.Enabled || (p.MatchIface ?? []).length > 0) continue
const pp = p.Priority ?? 0
const bp = prof?.Priority ?? 0
if (!prof || pp > bp || (pp === bp && p.Name < prof.Name)) prof = p
}
}
const prof = resolveActiveProfile()
// Enable first, then Disable, so a name in both ends up disabled (Disable wins).
const effective = rules.map((r) => Boolean(r.Enabled))
if (prof) {
+9 -129
View File
@@ -1,6 +1,6 @@
import './Alerts.css'
import { useCallback, useMemo, useState } from 'react'
import { Button, Toggle, useConfirm } from '../components'
import { Button, DetourSelect, Toggle, useConfirm } from '../components'
import type { Alert, Model } from '../api'
import {
buildAlert,
@@ -12,6 +12,8 @@ import {
TYPE_SWITCH_NOTE,
} from '../alertEdit'
import type { AlertDraft, AlertType } from '../alertEdit'
import type { DetourCatalog } from '../detour'
import { canonDetour, describeDetour, detourValues } from '../detour'
// The Alerts section — out-of-band notifications (Telegram bot / webhook) for
// kill-switch trips, apply failures, new devices and subscription expiry. It
@@ -72,62 +74,10 @@ function uniqueName(base: string, taken: Set<string>): string {
return `${seed}-${i}`
}
/** The live targets an alert's delivery can be pinned to (the picker). */
interface DetourCatalog {
groups: string[]
chains: string[]
egresses: { name: string; type: string }[]
nodes: string[]
}
/**
* Normalise a stored `Via` to a picker option value. Empty/`direct` ⇒
* `direct`; already-prefixed values (`group:`/`chain:`/`egress:`/`node:`) pass
* through; a bare legacy name is resolved against the catalog so a still-valid
* setup isn't mislabelled; anything unresolved is kept verbatim (shown stale).
*/
function canonDetour(raw: string | undefined, cat: DetourCatalog): string {
const d = (raw ?? '').trim()
if (!d || d.toLowerCase() === 'direct') return 'direct'
if (/^(node|group|chain|egress):/i.test(d)) return d
if (cat.egresses.some((e) => e.name === d)) return `egress:${d}`
if (cat.groups.includes(d)) return `group:${d}`
if (cat.chains.includes(d)) return `chain:${d}`
if (cat.nodes.includes(d)) return `node:${d}`
return d
}
/** Every valid option value for a catalog, including `direct`. */
function detourValues(cat: DetourCatalog): Set<string> {
const s = new Set<string>(['direct'])
for (const g of cat.groups) s.add(`group:${g}`)
for (const c of cat.chains) s.add(`chain:${c}`)
for (const e of cat.egresses) s.add(`egress:${e.name}`)
for (const n of cat.nodes) s.add(`node:${n}`)
return s
}
/** Describe a canonical detour value for the row readout. */
function describeDetour(
canon: string,
cat: DetourCatalog,
valid: Set<string>,
): { direct: boolean; prefix: string; name: string; missing: boolean } {
if (canon === 'direct') return { direct: true, prefix: '', name: '', missing: false }
const i = canon.indexOf(':')
const kind = i === -1 ? '' : canon.slice(0, i)
const name = i === -1 ? canon : canon.slice(i + 1)
const missing = !valid.has(canon)
let prefix = 'via'
if (kind === 'group') prefix = 'via group'
else if (kind === 'chain') prefix = 'via chain'
else if (kind === 'node') prefix = 'via node'
else if (kind === 'egress') {
const eg = cat.egresses.find((e) => e.name === name)
prefix = eg?.type === 'interface' ? 'via interface' : 'via egress'
}
return { direct: false, prefix, name, missing }
}
// The detour helpers (DetourCatalog, canonDetour, detourValues,
// describeDetour) and the picker itself now live in ../detour and
// components/DetourSelect. The note above used to say "if a third consumer
// ever appears, promote them then" -- it did, so they were.
// ---- section ----------------------------------------------------------------
@@ -542,6 +492,7 @@ function AlertForm({
<label className="alr-resp">
<span className="alr-resp-label mono">Deliver via</span>
<DetourSelect
className="alr-select alr-detour-select"
value={via}
catalog={catalog}
valid={valid}
@@ -671,6 +622,7 @@ function AlertRow({
<label className="alr-detour">
<span className="alr-detour-label mono">Deliver via</span>
<DetourSelect
className="alr-select alr-detour-select"
value={canon}
catalog={catalog}
valid={valid}
@@ -708,78 +660,6 @@ function AlertRow({
)
}
/** The live delivery picker: option list built from the Model's targets. */
function DetourSelect({
value,
catalog,
valid,
busy,
disabled,
ariaLabel,
onChange,
directLabel = 'Direct (no proxy)',
}: {
value: string // canonical value
catalog: DetourCatalog
valid: Set<string>
busy: boolean
disabled: boolean
ariaLabel: string
onChange: (v: string) => void
directLabel?: string
}) {
const missing = value !== 'direct' && !valid.has(value)
return (
<select
className="alr-select alr-detour-select"
value={value}
onChange={(e) => onChange(e.target.value)}
disabled={busy || disabled}
aria-label={ariaLabel}
>
<option value="direct">{directLabel}</option>
{catalog.groups.length > 0 && (
<optgroup label="Groups">
{catalog.groups.map((g) => (
<option key={g} value={`group:${g}`}>
Group {g} (balancer)
</option>
))}
</optgroup>
)}
{catalog.chains.length > 0 && (
<optgroup label="Chains">
{catalog.chains.map((c) => (
<option key={c} value={`chain:${c}`}>
Chain {c}
</option>
))}
</optgroup>
)}
{catalog.egresses.length > 0 && (
<optgroup label="Interfaces / egresses">
{catalog.egresses.map((e) => (
<option key={e.name} value={`egress:${e.name}`}>
Interface/egress {e.name}
{e.type ? ` (${e.type})` : ''}
</option>
))}
</optgroup>
)}
{catalog.nodes.length > 0 && (
<optgroup label="Nodes">
{catalog.nodes.map((n) => (
<option key={n} value={`node:${n}`}>
Node {n}
</option>
))}
</optgroup>
)}
{missing && <option value={value}>{value} (missing)</option>}
</select>
)
}
function EmptyPlate({ title, body }: { title: string; body: string }) {
return (
<div className="alr-empty">
+68
View File
@@ -135,12 +135,75 @@
text-transform: uppercase;
color: var(--faint);
}
/* Which layer of config the field edits. Constant, not state-dependent: this
control always writes `config globals`, whether or not a profile is currently
overriding it. A label that appeared only when overridden would read as an
alert instead of a scope. */
.dns-scope {
display: inline-block;
margin-left: 6px;
padding: 1px 5px;
border: 1px solid var(--groove);
border-radius: 2px;
font-size: 8.5px;
letter-spacing: var(--track-label, 0.18em);
color: var(--faint);
text-transform: uppercase;
}
.dns-readout-item dd {
margin: 0;
font-size: 12px;
letter-spacing: 0.04em;
color: var(--ink);
font-variant-numeric: tabular-nums;
/* Column so the "in effect now" line can sit under its own control rather
than beside it — the field stays the thing you edit, the line stays a
readout. */
display: flex;
flex-direction: column;
align-items: flex-end;
gap: 5px;
}
/* ---- "in effect now": stored value ≠ value the engine takes ---- */
.dns-ineffect {
display: flex;
align-items: baseline;
flex-wrap: wrap;
justify-content: flex-end;
gap: 6px;
margin: 0;
max-width: 100%;
font-size: 11px;
line-height: 1.4;
}
.dns-ineffect-turn {
color: var(--groove);
}
.dns-ineffect-label {
font-size: 9px;
letter-spacing: var(--track-label, 0.18em);
text-transform: uppercase;
color: var(--faint);
}
/* The live value. `--led-on` is the page's existing word for "this is what is
actually happening" (see .dns-path-name) — semantic, never the brand accent. */
.dns-ineffect-value {
font-weight: 600;
color: var(--led-on);
overflow-wrap: anywhere;
}
.dns-ineffect-src {
font-family: var(--font-sans);
color: var(--dim);
}
/* The verdict was computed against a different stored value than the one in the
field — amber, because it is a real "these two are not talking about the same
config" and the fix is to save. Not crit: nothing is broken, it is one save
behind. */
.dns-ineffect-stale {
font-family: var(--font-sans);
color: var(--amber);
}
/* ---- known-lists quick add ---- */
@@ -596,6 +659,11 @@
padding: 4px 8px;
font-size: 11.5px;
}
/* The list-fetch picker shares the readout row but carries route labels
("Interface/egress wan (interface)"), which do not survive 12rem. */
.dns-readout-select--wide {
max-width: 17rem;
}
/* the per-resolver DNS-path picker sits inline in the row */
.dns-detour {
+222 -144
View File
@@ -1,9 +1,10 @@
import './DNS.css'
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { Button, CatSuggest, Led, SrcPicker, Toggle, useConfirm } from '../components'
import { Button, CatSuggest, DetourSelect, Led, SrcPicker, Toggle, useConfirm } from '../components'
import {
apply as apiApply,
getConfig,
getEffectiveConfig,
getRulesetStatus,
putConfig,
updateRuleset as apiUpdateRuleset,
@@ -14,6 +15,7 @@ import type {
Blocklist,
Complete,
DNSRule,
EffectiveConfig,
Model,
Resolver,
RulesetStatus,
@@ -32,6 +34,10 @@ import {
} from '../dnsListEdit'
import { everyLabel, relFetch } from '../format'
import { attachedDeviceNames, dnsFilterOffNote, listRowState } from '../deviceLists'
import type { DetourCatalog } from '../detour'
import { canonDetour, catalogOf, describeDetour, detourValues } from '../detour'
import { effectiveLine } from '../effectiveView'
import type { EffectiveField, EffectiveLine } from '../effectiveView'
// The DNS / Blocklists page is a thin editor over the desired-state Model —
// exactly like Nodes.tsx. Every edit rewrites the relevant slice in-place, PUTs
@@ -129,62 +135,9 @@ const RESOLVER_TYPES: ReadonlyArray<{ id: string; label: string }> = [
{ id: 'fakeip', label: 'Fake-IP' },
]
/** The live targets a resolver's DNS traffic can be pinned to (the picker). */
interface DetourCatalog {
groups: string[]
chains: string[]
egresses: { name: string; type: string }[]
nodes: string[]
}
/**
* Normalise a stored `Detour` to a picker option value. Empty/`direct` ⇒
* `direct`; already-prefixed values (`group:`/`chain:`/`egress:`/`node:`) pass
* through; a bare legacy name is resolved against the catalog so a still-valid
* setup isn't mislabelled; anything unresolved is kept verbatim (shown stale).
*/
function canonDetour(raw: string | undefined, cat: DetourCatalog): string {
const d = (raw ?? '').trim()
if (!d || d.toLowerCase() === 'direct') return 'direct'
if (/^(node|group|chain|egress):/i.test(d)) return d
if (cat.egresses.some((e) => e.name === d)) return `egress:${d}`
if (cat.groups.includes(d)) return `group:${d}`
if (cat.chains.includes(d)) return `chain:${d}`
if (cat.nodes.includes(d)) return `node:${d}`
return d
}
/** Every valid option value for a catalog, including `direct`. */
function detourValues(cat: DetourCatalog): Set<string> {
const s = new Set<string>(['direct'])
for (const g of cat.groups) s.add(`group:${g}`)
for (const c of cat.chains) s.add(`chain:${c}`)
for (const e of cat.egresses) s.add(`egress:${e.name}`)
for (const n of cat.nodes) s.add(`node:${n}`)
return s
}
/** Describe a canonical detour value for the row readout. */
function describeDetour(
canon: string,
cat: DetourCatalog,
valid: Set<string>,
): { direct: boolean; prefix: string; name: string; missing: boolean } {
if (canon === 'direct') return { direct: true, prefix: '', name: '', missing: false }
const i = canon.indexOf(':')
const kind = i === -1 ? '' : canon.slice(0, i)
const name = i === -1 ? canon : canon.slice(i + 1)
const missing = !valid.has(canon)
let prefix = 'via'
if (kind === 'group') prefix = 'via group'
else if (kind === 'chain') prefix = 'via chain'
else if (kind === 'node') prefix = 'via node'
else if (kind === 'egress') {
const eg = cat.egresses.find((e) => e.name === name)
prefix = eg?.type === 'interface' ? 'via interface' : 'via egress'
}
return { direct: false, prefix, name, missing }
}
// The detour vocabulary (DetourCatalog / canonDetour / detourValues /
// describeDetour) and the <select> that renders it now live in ../detour and
// components/DetourSelect — one copy, shared with Nodes, Alerts and Profiles.
// ---- page ------------------------------------------------------------------
@@ -206,6 +159,41 @@ export default function DNS() {
void loadConfig()
}, [loadConfig])
// ---- what the ENGINE takes, computed by the daemon ------------------------
//
// GET /api/config/effective, fetched beside the config and re-fetched after
// every PUT and every apply. It is a SEPARATE request because the panel PUTs
// /api/config's body back verbatim and that decoder has DisallowUnknownFields,
// so a derived field there would 400 the whole write.
//
// The verdict is computed from the config ON DISK. Typing into a globals field
// therefore does not move the "in effect now" line until the edit is saved —
// which is right: until then the engine really is still using the profile's
// value. There is deliberately no client-side recompute to "fix" that; the
// recompute was the bug (a second copy of the rule, free to drift), and it has
// been deleted.
//
// A failed fetch is NOT read as "nothing is overridden": `effective` stays
// null and every row reports an unknown instead. See ../effectiveView.
const [effective, setEffective] = useState<EffectiveConfig | null>(null)
const [effError, setEffError] = useState<string | null>(null)
const [effPending, setEffPending] = useState(true)
const loadEffective = useCallback(async () => {
setEffPending(true)
try {
setEffective(await getEffectiveConfig())
setEffError(null)
} catch (e) {
setEffective(null)
setEffError(errText(e))
} finally {
setEffPending(false)
}
}, [])
useEffect(() => {
void loadEffective()
}, [loadEffective])
// ---- toast + persistent apply banner --------------------------------------
const [toast, setToast] = useState<string | null>(null)
const toastTimer = useRef<number | undefined>(undefined)
@@ -233,6 +221,9 @@ export default function DNS() {
await putConfig(next)
setDirty(true)
flash(okMsg)
// The PUT changed what is on disk, which is what the effective verdict is
// computed from — so the verdict is now one request old. Re-ask.
void loadEffective()
return true
} catch (e) {
setConfig(prev) // revert the optimistic edit
@@ -242,7 +233,7 @@ export default function DNS() {
setSaving(false)
}
},
[config, flash],
[config, flash, loadEffective],
)
const applyNow = useCallback(async () => {
@@ -255,13 +246,17 @@ export default function DNS() {
setDirty(false)
flash(r.changed ? 'Applied — data plane reconciled' : 'Applied — already up to date')
void loadConfig()
// An apply can re-pin the active profile (the WAN watcher writes
// Globals.ActiveProfile), so every one of the four verdicts can move
// without any field on this page changing.
void loadEffective()
}
} catch (e) {
flash(`Apply failed — ${errText(e)}`)
} finally {
setApplying(false)
}
}, [flash, loadConfig])
}, [flash, loadConfig, loadEffective])
// ---- derived slices -------------------------------------------------------
const globals = config?.Globals
@@ -360,31 +355,48 @@ export default function DNS() {
const alNames = useMemo(() => new Set(allowlists.map((a) => a.Name)), [allowlists])
// Live detour catalog — the picker's option list is built from the Model.
const detourCatalog = useMemo<DetourCatalog>(
() => ({
groups: asArray(config?.Groups).map((g) => g.Name),
chains: asArray(config?.Chains as { Name: string }[] | null | undefined).map((c) => c.Name),
egresses: asArray(config?.Egresses).map((e) => ({ name: e.Name, type: e.Type })),
nodes: asArray(config?.Nodes).map((n) => n.Name),
}),
[config],
)
const detourCatalog = useMemo<DetourCatalog>(() => catalogOf(config ?? {}), [config])
const detourValid = useMemo(() => detourValues(detourCatalog), [detourCatalog])
const resolverNames = useMemo(() => new Set(resolvers.map((r) => r.Name)), [resolvers])
// ---- stored here, possibly overridden by the active profile ---------------
//
// The three resolver slots and the list-fetch route are all Globals scalars a
// profile may override, so the value in the field is the value SAVED, not
// necessarily the value running. This page used to print the saved one as the
// answer, which on the production router read `local` while the engine —
// under the profile the WAN watcher had pinned — resolved endpoints through
// `yandex`. Each row now carries a separate "in effect now" line whenever the
// two differ, and stays quiet when they agree.
//
// The verdict comes from the DAEMON (`effective` above). It is not recomputed
// here — that copy existed for one wave and is deleted.
const fetchCanon = canonDetour(globals?.FetchDetour, detourCatalog)
const lineFor = useCallback(
(field: EffectiveField, onScreen: string | undefined): EffectiveLine =>
effectiveLine(effective, field, onScreen, effPending),
[effective, effPending],
)
// A fake-IP resolver in the default or fallback slot silently disables failover:
// the engine refuses to `evaluate` past a fake-IP server, so the chain never
// forms. Name whichever slot is affected so the fix is obvious.
//
// Read from the EFFECTIVE values where there ARE any: a profile that swaps a
// plain resolver in front of a fake-IP one has removed this problem, and a
// profile that swaps a fake-IP one in has created it. With no reading, fall
// back to the stored names — a warning about the saved config is worth more
// than no warning, and the two agree except under an override.
const fakeipRole = useMemo<string | null>(() => {
const isFake = (name: string | undefined) =>
!!name && resolvers.some((r) => r.Name === name && r.Type === 'fakeip')
const d = isFake(globals?.ResolverDefault)
const f = isFake(globals?.ResolverFallback)
const d = isFake(effective?.ResolverDefault.Value ?? globals?.ResolverDefault)
const f = isFake(effective?.ResolverFallback.Value ?? globals?.ResolverFallback)
if (d && f) return 'default and fallback'
if (d) return 'default'
if (f) return 'fallback'
return null
}, [resolvers, globals?.ResolverDefault, globals?.ResolverFallback])
}, [resolvers, effective, globals?.ResolverDefault, globals?.ResolverFallback])
const busy = saving || applying
@@ -711,6 +723,21 @@ export default function DNS() {
[config, save],
)
// Where list DATA is fetched from — blocklists, allowlists, rule-sets and the
// geo .srs files. `direct` is stored as the empty string so an untouched
// config keeps the shape it has always had.
const setFetchDetour = useCallback(
(v: string) => {
if (!config) return
const next = v === 'direct' ? '' : v
void save(
{ ...config, Globals: { ...config.Globals, FetchDetour: next } },
next ? `List fetches → ${next}` : 'List fetches → direct',
)
},
[config, save],
)
// ---- DNS-rule mutations ---------------------------------------------------
// Rules are held sorted by ascending Order (first match wins). A DNS rule has no
// Name in the contract, so list POSITION is its only handle — every mutation
@@ -802,42 +829,82 @@ export default function DNS() {
</div>
<dl className="dns-readout">
<div className="dns-readout-item">
<dt>default resolver</dt>
<dt>
default resolver <span className="dns-scope">globals</span>
</dt>
<dd>
<RoleSelect
value={globals?.ResolverDefault ?? ''}
names={resolverNames}
busy={busy}
disabled={!config}
ariaLabel="Default resolver"
ariaLabel="Default resolver (globals)"
onChange={setResolverDefault}
/>
<InEffectLine
line={lineFor('ResolverDefault', globals?.ResolverDefault)}
what="Default resolver"
/>
</dd>
</div>
<div className="dns-readout-item">
<dt>fallback</dt>
<dt>
fallback <span className="dns-scope">globals</span>
</dt>
<dd>
<RoleSelect
value={globals?.ResolverFallback ?? ''}
names={resolverNames}
busy={busy}
disabled={!config}
ariaLabel="Fallback resolver"
ariaLabel="Fallback resolver (globals)"
onChange={setResolverFallback}
/>
<InEffectLine
line={lineFor('ResolverFallback', globals?.ResolverFallback)}
what="Fallback resolver"
/>
</dd>
</div>
<div className="dns-readout-item">
<dt>endpoint resolver</dt>
<dt>
endpoint resolver <span className="dns-scope">globals</span>
</dt>
<dd>
<RoleSelect
value={globals?.EndpointResolver ?? ''}
names={resolverNames}
busy={busy}
disabled={!config}
ariaLabel="Endpoint resolver"
ariaLabel="Endpoint resolver (globals)"
onChange={setEndpointResolver}
/>
<InEffectLine
line={lineFor('EndpointResolver', globals?.EndpointResolver)}
what="Endpoint resolver"
/>
</dd>
</div>
<div className="dns-readout-item">
<dt>
list fetch route <span className="dns-scope">globals</span>
</dt>
<dd>
<DetourSelect
className="dns-select dns-readout-select dns-readout-select--wide"
value={fetchCanon}
catalog={detourCatalog}
valid={detourValid}
busy={busy}
disabled={!config}
ariaLabel="List fetch route (globals)"
directLabel="Direct — plain WAN"
onChange={setFetchDetour}
/>
<InEffectLine
line={lineFor('FetchDetour', globals?.FetchDetour)}
what="List fetch route"
/>
</dd>
</div>
</dl>
@@ -847,6 +914,41 @@ export default function DNS() {
<span className="mono"> none</span> to use the engine default; point it at a plain,
direct resolver so it can bootstrap before any tunnel is up.
</p>
<p className="dns-filter-sub dns-filter-note">
The list fetch route carries the DOWNLOADS — every blocklist, allowlist, rule-set and
geoip/geosite file. Send them through a tunnel when the uplink blocks the hosts they
live on; a list that can’t be downloaded isn’t applied, and the engine says so by name
instead of failing the apply. Each subscription still has its own fetch route on the
Nodes page.
</p>
{/* Three states, not two. "A profile is active and changed none of
these" and "no profile is active" are different facts, and the
endpoint reports ActiveProfile independently of whether it
overrode anything — so the page can say which. The third is "we
could not ask", which must not be drawn as either of the other
two: the whole premise here is that stored and effective can
differ, so a panel that goes quiet when it cannot measure is the
original defect with better manners. */}
{effError !== null ? (
<p className="dns-filter-sub dns-filter-note">
Couldn’t read the effective values — {effError}. The fields above show what is
STORED; if a profile is overriding any of them, this panel can’t currently tell you
which.{' '}
<button className="linkish" onClick={() => void loadEffective()}>
Retry
</button>
</p>
) : effective && effective.ActiveProfile !== '' ? (
<p className="dns-filter-sub dns-filter-note">
Profile <span className="mono">{effective.ActiveProfile}</span> is active. It can
override any of the four above — where it does, the value in effect is named under
the field.{' '}
<a className="linkish" href="#/profiles">
Edit its overrides
</a>
.
</p>
) : null}
</div>
</div>
@@ -1754,75 +1856,49 @@ function ListRow({
)
}
/** The live DNS-path picker: option list built from the Model's targets. */
function DetourSelect({
value,
catalog,
valid,
busy,
disabled,
ariaLabel,
onChange,
directLabel = 'Direct (no proxy)',
}: {
value: string // canonical value
catalog: DetourCatalog
valid: Set<string>
busy: boolean
disabled: boolean
ariaLabel: string
onChange: (v: string) => void
directLabel?: string
}) {
const missing = value !== 'direct' && !valid.has(value)
/**
* The one line that stops this card lying: what the engine takes RIGHT NOW,
* when the active profile overrides the stored value in the field above.
*
* Rendered only on a difference. A permanent "in effect now: <same thing>"
* under every field would be noise, and noise is how the one row that matters
* gets skipped — which is exactly what happened before it existed, with no row
* at all.
*
* Informational, not alarming: a profile overriding a global is the feature
* working, not a fault. No crit/amber tokens here, no LED — a turnstile glyph,
* the value, and the profile that decided it, linked to where it is edited.
*/
function InEffectLine({ line, what }: { line: EffectiveLine; what: string }) {
// `unknown` draws nothing HERE: the card carries one retry line for the whole
// group rather than four copies of the same failure under four fields.
if (line.kind !== 'in-effect') return null
const shown = line.value || 'none'
return (
<select
className="dns-select dns-detour-select"
value={value}
onChange={(e) => onChange(e.target.value)}
disabled={busy || disabled}
aria-label={ariaLabel}
<p
className="dns-ineffect"
title={
line.stale
? `${what} in effect now: ${shown} (profile ${line.profile}). This verdict was computed from the SAVED config, which no longer matches the value in the field — save and apply to bring them back together.`
: `${what} in effect now: ${shown} (profile ${line.profile})`
}
>
<option value="direct">{directLabel}</option>
{catalog.groups.length > 0 && (
<optgroup label="Groups">
{catalog.groups.map((g) => (
<option key={g} value={`group:${g}`}>
Group {g} (balancer)
</option>
))}
</optgroup>
)}
{catalog.chains.length > 0 && (
<optgroup label="Chains">
{catalog.chains.map((c) => (
<option key={c} value={`chain:${c}`}>
Chain {c}
</option>
))}
</optgroup>
)}
{catalog.egresses.length > 0 && (
<optgroup label="Interfaces / egresses">
{catalog.egresses.map((e) => (
<option key={e.name} value={`egress:${e.name}`}>
Interface/egress {e.name}
{e.type ? ` (${e.type})` : ''}
</option>
))}
</optgroup>
)}
{catalog.nodes.length > 0 && (
<optgroup label="Nodes">
{catalog.nodes.map((n) => (
<option key={n} value={`node:${n}`}>
Node {n}
</option>
))}
</optgroup>
)}
{missing && <option value={value}>{value} (missing)</option>}
</select>
<span className="dns-ineffect-turn" aria-hidden="true">
↳
</span>
<span className="dns-ineffect-label mono">in effect now</span>
<span className="dns-ineffect-value mono">{shown}</span>
<span className="dns-ineffect-src">
profile{' '}
<a className="linkish" href="#/profiles">
{line.profile}
</a>
</span>
{/* The daemon reports which stored value it judged. When that is not the
value in the input above, the row would otherwise pair a fresh field
with an old verdict and let it read as one statement. */}
{line.stale && <span className="dns-ineffect-stale">· judged the saved config</span>}
</p>
)
}
@@ -1959,6 +2035,7 @@ function ResolverFields({
<label className="dns-resp">
<span className="dns-resp-label mono">DNS path</span>
<DetourSelect
className="dns-select dns-detour-select"
value={draft.detour}
catalog={catalog}
valid={valid}
@@ -2246,6 +2323,7 @@ function ResolverRow({
<label className="dns-detour">
<span className="dns-detour-label mono">DNS path</span>
<DetourSelect
className="dns-select dns-detour-select"
value={canon}
catalog={catalog}
valid={valid}
+110 -119
View File
@@ -2,10 +2,11 @@ import './Nodes.css'
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import type { ReactNode } from 'react'
import type { LedVariant } from '../components'
import { Button, Led, Toggle, useConfirm } from '../components'
import { Button, DetourSelect, Led, Toggle, useConfirm } from '../components'
import {
apply as apiApply,
getConfig,
getEffectiveConfig,
getGroupsTest,
getStatus,
postGroupsTest,
@@ -15,6 +16,7 @@ import {
ApiError,
} from '../api'
import type {
EffectiveOverride,
GroupTestResult,
Model,
Node as NodeCfg,
@@ -35,10 +37,13 @@ import {
} from '../testResult'
import type { NodeTestRefusal, RowOrigin } from '../testResult'
import { fetchBadge, loudest, otherFindings } from '../subFetch'
import type { DetourCatalog } from '../detour'
import { canonDetour, catalogOf, detourValues } from '../detour'
import type { HeaderRow, SubForm } from '../subEdit'
import {
SUB_FORMATS,
SUB_PROTOS,
classifyFetchVia,
nextSubscription,
parseExpireAlert,
proxyFetchGoesDirect,
@@ -465,106 +470,9 @@ interface NodeGroupData {
// A named refusal the operator can read is a state worth offering; a hidden
// capability is not.
/** The live targets a sub's proxy fetch can be pinned to (the picker options). */
interface DetourCatalog {
groups: string[]
chains: string[]
egresses: { name: string; type: string }[]
nodes: string[]
}
/** Normalise a stored `FetchDetour` to a picker option value (empty ⇒ `direct`). */
function canonDetour(raw: string | undefined, cat: DetourCatalog): string {
const d = (raw ?? '').trim()
if (!d || d.toLowerCase() === 'direct') return 'direct'
if (/^(node|group|chain|egress):/i.test(d)) return d
if (cat.egresses.some((e) => e.name === d)) return `egress:${d}`
if (cat.groups.includes(d)) return `group:${d}`
if (cat.chains.includes(d)) return `chain:${d}`
if (cat.nodes.includes(d)) return `node:${d}`
return d
}
/** Every valid option value for a catalog, including `direct`. */
function detourValues(cat: DetourCatalog): Set<string> {
const s = new Set<string>(['direct'])
for (const g of cat.groups) s.add(`group:${g}`)
for (const c of cat.chains) s.add(`chain:${c}`)
for (const e of cat.egresses) s.add(`egress:${e.name}`)
for (const n of cat.nodes) s.add(`node:${n}`)
return s
}
/** Detour picker: Direct + groups / chains / interfaces-egresses / nodes. */
function DetourSelect({
value,
catalog,
valid,
disabled,
ariaLabel,
onChange,
}: {
value: string // canonical value
catalog: DetourCatalog
valid: Set<string>
disabled: boolean
ariaLabel: string
onChange: (v: string) => void
}) {
const missing = value !== 'direct' && !valid.has(value)
return (
<select
className="fp-input"
value={value}
onChange={(e) => onChange(e.target.value)}
disabled={disabled}
aria-label={ariaLabel}
>
{/* Under FetchVia=proxy this option is not "no preference" — it is the
plain WAN, with the real address. The label says so rather than leaving
the word `Direct` to be read as a default. */}
<option value="direct">Direct — plain WAN, real address</option>
{catalog.groups.length > 0 && (
<optgroup label="Groups">
{catalog.groups.map((g) => (
<option key={g} value={`group:${g}`}>
Group {g} (balancer)
</option>
))}
</optgroup>
)}
{catalog.chains.length > 0 && (
<optgroup label="Chains">
{catalog.chains.map((c) => (
<option key={c} value={`chain:${c}`}>
Chain {c}
</option>
))}
</optgroup>
)}
{catalog.egresses.length > 0 && (
<optgroup label="Interfaces / egresses">
{catalog.egresses.map((e) => (
<option key={e.name} value={`egress:${e.name}`}>
Interface/egress {e.name}
{e.type ? ` (${e.type})` : ''}
</option>
))}
</optgroup>
)}
{catalog.nodes.length > 0 && (
<optgroup label="Nodes">
{catalog.nodes.map((n) => (
<option key={n} value={`node:${n}`}>
Node {n}
</option>
))}
</optgroup>
)}
{missing && <option value={value}>{value} (missing)</option>}
</select>
)
}
// The four helpers this file used to keep locally (DetourCatalog, canonDetour,
// detourValues and the <select> itself) now live in ../detour and
// components/DetourSelect, shared with DNS, Alerts and Profiles.
// ---- page ------------------------------------------------------------------
@@ -734,6 +642,26 @@ export default function Nodes() {
const [saving, setSaving] = useState(false)
const [applying, setApplying] = useState(false)
// What a subscription with NO `fetch_via` inherits — GET /api/config/effective,
// the DAEMON's answer. Schema v3 made the absent option mean "the general fetch
// route decides" rather than `direct`, and that route can itself be overridden
// by the active profile, so the picker's third state has to NAME a value this
// page is in no position to compute. `null` until it arrives, or if it cannot
// be read: the option then says it does not know rather than naming a route
// that might be the wrong one.
const [generalFetch, setGeneralFetch] = useState<EffectiveOverride | null>(null)
const loadEffective = useCallback(async () => {
try {
setGeneralFetch((await getEffectiveConfig()).FetchDetour)
} catch {
// Older daemon, or the read failed. The picker says it does not know.
setGeneralFetch(null)
}
}, [])
useEffect(() => {
void loadEffective()
}, [loadEffective])
/**
* Optimistically apply `next` to the UI, PUT the whole Model, and mark the
* config dirty so the Apply banner shows. Reverts on failure.
@@ -747,6 +675,9 @@ export default function Nodes() {
await putConfig(next)
setDirty(true)
flash(okMsg)
// The general fetch route lives in globals and is judged from disk, so a
// PUT can move what an unset `fetch_via` inherits. Re-ask.
void loadEffective()
return true
} catch (e) {
setConfig(prev) // revert the optimistic edit
@@ -756,7 +687,7 @@ export default function Nodes() {
setSaving(false)
}
},
[config, flash],
[config, flash, loadEffective],
)
const applyNow = useCallback(async () => {
@@ -769,6 +700,9 @@ export default function Nodes() {
setDirty(false)
flash(r.changed ? 'Applied — data plane reconciled' : 'Applied — already up to date')
void loadConfig()
// An apply can re-pin the active profile, which moves the general fetch
// route without any field on this page changing.
void loadEffective()
}
} catch (e) {
flash(`Apply failed — ${errText(e)}`)
@@ -778,7 +712,7 @@ export default function Nodes() {
// ones the operator just fixed.
void loadFindings()
}
}, [flash, loadConfig, loadFindings])
}, [flash, loadConfig, loadFindings, loadEffective])
// ---- node mutations -------------------------------------------------------
const nodes = useMemo(() => asArray(config?.Nodes), [config])
@@ -786,17 +720,21 @@ export default function Nodes() {
const subsOn = subs.filter((s) => s.Enabled).length
// Live detour catalog for the proxy-fetch picker — built from the Model.
const detourCatalog = useMemo<DetourCatalog>(
() => ({
groups: asArray(config?.Groups).map((g) => g.Name),
chains: asArray(config?.Chains).map((c) => c.Name),
egresses: asArray(config?.Egresses).map((e) => ({ name: e.Name, type: e.Type })),
nodes: nodes.map((n) => n.Name),
}),
[config, nodes],
)
const detourCatalog = useMemo<DetourCatalog>(() => catalogOf(config ?? {}), [config])
const detourValid = useMemo(() => detourValues(detourCatalog), [detourCatalog])
// What a subscription with NO `fetch_via` inherits. Schema v3 made the absent
// option mean "the general fetch route decides" rather than `direct`, so the
// picker's third state has to NAME the value it defers to — an option reading
// "not set" over a route that quietly moves the feed into a tunnel is the same
// silence this whole change is undoing.
//
// It is the DAEMON's answer (GET /api/config/effective), not a local
// recomputation, because the general route can be overridden by the active
// profile and this page must not grow a second copy of that rule. `null` until
// it arrives, or if it cannot be read — the option then names no route rather
// than naming the wrong one.
// Egress names for the per-node "dial via egress" picker. Empty ⇒ the control is
// hidden entirely (nothing to pick from).
const egressNames = useMemo(() => asArray(config?.Egresses).map((e) => e.Name), [config])
@@ -1513,6 +1451,7 @@ export default function Nodes() {
busy={busy}
catalog={detourCatalog}
valid={detourValid}
generalFetch={generalFetch}
findings={subFindings.get(s.Name) ?? EMPTY_FINDINGS}
onToggle={(on) => toggleSub(i, on)}
onDelete={() => removeSub(i)}
@@ -2120,6 +2059,7 @@ function SubRow({
busy,
catalog,
valid,
generalFetch,
findings,
onToggle,
onDelete,
@@ -2131,6 +2071,8 @@ function SubRow({
busy: boolean
catalog: DetourCatalog
valid: Set<string>
/** What an unset `fetch_via` defers to — passed down to the options editor. */
generalFetch: EffectiveOverride | null
/** What the last apply said about THIS subscription; empty when it said nothing. */
findings: StatusWarning[]
onToggle: (on: boolean) => void
@@ -2252,6 +2194,7 @@ function SubRow({
busy={busy}
catalog={catalog}
valid={valid}
generalFetch={generalFetch}
onEdit={onEdit}
onUpdate={onUpdate}
/>
@@ -2369,10 +2312,30 @@ function SubAccount({ sub }: { sub: Subscription }) {
// which survive a restart precisely so they are readable when a refetch is
// impossible — cannot be cleared by a form that has no control for them.
const FETCH_VIA: { value: string; label: string }[] = [
{ value: 'direct', label: 'Direct' },
{ value: 'proxy', label: 'Proxy (through the tunnel)' },
]
// `fetch_via` has THREE states in schema v3, and the third one is spelled by the
// option being absent. It is listed first because it is what a new subscription
// gets, and it is the only one whose meaning lives somewhere else — so it names
// that somewhere else instead of saying "default".
//
// `Direct` is deliberately NOT worded as "no proxy". It is a decision that
// outranks the general route: a feed marked Direct stays in the clear on the day
// the operator sends every other fetch through a tunnel.
//
// `general` is the daemon's reading, or null when there is none. With no reading
// the option says so instead of guessing a route: naming the wrong one is worse
// than naming none, and the two are indistinguishable to whoever reads it.
function fetchViaOptions(general: EffectiveOverride | null): { value: string; label: string }[] {
let inherits = 'route unknown — couldn’t read the general setting'
if (general) {
const route = general.Value || 'direct'
inherits = general.Profile ? `${route}, from profile ${general.Profile}` : route
}
return [
{ value: '', label: `Not set — use the general fetch route (${inherits})` },
{ value: 'direct', label: 'Direct — always in the clear, whatever the general route is' },
{ value: 'proxy', label: 'Proxy — through this subscription’s own detour' },
]
}
function SubOptions({
id,
@@ -2380,6 +2343,7 @@ function SubOptions({
busy,
catalog,
valid,
generalFetch,
onEdit,
onUpdate,
}: {
@@ -2388,6 +2352,8 @@ function SubOptions({
busy: boolean
catalog: DetourCatalog
valid: Set<string>
/** What an unset `fetch_via` defers to — named in the picker's first option. */
generalFetch: EffectiveOverride | null
onEdit: (patch: Subscription) => Promise<boolean>
onUpdate: (name: string) => Promise<void>
}) {
@@ -2424,7 +2390,9 @@ function SubOptions({
}
}, [onUpdate, sub.Name])
const proxy = draft.FetchVia === 'proxy'
const viaMode = classifyFetchVia(draft.FetchVia)
const proxy = viaMode === 'proxy'
const viaOptions = useMemo(() => fetchViaOptions(generalFetch), [generalFetch])
// Read from the DRAFT, not the saved sub: the warning has to appear the moment
// the picker lands on Direct, not one save later.
const leaksDirect = proxyFetchGoesDirect({
@@ -2480,17 +2448,36 @@ function SubOptions({
onChange={(e) => set('UpdateInterval', e.target.value)}
/>
</OptField>
<OptField label="Fetch via" hint="Proxy = pull through the tunnel for a blocked host">
<OptField
label="Fetch via"
wide
hint={
viaMode === 'inherit'
? 'Not set: this feed follows the general fetch route, so it moves when that setting moves. Pick Direct to pin it in the clear, or Proxy to give it a route of its own.'
: viaMode === 'direct'
? 'Pinned in the clear. It stays direct even if the general fetch route is later sent through a tunnel.'
: viaMode === 'proxy'
? 'Pulled through the route chosen below — use it for a feed host the uplink blocks.'
: 'This value is not one the daemon accepts, so it refuses to guess a route rather than picking one for you. Choose Not set, Direct or Proxy.'
}
>
<select
className="fp-input"
value={draft.FetchVia}
onChange={(e) => set('FetchVia', e.target.value)}
aria-label={`Fetch via for ${sub.Name}`}
>
{FETCH_VIA.map((o) => (
{viaOptions.map((o) => (
<option key={o.value} value={o.value}>
{o.label}
</option>
))}
{/* An unrecognised stored value is shown, not silently replaced by
whichever option happens to be first — that swap is a route
decision, and it is not the picker's to make. */}
{viaMode === 'unknown' && (
<option value={draft.FetchVia}>{draft.FetchVia} (not a valid value)</option>
)}
</select>
</OptField>
{proxy && (
@@ -2504,6 +2491,10 @@ function SubOptions({
valid={valid}
disabled={disabled}
ariaLabel={`Proxy-fetch detour for ${sub.Name}`}
/* Under FetchVia=proxy this option is not "no preference" — it is
the plain WAN, with the real address. The label says so rather
than leaving the word `Direct` to be read as a default. */
directLabel="Direct — plain WAN, real address"
onChange={(v) => set('FetchDetour', v)}
/>
</OptField>
+13
View File
@@ -331,6 +331,19 @@
font-size: 12px;
padding: 0 2px;
}
/* A scalar override (resolver slot / list-fetch route). Deliberately NOT the
on/off pair's green and red: it flips nothing on, it repoints a value, and
borrowing their semantics would make a routine profile setting read as a
verdict. */
.pf-chip--soft {
color: var(--dim);
font-family: var(--font-mono);
font-size: 10.5px;
letter-spacing: 0.04em;
}
.pf-chip--soft .mono {
color: var(--ink);
}
/* ---- row actions ---- */
.pf-row-actions {
+182 -24
View File
@@ -1,8 +1,10 @@
import './Profiles.css'
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { Button, Led, Toggle, useConfirm } from '../components'
import { Button, DetourSelect, Led, Toggle, useConfirm } from '../components'
import { apply as apiApply, getConfig, getInterfaces, putConfig, ApiError } from '../api'
import type { Interface, Model, Profile } from '../api'
import type { DetourCatalog } from '../detour'
import { canonDetour, catalogOf, detourValues } from '../detour'
// The Profiles page is a thin editor over the desired-state Model — the same
// save→apply split as Settings / DNS / Routing. Every edit rewrites a slice in
@@ -24,6 +26,60 @@ const asArray = <T,>(a: T[] | null | undefined): T[] => (a ? a : [])
const ruleSummary = (names: string[], max = 2): string =>
names.length <= max ? names.join(', ') : `${names.slice(0, max).join(', ')} +${names.length - max} more`
// ---- the scalar overrides --------------------------------------------------
//
// Four Globals scalars a profile can repoint while it is active. They share one
// contract, so they are described once and rendered from the description:
//
// "" means NO OVERRIDE, and inheritance is PER FIELD — a profile may set only
// the fallback resolver and keep the global default. There is deliberately no
// control for "override with the globals value": leaving the override off
// already does exactly that, and a second way to say it would be a second
// thing to keep in step.
//
// The detour one is NOT interchangeable with the resolvers despite sharing the
// contract: for a route, `""` and `direct` are different inputs — `""` inherits,
// an explicit `direct` forces the plain WAN over a globals setting that tunnels.
// That is why its picker carries a separate "No override" option instead of
// letting `direct` stand in for one.
/** The Profile fields that override a Globals scalar of the same name. */
type ScalarField = 'ResolverDefault' | 'ResolverFallback' | 'EndpointResolver' | 'FetchDetour'
const SCALAR_OVERRIDES: ReadonlyArray<{ field: ScalarField; chip: string }> = [
{ field: 'ResolverDefault', chip: 'default dns' },
{ field: 'ResolverFallback', chip: 'fallback dns' },
{ field: 'EndpointResolver', chip: 'endpoint dns' },
{ field: 'FetchDetour', chip: 'list fetch' },
]
/** The three resolver slots, in the order the DNS page lists them. */
const RESOLVER_OVERRIDES: ReadonlyArray<{
field: Extract<ScalarField, 'ResolverDefault' | 'ResolverFallback' | 'EndpointResolver'>
label: string
toast: string
hint: string
}> = [
{
field: 'ResolverDefault',
label: 'Default resolver',
toast: 'Default resolver',
hint: 'Where queries go when no DNS rule matches. Point it at a resolver the active uplink can actually reach — a carrier that refuses :443 to the public DoH addresses takes the whole DNS path down with it.',
},
{
field: 'ResolverFallback',
label: 'Fallback resolver',
toast: 'Fallback resolver',
hint: 'Tried when the default fails. Overriding this alone is fine — the default keeps its globals value.',
},
{
field: 'EndpointResolver',
label: 'Endpoint resolver',
toast: 'Endpoint resolver',
hint: 'Looks up the server domains in your proxy configs while this profile is active — e.g. a plain resolver on SIM, DoH on Wi-Fi. It resolves before any tunnel is up, so it has to work on the bare uplink.',
},
]
// Pull Name strings out of a model slice (Rules / Resolvers) for the pickers.
type Named = { Name?: unknown }
function namesOf(v: unknown): string[] {
@@ -130,6 +186,22 @@ export default function Profiles() {
const profiles = useMemo<Profile[]>(() => asArray(config?.Profiles), [config])
const ruleNames = useMemo(() => namesOf(config?.Rules), [config])
const resolverNames = useMemo(() => namesOf(config?.Resolvers), [config])
// The routes a profile's list-fetch override can point at — the same catalog
// and the same picker every other page uses, so the vocabulary is one
// vocabulary (see ../detour).
const detourCatalog = useMemo<DetourCatalog>(() => catalogOf(config ?? {}), [config])
const detourValid = useMemo(() => detourValues(detourCatalog), [detourCatalog])
// What the profile overrides are measured against. Shown beside each override
// so "No override" is a statement about a known value, not a blank.
const globalFallbacks = useMemo(
() => ({
ResolverDefault: config?.Globals?.ResolverDefault ?? '',
ResolverFallback: config?.Globals?.ResolverFallback ?? '',
EndpointResolver: config?.Globals?.EndpointResolver ?? '',
FetchDetour: canonDetour(config?.Globals?.FetchDetour, detourCatalog),
}),
[config, detourCatalog],
)
const profileNames = useMemo(() => new Set(profiles.map((p) => p.Name)), [profiles])
const active = globals?.ActiveProfile ?? ''
@@ -325,6 +397,9 @@ export default function Profiles() {
ruleNames={ruleNames}
interfaces={interfaces}
resolverNames={resolverNames}
detourCatalog={detourCatalog}
detourValid={detourValid}
globalFallbacks={globalFallbacks}
taken={profileNames}
onPatch={(patch, msg) => patchProfile(p.Name, patch, msg)}
onRename={(nn) => onRename(p.Name, nn)}
@@ -550,6 +625,12 @@ function AddProfileForm({
// ---- one profile: summary row + expandable editor --------------------------
/** The globals each override is measured against, keyed by the field it overrides. */
type GlobalFallbacks = Record<
'ResolverDefault' | 'ResolverFallback' | 'EndpointResolver' | 'FetchDetour',
string
>
function ProfileRow({
profile,
isActive,
@@ -560,6 +641,9 @@ function ProfileRow({
ruleNames,
interfaces,
resolverNames,
detourCatalog,
detourValid,
globalFallbacks,
taken,
onPatch,
onRename,
@@ -574,6 +658,9 @@ function ProfileRow({
ruleNames: string[]
interfaces: Interface[]
resolverNames: string[]
detourCatalog: DetourCatalog
detourValid: Set<string>
globalFallbacks: GlobalFallbacks
taken: Set<string>
onPatch: (patch: Partial<Profile>, msg: string) => void
onRename: (newName: string) => void
@@ -582,6 +669,14 @@ function ProfileRow({
const iface = asArray(profile.MatchIface)
const enableRules = asArray(profile.EnableRules)
const disableRules = asArray(profile.DisableRules)
// Scalar overrides belong on the COLLAPSED row too. A profile that silently
// repoints the resolvers is the whole reason the DNS page had to grow an "in
// effect now" line; a row that only listed rule flips let the operator scroll
// past the profile actually doing it.
const scalars = SCALAR_OVERRIDES.map((s) => ({
label: s.chip,
value: (profile[s.field] ?? '').trim(),
})).filter((s) => s.value !== '')
return (
<li className={profile.Enabled ? 'pf-row' : 'pf-row off'}>
@@ -640,6 +735,16 @@ function ProfileRow({
rules off: {ruleSummary(disableRules)}
</span>
)}
{scalars.map((s) => (
<span
key={s.label}
className="pf-chip pf-chip--soft"
title={`While active, ${s.label} is ${s.value} instead of the globals value`}
>
<b>{s.label}</b>
<span className="mono">{s.value}</span>
</span>
))}
</div>
</div>
<div className="pf-row-actions">
@@ -671,6 +776,9 @@ function ProfileRow({
ruleNames={ruleNames}
interfaces={interfaces}
resolverNames={resolverNames}
detourCatalog={detourCatalog}
detourValid={detourValid}
globalFallbacks={globalFallbacks}
taken={taken}
onPatch={onPatch}
onRename={onRename}
@@ -688,6 +796,9 @@ function ProfileEditor({
ruleNames,
interfaces,
resolverNames,
detourCatalog,
detourValid,
globalFallbacks,
taken,
onPatch,
onRename,
@@ -697,6 +808,9 @@ function ProfileEditor({
ruleNames: string[]
interfaces: Interface[]
resolverNames: string[]
detourCatalog: DetourCatalog
detourValid: Set<string>
globalFallbacks: GlobalFallbacks
taken: Set<string>
onPatch: (patch: Partial<Profile>, msg: string) => void
onRename: (newName: string) => void
@@ -791,36 +905,80 @@ function ProfileEditor({
<fieldset className="pf-eblock">
<legend className="pf-elegend">Overrides while active</legend>
{RESOLVER_OVERRIDES.map((o) => {
const value = (profile[o.field] ?? '').trim()
const global = globalFallbacks[o.field]
return (
<div className="pf-erow" key={o.field}>
<span className="pf-elabel">{o.label}</span>
<div className="pf-erow-ctl">
<select
className="pf-select"
value={value}
onChange={(e) =>
onPatch(
{ [o.field]: e.target.value } as Partial<Profile>,
e.target.value
? `${o.toast} → ${e.target.value}`
: `${o.toast} override cleared`,
)
}
disabled={busy}
aria-label={`${o.label} override`}
>
{/* Names the value it falls back to, so "no override" is a
statement about a known resolver rather than a blank. */}
<option value="">
No override{global ? ` — globals: ${global}` : ' — globals: none'}
</option>
{resolverNames.map((rn) => (
<option key={rn} value={rn}>
{rn}
</option>
))}
{value !== '' && !resolverNames.includes(value) && (
<option value={value}>{value} (missing)</option>
)}
</select>
<p className="pf-ehint">{o.hint}</p>
</div>
</div>
)
})}
<div className="pf-erow">
<span className="pf-elabel">Endpoint resolver</span>
<span className="pf-elabel">List fetch route</span>
<div className="pf-erow-ctl">
<select
<DetourSelect
className="pf-select"
value={profile.EndpointResolver ?? ''}
onChange={(e) =>
/* Canonicalise only a value that is THERE. canonDetour maps '' to
'direct', so running it unconditionally would turn "this profile
says nothing about fetches" into "this profile forces the plain
WAN" — and then display that invention as the effective route. */
value={
(profile.FetchDetour ?? '').trim()
? canonDetour(profile.FetchDetour, detourCatalog)
: ''
}
catalog={detourCatalog}
valid={detourValid}
busy={busy}
ariaLabel="List fetch route override"
inheritLabel={`No override — globals: ${globalFallbacks.FetchDetour}`}
directLabel="Direct — plain WAN"
onChange={(v) =>
onPatch(
{ EndpointResolver: e.target.value },
e.target.value
? `Endpoint resolver → ${e.target.value}`
: 'Endpoint resolver override cleared',
{ FetchDetour: v },
v ? `List fetches → ${v}` : 'List fetch override cleared',
)
}
disabled={busy}
aria-label="Endpoint resolver"
>
<option value="">No override</option>
{resolverNames.map((rn) => (
<option key={rn} value={rn}>
{rn}
</option>
))}
{profile.EndpointResolver && !resolverNames.includes(profile.EndpointResolver) && (
<option value={profile.EndpointResolver}>{profile.EndpointResolver} (missing)</option>
)}
</select>
/>
<p className="pf-ehint">
Overrides which resolver looks up your proxy server domains while this profile is active
— e.g. a plain resolver on SIM, DoH on Wi-Fi.
Where blocklists, allowlists, rule-sets and geo files are downloaded from while this
profile is active. Pick a route the uplink can reach — a list that can’t be
downloaded isn’t applied and matches nothing until it loads.{' '}
<b>Direct is not the same as no override</b>: it forces the plain WAN even when
globals sends fetches through a tunnel.
</p>
</div>
</div>
+117
View File
@@ -19,6 +19,7 @@ import assert from 'node:assert/strict'
import {
SUB_FORMATS,
SUB_PROTOS,
classifyFetchVia,
formatExpireAlert,
joinFilterList,
nextSubscription,
@@ -254,6 +255,122 @@ test('a saved proxy sub with the picker on Direct trips the predicate', () => {
assert.equal(proxyFetchGoesDirect(out), true)
})
// ---- `fetch_via` has THREE states, and one of them is spelled by absence -----
//
// WHAT THESE PROTECT, in the order they matter:
//
// 1. AN EXPLICIT `direct` MUST SURVIVE A SAVE. The merge wrote
// `proxy ? 'proxy' : undefined`, so a user's deliberate "always in the
// clear" was stored as an ABSENT option — which in schema v3 means
// "inherit the general fetch route". Nothing failed, nothing was logged,
// and the two behave identically right up until a general fetch route is
// set, at which point every such feed silently moves into a tunnel. This
// is the regression, so it is the first test.
// 2. THE INHERIT STATE MUST BE REACHABLE IN BOTH DIRECTIONS. `subToForm`
// collapsed absent into `'direct'` on load, so the state could not be seen;
// the merge could not write it either. A round-trip is the only test that
// catches a state that is lost on one leg and re-invented on the other.
// 3. UNKNOWN IS NOT A MODE, AND NOT AN ERASURE. The Go side refuses to guess
// (SubscriptionFetchDetour returns ok=false). The panel must neither guess
// on its behalf nor quietly normalise the value away — the stored typo is
// the evidence, and the value it used to be read as was `direct`: a fetch
// that goes out in the clear and SUCCEEDS, so the outcome reveals nothing.
// 4. CONTROL: `proxy` STILL WORKS, WITH ITS DETOUR. A rule that turns every
// state into "not set" would satisfy 1–3 read carelessly.
/** Every state, both spellings of the absent one, through load → save → load. */
test('all three fetch_via states survive a full round trip', () => {
const cases: { stored: string | undefined; form: string; saved: string | undefined }[] = [
{ stored: undefined, form: '', saved: undefined }, // absent ⇒ inherit
{ stored: '', form: '', saved: undefined }, // empty ⇒ inherit
{ stored: 'direct', form: 'direct', saved: 'direct' }, // explicit, and KEPT
{ stored: 'proxy', form: 'proxy', saved: 'proxy' },
]
for (const c of cases) {
const s = stored({ FetchVia: c.stored, FetchDetour: c.stored === 'proxy' ? 'group:auto' : undefined })
const form = subToForm(s, c.stored === 'proxy' ? 'group:auto' : 'direct')
assert.equal(form.FetchVia, c.form, `load ${JSON.stringify(c.stored)}`)
const out = nextSubscription(s, form, undefined)
assert.equal(out.FetchVia, c.saved, `save ${JSON.stringify(c.stored)}`)
// …and back again: the second load must land on the same form value, or the
// state is being re-invented on one of the two legs.
const again = subToForm(out, c.stored === 'proxy' ? 'group:auto' : 'direct')
assert.equal(again.FetchVia, c.form, `reload ${JSON.stringify(c.stored)}`)
}
})
test('an explicit Direct is written out — absence means inherit now, not direct', () => {
const s = stored({ FetchVia: 'direct' })
const out = nextSubscription(s, subToForm(s, 'direct'), undefined)
assert.equal(out.FetchVia, 'direct')
assert.ok(
Object.prototype.hasOwnProperty.call(out, 'FetchVia') && out.FetchVia !== undefined,
'the key must be PRESENT: an omitted fetch_via is the inherit state, so dropping it hands this feed to whatever the general fetch route becomes',
)
// The shape that actually reaches the daemon, since PUT sends JSON.
assert.equal(JSON.parse(JSON.stringify(out)).FetchVia, 'direct')
})
test('choosing Not set clears the option entirely — that is how inherit is written', () => {
const s = stored({ FetchVia: 'direct' })
const out = nextSubscription(s, { ...subToForm(s, 'direct'), FetchVia: '' }, undefined)
assert.equal(out.FetchVia, undefined)
assert.equal('FetchVia' in JSON.parse(JSON.stringify(out)), false, 'inherit is the ABSENT option')
})
test('switching a subscription from inherit to Direct is a real, saved change', () => {
// The user path the defect hid: open a feed that follows the general route,
// pin it in the clear, save. Before the fix the save was a no-op.
const s = stored({ FetchVia: undefined })
const form = subToForm(s, 'direct')
assert.equal(form.FetchVia, '', 'it must load as inherit, not as direct')
const out = nextSubscription(s, { ...form, FetchVia: 'direct' }, undefined)
assert.equal(out.FetchVia, 'direct')
assert.notEqual(out.FetchVia, s.FetchVia, 'the save has to change something')
})
test('classifyFetchVia is closed, positive, and case/space-insensitive', () => {
const table: Record<string, ReturnType<typeof classifyFetchVia>> = {
'': 'inherit',
' ': 'inherit',
direct: 'direct',
DIRECT: 'direct',
' Direct ': 'direct',
proxy: 'proxy',
PROXY: 'proxy',
' Proxy ': 'proxy',
}
for (const [raw, want] of Object.entries(table)) {
assert.equal(classifyFetchVia(raw), want, JSON.stringify(raw))
}
assert.equal(classifyFetchVia(undefined), 'inherit')
// Everything else is a named defect, never a working mode — above all never
// `direct`, which is what it silently became before.
for (const bad of ['tunnel', 'prox', 'yes', 'true', 'group:auto', 'inherit']) {
assert.equal(classifyFetchVia(bad), 'unknown', bad)
}
})
test('an unrecognised fetch_via is neither guessed nor erased', () => {
const s = stored({ FetchVia: 'tunnel' })
const form = subToForm(s, 'direct')
assert.equal(form.FetchVia, 'tunnel', 'shown as it is stored, so it can be seen and fixed')
const out = nextSubscription(s, form, undefined)
assert.equal(out.FetchVia, 'tunnel', 'rewriting it would pick a route nobody chose')
// And it must not be read as any of the three real modes.
assert.equal(proxyFetchGoesDirect(out), false)
})
test('control — proxy still round-trips and keeps its own detour', () => {
const s = stored({ FetchVia: 'proxy', FetchDetour: 'group:auto' })
const out = nextSubscription(s, subToForm(s, 'group:auto'), undefined)
assert.equal(out.FetchVia, 'proxy')
assert.equal(out.FetchDetour, 'group:auto')
assert.equal(proxyFetchGoesDirect(out), false)
})
// ---- list + header parsing ----------------------------------------------------
test('filter lists split on commas and newlines, trimmed and deduped', () => {
+111 -4
View File
@@ -61,6 +61,99 @@ export const joinFilterList = (a: string[] | null | undefined): string => (a ? a
export type ParseResult<T> = { ok: true; value: T } | { ok: false; error: string }
// ---- `fetch_via`: three states, and one of them has no spelling -------------
//
// Schema v3 changed what an ABSENT `fetch_via` means, and the change is invisible
// from the outside: it used to mean `direct`, and now it means INHERIT — the
// general `fetch_detour` (globals, or the active profile's override) decides.
//
// The panel got that wrong in both directions and neither one raised anything:
//
// READ `FetchVia: s.FetchVia === 'proxy' ? 'proxy' : 'direct'` collapsed the
// inherit state into `direct` on load, so it could not be seen.
// WRITE `FetchVia: proxy ? 'proxy' : undefined` wrote a user's explicit
// `Direct` as an ABSENT option, so it could not be kept either. Every
// save of every non-proxy subscription silently converted "always in the
// clear" into "follow the general setting" — and the two behave
// identically right up until somebody sets a general fetch route, at
// which point those feeds move into a tunnel with no message anywhere.
//
// So the form value is the raw stored vocabulary, not a boolean in disguise:
// `''` (inherit), `'direct'`, `'proxy'`. Mirrors model.ClassifyFetchVia.
/** What a raw `fetch_via` means. Mirrors model.FetchViaMode — CLOSED and
* POSITIVE: an unrecognised value gets its own member instead of being folded
* into one of the real ones. */
export type FetchViaMode = 'inherit' | 'direct' | 'proxy' | 'unknown'
/** The two spellings the option accepts. Absent is the third state and has no
* spelling — that is why this list has two entries and the picker has three. */
export const FETCH_VIA_VALUES = ['direct', 'proxy'] as const
/**
* Classify a raw `fetch_via`, exactly as model.ClassifyFetchVia does: case- and
* space-insensitive, with everything unrecognised landing on `unknown` rather
* than on a working mode.
*
* `unknown` is not pedantry. The value it used to be silently treated as was
* `direct` — a fetch that goes out over the plain WAN with this router's real
* address and SUCCEEDS, so nothing about the outcome reveals the typo. The Go
* side refuses to guess (SubscriptionFetchDetour returns ok=false); the panel
* must not guess on its behalf either.
*/
export function classifyFetchVia(v: string | undefined): FetchViaMode {
switch ((v ?? '').trim().toLowerCase()) {
case '':
return 'inherit'
case 'direct':
return 'direct'
case 'proxy':
return 'proxy'
default:
return 'unknown'
}
}
/**
* The value the picker binds to: the canonical spelling for the three real
* modes, and an unrecognised value kept VERBATIM so the editor can show it as
* stale instead of quietly rewriting somebody's config into a mode they did not
* choose.
*/
export function fetchViaFormValue(raw: string | undefined): string {
switch (classifyFetchVia(raw)) {
case 'inherit':
return ''
case 'direct':
return 'direct'
case 'proxy':
return 'proxy'
case 'unknown':
return (raw ?? '').trim()
}
}
/**
* The value to STORE for a form choice. `undefined` writes no option at all,
* which is the only way to express inherit — and, critically, is NOT what an
* explicit `direct` writes any more.
*/
export function fetchViaStored(formValue: string): string | undefined {
switch (classifyFetchVia(formValue)) {
case 'inherit':
return undefined
case 'direct':
return 'direct'
case 'proxy':
return 'proxy'
case 'unknown':
// Preserved, not normalised. The daemon names it as a defect
// (ValidateSubscriptions); rewriting it here would erase the evidence and
// pick a route on the operator's behalf.
return formValue.trim()
}
}
/**
* True when this subscription SAYS it fetches through the tunnel and does not.
*
@@ -80,12 +173,18 @@ export type ParseResult<T> = { ok: true; value: T } | { ok: false; error: string
*
* `direct` spelled out explicitly counts the same as empty: it resolves to the
* same tag, and picking it deliberately does not make the address any more hidden.
* That is about the DETOUR. It is not true of `fetch_via`, where empty and
* `direct` became different states in schema v3 — see the note below.
*
* Only `fetch_via=proxy` is judged here. `direct` is an honest choice and
* inherit expresses no opinion, so neither can be "saying proxy and not meaning
* it", which is the single thing this predicate detects.
*/
export function proxyFetchGoesDirect(s: {
FetchVia?: string
FetchDetour?: string
}): boolean {
if ((s.FetchVia ?? '').trim().toLowerCase() !== 'proxy') return false
if (classifyFetchVia(s.FetchVia) !== 'proxy') return false
const d = (s.FetchDetour ?? '').trim()
return d === '' || d.toLowerCase() === 'direct'
}
@@ -141,6 +240,11 @@ export interface SubForm {
Name: string
URL: string
UpdateInterval: string
/**
* `''` (inherit — the general fetch route decides) | `'direct'` | `'proxy'`,
* or an unrecognised stored value carried through verbatim. NOT a boolean:
* see the `fetch_via` note above for what treating it as one cost.
*/
FetchVia: string
/** Canonical: `direct` | `group:x` | `node:x` | `egress:x` | `chain:x`. */
FetchDetour: string
@@ -167,7 +271,7 @@ export function subToForm(s: Subscription, canonDetour: string): SubForm {
Name: s.Name,
URL: s.URL,
UpdateInterval: s.UpdateInterval ?? '',
FetchVia: s.FetchVia === 'proxy' ? 'proxy' : 'direct',
FetchVia: fetchViaFormValue(s.FetchVia),
FetchDetour: canonDetour,
UA: s.UA ?? '',
HWID: s.HWID ?? '',
@@ -215,7 +319,7 @@ export function nextSubscription(
f: SubForm,
expireAlertDays: number | undefined,
): Subscription {
const proxy = f.FetchVia === 'proxy'
const proxy = classifyFetchVia(f.FetchVia) === 'proxy'
const headers = f.Headers.filter((h) => h.key.trim()).map(
(h) => `${h.key.trim()}: ${h.value.trim()}`,
)
@@ -227,7 +331,10 @@ export function nextSubscription(
Name: f.Name.trim() || base.Name,
URL: f.URL.trim() || base.URL,
UpdateInterval: str(f.UpdateInterval),
FetchVia: proxy ? 'proxy' : undefined,
// Three states, and only ONE of them writes nothing. An explicit `direct`
// must be written out: absence means inherit now, so collapsing the two
// would hand this feed to whatever the general fetch route becomes later.
FetchVia: fetchViaStored(f.FetchVia),
// The detour only rides along when fetching via proxy and it isn't plain Direct.
FetchDetour: proxy && f.FetchDetour !== 'direct' ? f.FetchDetour : undefined,
UA: str(f.UA),
+11 -2
View File
@@ -66,7 +66,7 @@ import type { StatusWarning, Subscription } from './api'
// Explicit extension: this module is exercised by `npm test` under node's own
// loader, which does not do bundler-style extension guessing. Same reason as
// intercept.ts's import of planeState.
import { proxyFetchGoesDirect } from './subEdit.ts'
import { classifyFetchVia, proxyFetchGoesDirect } from './subEdit.ts'
/**
* What the row should say about the fetch route. A CLOSED set — the caller
@@ -211,7 +211,16 @@ export function fetchBadge(sub: Subscription, findings: StatusWarning[]): FetchB
}
}
if ((sub.FetchVia ?? '') !== 'proxy') return null
// Read through the shared classifier, not a bare `!== 'proxy'`. The daemon
// compares case- and space-insensitively (model.ClassifyFetchVia), so a stored
// `Proxy` is proxy to the engine; a strict compare here would have drawn no
// badge at all over a configuration that can leak.
//
// Both other real states pass through silently AND HONESTLY: an explicit
// `direct` claims nothing about a tunnel, and `inherit` claims nothing either
// — its route is the general fetch route, which this row does not own and the
// options editor names in full.
if (classifyFetchVia(sub.FetchVia) !== 'proxy') return null
if (proxyFetchGoesDirect(sub)) {
// No URL is the only quiet answer, and it is quiet because the fetch is
+67 -6
View File
@@ -413,6 +413,44 @@ func (a *Applier) HTTPClient(via string) (*http.Client, error) {
return a.eng.HTTPClient(via)
}
// subFetchNeedsEngine reports whether a RESOLVED subscription fetch has to be
// dialled through the running box, or may go straight out of this process
// (subscribe.Fetch treats a nil client as a plain direct client).
//
// The switch is closed and positive over model.FetchViaMode, and the three live
// arms are three different decisions rather than one test on the resolved string:
//
// - `direct` is the operator saying "in the clear", explicitly. No engine, and
// no dependency on one — which is what makes it the documented escape from the
// first-boot deadlock (see subBootstrapDeadlockMessage).
// - `proxy` goes through the engine ALWAYS, empty detour included. That is not an
// oversight being preserved: an empty detour resolves to the box's own `direct`
// outbound, apply/warnings.go grades that critical by name, and quietly moving
// it onto a process-local client instead would change which of the two the
// operator is being warned about while leaving the disclosure identical.
// - ABSENT inherits the general detour, so it needs the engine exactly when that
// detour names something other than `direct`. This arm is the whole behaviour
// change of schema v3, and the migration wrote an explicit `direct` onto every
// existing subscription so no deployed config reaches it by accident.
//
// FetchViaUnknown cannot arrive: SubscriptionFetchDetour refuses it and the caller
// returns before asking. It is listed anyway so a value that is not a decision can
// never be answered as if it were one.
func subFetchNeedsEngine(via model.FetchViaMode, detour string) bool {
switch via {
case model.FetchViaDirect:
return false
case model.FetchViaProxy:
return true
case model.FetchViaInherit:
d := strings.TrimSpace(detour)
return d != "" && !strings.EqualFold(d, model.TargetDirect)
case model.FetchViaUnknown:
return false
}
return false
}
// chainViaPrefix is the `via`/detour selector that names a multi-hop chain.
const chainViaPrefix = "chain:"
@@ -660,7 +698,11 @@ func (a *Applier) runningTags() map[string]bool {
// outbound named by its FetchDetour (group:/node:/egress:/chain:/direct); a detour
// that cannot be resolved FAILS the update rather than falling back — a silent
// direct fetch would put the feed, and the owner's real address, on the plain WAN.
// With FetchVia!="proxy" the fetch is DIRECT, as configured.
// With FetchVia=="direct" the fetch is in the clear, as configured. With FetchVia
// ABSENT the GENERAL detour applies (globals.fetch_detour, or the active WAN
// profile's override) — see subFetchNeedsEngine for the whole table, and
// model.SubscriptionFetchDetour for the resolution the three consumers share.
// Any OTHER value of FetchVia is refused by name rather than demoted to direct.
// Zero-node safety is inherited from subscribe.UpdateSubscription
// (a bad body leaves the cache untouched); nothing is written on a failed parse.
// Returns the node count now cached.
@@ -733,13 +775,32 @@ func (a *Applier) UpdateSubscription(name string) (added int, err error) {
"an explicit request was made for it by name. The scheduled refresh does not touch it.")
}
// Resolve the fetch client: through the tunnel when fetch_via=proxy, else direct
// (subscribe.Fetch treats a nil client as a plain direct client).
// Resolve the fetch client. THREE states now, not two, and the third is the
// point: `fetch_via` ABSENT no longer means `direct`, it means "use the general
// fetch detour" (globals.fetch_detour, or the active WAN profile's override) —
// which is the setting that exists so a router behind an operator whitelist can
// reach its provider at all. model.SubscriptionFetchDetour is the ONE resolver
// for that table, shared with generate and the panel, so the three cannot drift
// into three different answers again.
//
// A FOURTH state is what the two-way test used to swallow. `EqualFold(FetchVia,
// "proxy")` sent every unrecognised value — a typo, a word from another product
// — down the else branch and out over the plain WAN, successfully, with the feed
// URL and this router's real address on it and nothing anywhere to see. ok=false
// is that state named; there is no safe side to pick for it, so this refuses.
var client *http.Client
if strings.EqualFold(strings.TrimSpace(sub.FetchVia), "proxy") {
client, err = a.HTTPClient(sub.FetchDetour)
prof, _ := model.ResolveActiveProfile(m) // the warnings are the apply path's to report
detour, viaOK := model.SubscriptionFetchDetour(*sub, m.Globals, prof)
if !viaOK {
return 0, fmt.Errorf("subscription %q: fetch_via %q is not a value this option has — "+
"the accepted values are %s, and leaving it out means \"use the general fetch_detour\". "+
"REFUSING to fetch rather than guess: the guess this used to make was a fetch in the clear",
name, sub.FetchVia, strings.Join(model.FetchViaNames, "/"))
}
if subFetchNeedsEngine(model.ClassifyFetchVia(sub.FetchVia), detour.Value) {
client, err = a.HTTPClient(detour.Value)
if err != nil {
return 0, fmt.Errorf("subscription %q detour %q: %w", name, sub.FetchDetour, err)
return 0, fmt.Errorf("subscription %q detour %q (from %s): %w", name, detour.Value, detour.From, err)
}
// engine.HTTPClient builds a FRESH http.Transport per call, and its dialer
// closes over the resolved outbound. Dropping the client on the floor leaves
+545
View File
@@ -0,0 +1,545 @@
package apply
// Three gradings this file owns, and one plumbing job that had never been wired.
//
// 1. FETCH-DETOUR-NOT-APPLIED — the operator aimed every list/geo/rule-set
// download at a tunnel, the name resolves to nothing, and the downloads are
// going out over the plain WAN instead. Critical, and driven from the REAL
// producer so a reword in generate fails here rather than going quiet.
// 2. RULESET-FROM-CACHE — the source could not be checked and the engine's cache
// holds a copy it can load, so the list IS applied and IS matching. Degraded,
// NOT critical. Behind a whitelisting SIM operator this is the STEADY state,
// so grading it critical means a permanently red banner over working lists.
// 3. model.ValidateProfiles — written, tested, and called by nothing. A check
// nobody runs reads exactly like a check that passes.
import (
"strings"
"testing"
"github.com/sagernet/sing-box/shater/generate"
"github.com/sagernet/sing-box/shater/model"
"github.com/sagernet/sing-box/shater/netplane"
)
// unfetchableSRSURL is a rule-set URL that fails at http.NewRequest, before any
// socket is opened. That is deliberate: the tests below need generate to ATTEMPT a
// remote rule-set (which is what makes it resolve the fetch detour) without the
// suite doing real network I/O, and without a three-second timeout in the middle
// of a unit test. The ".srs" suffix is load-bearing — ruleSetURLIsEngineNative
// decides remote-vs-locally-compiled by extension alone.
const unfetchableSRSURL = "http://%zz/blocklist.srs"
// fetchDetourModel is a router-shaped configuration with one remote blocklist, so
// the fetch detour has something to be stamped on, plus the node and group a
// working detour would name.
func fetchDetourModel(globalsDetour string, profiles ...model.Profile) *model.Model {
g := model.DefaultGlobals()
g.KillSwitch = "closed"
g.Untunnelable = netplane.UntunnelableBlock
g.DNSFilter = true
g.ResolverDefault = "up"
g.FetchDetour = globalsDetour
if len(profiles) > 0 {
g.ActiveProfile = profiles[0].Name
}
return &model.Model{
Globals: g,
Profiles: profiles,
Inbounds: []model.Inbound{{
Name: "lan", Enabled: true, Type: "tproxy", Network: "lan",
TproxyPort: 12345, TCP: true, UDP: true,
}},
Resolvers: []model.Resolver{{Name: "up", Type: "doh", Address: "https://dns.quad9.net/dns-query"}},
Nodes: []model.Node{{
Name: "tokyo", Enabled: true,
URI: "vless://11111111-1111-1111-1111-111111111111@example.com:443?security=tls&sni=example.com#tokyo",
}},
Groups: []model.Group{{Name: "auto", Source: "manual", Strategy: "single", Nodes: []string{"tokyo"}}},
Blocklists: []model.Blocklist{{
Name: "apple", Enabled: true, Source: "url", URL: unfetchableSRSURL, UpdateInterval: "24h",
}},
}
}
// findingsMentioning returns the PUBLISHED findings whose text contains sub. It
// never searches by tag: the tag is what is under test and must not also be the
// search key, or a renamed tag would simply return nothing and pass.
func findingsMentioning(t *testing.T, m *model.Model, sub string) []Warning {
t.Helper()
_, warns, err := generate.GenerateWithWarnings(m)
if err != nil {
t.Fatalf("generate.GenerateWithWarnings: %v", err)
}
if len(warns) == 0 {
t.Fatalf("the producer emitted NO warnings at all — the fixture is what failed, not the grading")
}
var out []Warning
for _, w := range gatherWarnings(m, warns, nil, nil) {
if strings.Contains(w.Message, sub) {
out = append(out, w)
}
}
return out
}
// TestFetchDetourNotAppliedIsCriticalFromTheRealProducer is the coupling, not a
// fixture: it configures a fetch detour that names nothing, runs the real
// generator, and asks the real classifier what the real text grades as. The only
// way it passes is if the tag generate stamps is the tag notAppliedTags knows.
//
// MUTATION THAT MUST KILL IT: rename fetchDetourNotAppliedTag in
// generate/dnsfilter.go, or drop tagFetchDetourNotApplied from notAppliedTags.
func TestFetchDetourNotAppliedIsCriticalFromTheRealProducer(t *testing.T) {
got := findingsMentioning(t, fetchDetourModel("group:ghost"), "group:ghost")
if len(got) != 1 {
t.Fatalf("want exactly 1 finding naming the bad detour, got %d: %+v", len(got), got)
}
w := got[0]
if w.Severity != SeverityCritical {
t.Fatalf("graded %q, want %q. Every list, geo and rule-set download is leaving over the plain "+
"WAN with the router's real address while the configuration says otherwise — if the panel's "+
"banner is dark for that, it is dark for the whole class.\nmessage:\n%s",
w.Severity, SeverityCritical, w.Message)
}
if w.Section != "globals" || w.Name != "fetch_detour" {
t.Errorf("a globals-level fault must be attributed to globals; section=%q name=%q", w.Section, w.Name)
}
if !strings.Contains(w.Message, tagFetchDetourNotApplied) {
t.Errorf("the tag must survive into the published message — it is the `logread` grep handle:\n%s", w.Message)
}
// CONTROL 1: the same model with a detour that RESOLVES must produce nothing.
// Without it, a classifier that graded every generate warning critical would
// pass everything above.
for _, w := range findingsMentioning(t, fetchDetourModel("group:auto"), "fetch_detour") {
if strings.Contains(w.Message, tagFetchDetourNotApplied) {
t.Errorf("a detour that resolves must produce no not-applied finding:\n%s", w.Message)
}
}
// CONTROL 2: and so must NOT CONFIGURING the option at all.
for _, w := range findingsMentioning(t, fetchDetourModel(""), "fetch_detour") {
if strings.Contains(w.Message, tagFetchDetourNotApplied) {
t.Errorf("an unset optional override is not a fault:\n%s", w.Message)
}
}
}
// TestFetchDetourFromAProfileIsBadgedOnThatProfile: the fault is attributed to
// whoever wrote the value. A profile override only misbehaves on its own uplink,
// so an operator who cannot see WHICH profile carries the bad name has nothing to
// act on — and the panel would badge the wrong page.
func TestFetchDetourFromAProfileIsBadgedOnThatProfile(t *testing.T) {
m := fetchDetourModel("direct", model.Profile{
Name: "mobile-uplink", Enabled: true, FetchDetour: "group:sim-bypass",
})
got := findingsMentioning(t, m, "group:sim-bypass")
var tagged []Warning
for _, w := range got {
if strings.Contains(w.Message, tagFetchDetourNotApplied) {
tagged = append(tagged, w)
}
}
if len(tagged) != 1 {
t.Fatalf("want exactly 1 tagged finding, got %d: %+v", len(tagged), got)
}
if tagged[0].Severity != SeverityCritical {
t.Errorf("severity = %q, want critical", tagged[0].Severity)
}
if tagged[0].Section != "profile" || tagged[0].Name != "mobile-uplink" {
t.Errorf("section=%q name=%q, want profile/mobile-uplink so the panel badges the profile that "+
"carries the value", tagged[0].Section, tagged[0].Name)
}
}
// --- 4b: a stale list is not a missing list ----------------------------------
// The two texts, VERBATIM from generate/ruleset.go remoteRuleSetAs. They are typed
// out here because apply cannot stub generate's reachability probe or seed
// sing-box's cache DB from outside the package, so the producer-driven form is not
// available for this pair. The producer half of the binding lives in
// generate/fetchdetour_lists_test.go
// (TestFetchDetourCachedListAlsoSaysAppliedFromTheCache), which drives the REAL
// cold-start branch and asserts the phrase this file grades on still appears in
// it — with the no-cache control beside it. Reword the branch and that test fails
// by name.
const (
rulesetFromCacheText = `RULESET-FROM-CACHE: blocklist "apple" could not be checked against its source ` +
`(its source "https://x/geosite-apple.srs" is unreachable right now), but the cache holds a copy of ` +
`rule-set "bl-apple-apple" that the engine can load (fetched 2026-07-26T09:00:00Z), so it is ` +
`APPLIED FROM THE CACHE — this is what lets a reboot with no working WAN still come up with its lists. ` +
`That copy will not change until the source is reachable again, so the list is as old as the timestamp says.`
rulesetNotAppliedText = `RULESET-NOT-APPLIED: blocklist "apple" is configured but NOT ACTIVE: its source ` +
`"https://x/geosite-apple.srs" is unreachable right now, so rule-set "bl-apple-apple" was omitted and ` +
`matches NOTHING until it loads (a blocklist blocks nothing; a routing rule is skipped). ` +
`There is no usable copy in the cache either, so there is nothing to fall back to.`
)
// TestCachedRulesetIsDegradedAndOmittedRulesetIsCritical is the WHOLE instrument
// in one test, because a one-sided assertion here is worthless.
//
// The classifier had exactly one behaviour for both texts — section "blocklist" is
// in protectionSections, so anything about it graded critical — and the panel
// painted a list that IS loaded and IS matching in the same red as a list that is
// absent. On the router this ships to, the cached case is not a blip: the SIM
// operator's whitelist makes the source permanently unreachable, so that red was
// going to be lit on every apply, forever, until nobody read the next one.
//
// The POSITIVE CONTROL is the second half: the same instrument, over the sibling
// text produced by the sibling branch of the same function, must still say
// critical. Without it, "the marker demoted it" is indistinguishable from "this
// classifier cannot produce a critical for a blocklist at all" — which is the
// failure mode a marker list invites, and the one this project has already paid
// for once.
func TestCachedRulesetIsDegradedAndOmittedRulesetIsCritical(t *testing.T) {
cached := oneGenerateFinding(t, rulesetFromCacheText)
if cached.Severity != SeverityWarning {
t.Errorf("a list served FROM THE CACHE graded %q, want %q — it is applied and it is matching; the "+
"copy is stale, which is a degradation, not a protection gap. A permanent red here is how the "+
"next real red stops being read.", cached.Severity, SeverityWarning)
}
omitted := oneGenerateFinding(t, rulesetNotAppliedText)
if omitted.Severity != SeverityCritical {
t.Fatalf("POSITIVE CONTROL FAILED: a list that was OMITTED graded %q, want %q. The demotion above "+
"proves nothing if this instrument can no longer produce a critical for a blocklist at all.",
omitted.Severity, SeverityCritical)
}
// Both must still be attributed to the list, or the panel cannot tell the
// operator WHICH list it is talking about in either state.
for _, w := range []Warning{cached, omitted} {
if w.Section != "blocklist" || w.Name != "apple" {
t.Errorf("section=%q name=%q, want blocklist/apple:\n%s", w.Section, w.Name, w.Message)
}
}
}
// oneGenerateFinding publishes a single generate-channel text and returns the one
// finding it becomes.
func oneGenerateFinding(t *testing.T, text string) Warning {
t.Helper()
got := collectWarnings(blockModel(), []string{text}, nil, nil)
var out []Warning
for _, w := range got {
if w.Section == "blocklist" {
out = append(out, w)
}
}
if len(out) != 1 {
t.Fatalf("want exactly 1 blocklist finding, got %d: %+v", len(out), got)
}
return out[0]
}
// --- 4a: the validator nobody called -----------------------------------------
// TestValidateProfilesReachesTheStatus: model.ValidateProfiles existed, was
// tested, and was referenced by nothing but its own test — model.(*Model).Validate
// does not call it, so every per-profile override went unchecked on every path
// that matters. This pins that its findings now reach the published set, with the
// Section/Name the panel needs to badge the profile.
//
// A profile override is the setting whose failure is invisible by construction: it
// only applies while that profile is active, so a typo in the SIM profile's
// resolver is not observable at all until the ethernet cable comes out — at which
// point DNS stops and the config that broke it looks exactly like the config that
// worked.
func TestValidateProfilesReachesTheStatus(t *testing.T) {
m := blockModel()
m.Resolvers = []model.Resolver{{Name: "quad9", Type: "tls", Address: "9.9.9.9"}}
m.Profiles = []model.Profile{{
Name: "mobile-uplink", Enabled: true, ResolverDefault: "yandex-typo",
}}
var found []Warning
for _, w := range collectWarnings(m, nil, nil, nil) {
if w.Section == "profile" {
found = append(found, w)
}
}
if len(found) != 1 {
t.Fatalf("want exactly 1 profile finding, got %d: %+v", len(found), found)
}
for _, want := range []string{"resolver_default", "yandex-typo", "config resolver"} {
if !strings.Contains(found[0].Message, want) {
t.Errorf("the finding must contain %q:\n%s", want, found[0].Message)
}
}
if found[0].Name != "mobile-uplink" {
t.Errorf("Name = %q, want the profile's own name", found[0].Name)
}
// CONTROL: the SAME model with the resolver actually defined must be silent. A
// validator that fired unconditionally would pass everything above.
ok := blockModel()
ok.Resolvers = []model.Resolver{{Name: "yandex-typo", Type: "udp", Address: "77.88.8.8"}}
ok.Profiles = m.Profiles
for _, w := range collectWarnings(ok, nil, nil, nil) {
if w.Section == "profile" {
t.Errorf("a profile whose override names a resolver that exists must be silent:\n%s", w.Message)
}
}
}
// --- PROFILE-OVERRIDE-NOT-APPLIED: graded by the tag, not by one word --------
// TestProfileOverrideNotAppliedIsGradedStructurally.
//
// generate/dns.go emits this when the active WAN profile names a resolver that is
// not usable: the override is dead and the globals value resolves instead. It
// ALREADY graded critical before the tag was added — by accident. The producer's
// sentence contains the word "inert", criticalMarkers contains "inert", and that is
// the whole of why. The word is there because it is the right word; nobody was
// aiming at the classifier, and one rewording would have dropped a real DNS fault a
// full severity with every test in the tree still green.
//
// So the assertions are in three steps, and the middle one is the actual claim:
//
// 1. the REAL producer's REAL text grades critical (and is attributed);
// 2. the same text with the word "inert" gone STILL grades critical — that is what
// "structural" means, and it is false of the old classifier;
// 3. POSITIVE CONTROL: strip the TAG as well and the grading drops. Without this,
// step 2 would be satisfied by a classifier that had simply started returning
// critical for everything.
func TestProfileOverrideNotAppliedIsGradedStructurally(t *testing.T) {
m := fetchDetourModel("", model.Profile{
Name: "mobile-uplink", Enabled: true, ResolverDefault: "yandex-typo",
})
real := producedWarningWithTag(t, m, tagProfileOverrideNotApplied)
first := oneProfileFinding(t, real)
if first.Severity != SeverityCritical {
t.Fatalf("the real producer text graded %q, want %q:\n%s", first.Severity, SeverityCritical, real)
}
if first.Name != "mobile-uplink" {
t.Errorf("Name = %q, want the profile that carries the bad override", first.Name)
}
// (2) THE CLAIM. Nothing about the grading may depend on that one word.
noInert := strings.ReplaceAll(real, "inert", "not in force")
if noInert == real {
t.Fatalf("fixture: the producer text no longer contains \"inert\", so step 2 tests nothing. "+
"Re-read it and pick the word this step is supposed to neutralise:\n%s", real)
}
if got := oneProfileFinding(t, noInert).Severity; got != SeverityCritical {
t.Fatalf("with the word \"inert\" reworded the grading fell to %q. The severity is still riding on "+
"a single word in someone else's prose, which is what %q was added to stop.",
got, tagProfileOverrideNotApplied)
}
// (3) POSITIVE CONTROL. Remove the tag too and the instrument must say something
// ELSE, or steps 1 and 2 prove only that it cannot say anything but critical.
noTag := strings.TrimPrefix(noInert, tagProfileOverrideNotApplied+": ")
if noTag == noInert {
t.Fatalf("fixture: the producer text does not start with %q, so the control cannot strip it:\n%s",
tagProfileOverrideNotApplied, noInert)
}
if got := oneProfileFinding(t, noTag).Severity; got == SeverityCritical {
t.Fatalf("CONTROL FAILED: an untagged, unmarked profile note also graded critical, so this " +
"classifier cannot distinguish anything and steps 1-2 measured nothing.")
}
}
// producedWarningWithTag runs the real generator and returns the one warning
// carrying tag, failing if there is not exactly one.
func producedWarningWithTag(t *testing.T, m *model.Model, tag string) string {
t.Helper()
_, warns, err := generate.GenerateWithWarnings(m)
if err != nil {
t.Fatalf("generate.GenerateWithWarnings: %v", err)
}
var hits []string
for _, w := range warns {
if strings.HasPrefix(w, tag+":") {
hits = append(hits, w)
}
}
if len(hits) != 1 {
t.Fatalf("want exactly 1 %s warning from the real producer, got %d. The fixture is what failed, "+
"not the grading:\n %s", tag, len(hits), strings.Join(warns, "\n "))
}
return hits[0]
}
// oneProfileFinding publishes one generate-channel text and returns the single
// finding filed under section "profile".
func oneProfileFinding(t *testing.T, text string) Warning {
t.Helper()
var out []Warning
for _, w := range collectWarnings(blockModel(), []string{text}, nil, nil) {
if w.Section == "profile" {
out = append(out, w)
}
}
if len(out) != 1 {
t.Fatalf("want exactly 1 profile finding, got %d, for:\n%s", len(out), text)
}
return out[0]
}
// --- the three-state fetch_via, as the fetch itself reads it -----------------
// TestSubFetchNeedsEngine pins the table apply.UpdateSubscription decides on.
//
// The row that matters is the LAST group: `fetch_via` ABSENT no longer means
// direct. It inherits the general detour, so the same subscription needs the
// engine or does not depending on a setting written somewhere else entirely —
// which is exactly the state the old `EqualFold(FetchVia, "proxy")` could not
// express and answered `direct` for.
func TestSubFetchNeedsEngine(t *testing.T) {
cases := []struct {
name string
via model.FetchViaMode
detour string
want bool
}{
{"explicit direct never needs the engine", model.FetchViaDirect, "", false},
{"explicit direct, stale detour beside it, still no engine", model.FetchViaDirect, "group:auto", false},
// Unchanged on purpose: an empty detour under `proxy` resolves to the box's
// own direct outbound, and warnings.go grades that critical by name. Moving
// it onto a process-local client would change which finding the operator
// gets while leaving the disclosure identical.
{"proxy with a detour", model.FetchViaProxy, "group:auto", true},
{"proxy with NO detour still goes through the box", model.FetchViaProxy, "", true},
{"absent + no general detour = plain fetch", model.FetchViaInherit, "", false},
{"absent + general detour 'direct' = plain fetch", model.FetchViaInherit, "direct", false},
{"absent + general detour 'DIRECT' = plain fetch", model.FetchViaInherit, " DIRECT ", false},
{"absent + general detour naming a group = through the box", model.FetchViaInherit, "group:sim-bypass", true},
{"absent + general detour naming a chain = through the box", model.FetchViaInherit, "chain:swan", true},
// Never reached (SubscriptionFetchDetour refuses first), and listed so a
// value that is not a decision can never be answered as if it were one.
{"unknown is not answered as proxy", model.FetchViaUnknown, "group:auto", false},
}
for _, tc := range cases {
if got := subFetchNeedsEngine(tc.via, tc.detour); got != tc.want {
t.Errorf("%s: subFetchNeedsEngine(%v, %q) = %v, want %v", tc.name, tc.via, tc.detour, got, tc.want)
}
}
}
// TestInheritedDetourThatNamesNothingIsReported closes the gap the three-state
// contract opened on this side.
//
// Before, a subscription with no `fetch_via` was fetched in the clear, always, and
// could not fail for a reason written somewhere else. Now the general detour
// applies to it — so a typo in `globals.fetch_detour` stops every such
// subscription refreshing, and nothing else would say so: generate's
// FETCH-DETOUR-NOT-APPLIED needs a remote rule-set to stamp the detour on, and a
// router with none has no producer for this at all.
//
// It is a `warning`, not a critical: HTTPClient refuses the unknown name, so the
// fetch FAILS rather than falling back to the clear. Nothing is disclosed; a node
// list goes stale.
func TestInheritedDetourThatNamesNothingIsReported(t *testing.T) {
m := blockModel()
m.Globals.FetchDetour = "group:ghost"
// A manual node, so this is NOT also the first-boot deadlock. The two findings
// are different faults with different fixes, and a fixture that triggers both
// would let either one satisfy the assertions below.
m.Nodes = []model.Node{{Name: "tokyo", Enabled: true, URI: "ss://tokyo"}}
m.Subscriptions = []model.Subscription{{
Name: "qomar", Enabled: true, URL: "https://provider.example/sub?token=t",
}}
got := subscriptionFindings(t, m)
if len(got) != 1 {
t.Fatalf("want exactly 1 subscription finding, got %d: %+v", len(got), got)
}
if got[0].Severity != SeverityWarning {
t.Errorf("severity = %q, want %q — a refused fetch discloses nothing", got[0].Severity, SeverityWarning)
}
for _, want := range []string{
"sets no `fetch_via`", // why this row is affected at all
"`globals.fetch_detour`", // and WHERE the operator must go to fix it
`no group named "ghost"`, // what is actually wrong
"every subscription", // the blast radius: it is not this row's fault
} {
if !strings.Contains(got[0].Message, want) {
t.Errorf("the message must contain %q:\n%s", want, got[0].Message)
}
}
// CONTROL 1: a general detour that RESOLVES must be silent.
ok := blockModel()
ok.Globals.FetchDetour = "node:tokyo"
ok.Nodes = []model.Node{{Name: "tokyo", Enabled: true, URI: "ss://tokyo"}}
ok.Subscriptions = m.Subscriptions
if got := subscriptionFindings(t, ok); len(got) != 0 {
t.Errorf("a general detour that names a real node must be silent: %+v", got)
}
// CONTROL 2: and so must NO general detour, which is what "not set" has always
// meant and must keep meaning.
plain := blockModel()
plain.Nodes = m.Nodes
plain.Subscriptions = m.Subscriptions
if got := subscriptionFindings(t, plain); len(got) != 0 {
t.Errorf("with no general detour the fetch is plain, exactly as before: %+v", got)
}
// CONTROL 3: an EXPLICIT `fetch_via=direct` opts out of the general setting, so
// the bad globals value must not be reported against it. This is the whole point
// of the migration having written `direct` onto every existing subscription.
explicit := blockModel()
explicit.Globals.FetchDetour = "group:ghost"
// The node again: this control is about ONE thing, and a nodeless model would
// also arm the deadlock detector, so a regression there could fail it for a
// reason that has nothing to do with what it asserts.
explicit.Nodes = m.Nodes
explicit.Subscriptions = []model.Subscription{{
Name: "qomar", Enabled: true, URL: "https://provider.example/sub?token=t", FetchVia: "direct",
}}
if got := subscriptionFindings(t, explicit); len(got) != 0 {
t.Errorf("an explicit fetch_via=direct ignores the general detour and must not be badged: %+v", got)
}
}
// subscriptionFindings is what the panel receives for section "subscription".
func subscriptionFindings(t *testing.T, m *model.Model) []Warning {
t.Helper()
var out []Warning
for _, w := range collectWarnings(m, nil, nil, nil) {
if w.Section == subscriptionSection {
out = append(out, w)
}
}
return out
}
// TestBootstrapDeadlockSeesAnInheritedDetour: the deadlock detector asks "is there
// any subscription that could be fetched with NO outbound?", and it used to answer
// that from `fetch_via` alone. Once an absent `fetch_via` began inheriting
// globals.fetch_detour, that answer became wrong in the expensive direction — the
// detector would declare the deadlock broken by a subscription that is itself part
// of it, and go silent on a router whose LAN is dark and whose panel looks healthy.
func TestBootstrapDeadlockSeesAnInheritedDetour(t *testing.T) {
stuck := blockModel()
stuck.Globals.FetchDetour = "group:auto"
stuck.Subscriptions = []model.Subscription{{
Name: "qomar", Enabled: true, URL: "https://provider.example/sub?token=t",
// No FetchVia at all: the general detour applies.
}}
var got []Warning
for _, w := range collectWarnings(stuck, nil, nil, nil) {
if strings.Contains(w.Message, "DEADLOCK") {
got = append(got, w)
}
}
if len(got) != 1 {
t.Fatalf("want the deadlock named exactly once, got %d: %+v", len(got), got)
}
if got[0].Severity != SeverityCritical || got[0].Name != "qomar" {
t.Errorf("severity=%q name=%q, want critical/qomar", got[0].Severity, got[0].Name)
}
// CONTROL: the same subscription with NO general detour can bootstrap in the
// clear, so there is no deadlock and the detector must stay silent. Without
// this, a detector that fired on every proxy-less config would pass above.
free := blockModel()
free.Subscriptions = stuck.Subscriptions
for _, w := range collectWarnings(free, nil, nil, nil) {
if strings.Contains(w.Message, "DEADLOCK") {
t.Errorf("a subscription that fetches in the clear breaks the deadlock; it must not be reported:\n%s",
w.Message)
}
}
}
+175 -26
View File
@@ -114,6 +114,26 @@ const (
tagRulesetNotApplied = "RULESET-NOT-APPLIED" // generate/ruleset.go:821,997,1242
tagDNSFilterNotApplied = "DNS-FILTER-NOT-APPLIED" // generate/dnsfilter.go:132
tagDeviceFilterNotApplied = "DEVICE-FILTER-NOT-APPLIED" // generate/devices.go deviceFilterNotAppliedTag
// generate/dnsfilter.go fetchDetourNotAppliedTag. The operator pointed every
// list/geo/rule-set download at a tunnel and the download is going out over the
// plain WAN instead, because the name resolves to nothing this config builds.
// Two protections are off at once — the disclosure fetch_detour prevents, and,
// on an uplink that refuses those sources, the lists themselves. It carries an
// entity prefix (`profile "x"` / `globals "fetch_detour"`), but neither section
// is in protectionSections, so without the tag it would publish as a plain
// warning.
tagFetchDetourNotApplied = "FETCH-DETOUR-NOT-APPLIED"
// generate/dns.go profileOverrideNotAppliedTag. The active WAN profile named a
// resolver that is not usable, so its DNS override is INERT and the globals
// value is what resolves while that profile is live.
//
// It already graded critical before this entry existed — but by ACCIDENT: the
// producer's sentence happens to contain the word "inert", which is in
// criticalMarkers. That word is there because it is the right word, not because
// anyone was aiming at the classifier, and one rewording would have dropped the
// finding a whole severity with every test still green. The tag makes the
// grading STRUCTURAL, which is the same reason the three above exist.
tagProfileOverrideNotApplied = "PROFILE-OVERRIDE-NOT-APPLIED"
)
var notAppliedTags = []string{
@@ -131,6 +151,8 @@ var notAppliedTags = []string{
// wash that distinction out — a device that is simply off the network right now
// is not a protection gap.
tagDeviceFilterNotApplied,
tagFetchDetourNotApplied,
tagProfileOverrideNotApplied,
}
// criticalMarkers are substrings that identify a warning as "protection you
@@ -217,8 +239,26 @@ var protectionSections = map[string]bool{
// compiled earlier and keeps blocking, which is a degradation, not a gap.
var degradedProtectionMarkers = []string{
"continuing with the copy compiled earlier", // generate/ruleset.go:819 — stale but blocking
"is IGNORED", // generate/ruleset.go:445,472 — a redundant field, the list loads
"bad update_interval", // generate/ruleset.go:862,1010 — falls back to the default interval
// generate/ruleset.go remoteRuleSetAs, the RULESET-FROM-CACHE branch: the source
// could not be checked, and sing-box's own cache holds a copy this build decodes,
// so the rule-set IS HANDED TO THE ENGINE and IS matching. That is the sibling of
// the marker above — the local-compile path says "the copy compiled earlier", the
// engine-cache path says this — and it was graded critical purely because its
// section is "blocklist"/"ruleset".
//
// This one is not a rounding error. Behind the SIM uplink the source is
// PERMANENTLY unreachable, so this warning is the STEADY STATE: the panel's alarm
// banner would be red on every apply forever, over lists that are working. That is
// how the next red — RULESET-NOT-APPLIED, the one where the list really is absent —
// stops being read. The two must not look the same, and by this file's own
// definition they are not: "protection you configured is not in effect" is false
// here and true there.
//
// The copy is STALE, and the message says how stale (it carries the fetch
// timestamp), which is exactly the "degraded, not absent" this list is for.
"APPLIED FROM THE CACHE", // generate/ruleset.go remoteRuleSetAs — stale but matching
"is IGNORED", // generate/ruleset.go:445,472 — a redundant field, the list loads
"bad update_interval", // generate/ruleset.go:862,1010 — falls back to the default interval
}
// infoMarkers identify operational notes that are not protection gaps.
@@ -435,6 +475,36 @@ func gatherWarnings(m *model.Model, generateWarnings, netplaneWarnings []string,
// for the same reason as the two above — no producer emits it, because from
// every producer's point of view the configuration is fine.
out = append(out, subscriptionBootstrapWarnings(m)...)
// Per-profile overrides that name a resolver or a detour target this
// configuration does not have.
//
// THIS CALL IS THE POINT. model.ValidateProfiles was written, tested and then
// referenced by nothing but its own test — model.(*Model).Validate does not call
// it, so every profile override went unchecked on every path. That is the defect
// class this project has already paid for once (CI running two tests out of 116
// files): a check nobody runs reads exactly like a check that passes.
//
// It is called HERE rather than folded into model.Validate because Validate is
// another tree's file this pass; the two are equivalent from the panel's side,
// since configWarnings and this slice land in the same published set with the
// same Section/Name shape. If model.Validate ever adopts it, delete this and the
// findings keep arriving — but do not delete this before that.
//
// Severity: whatever classify() makes of the text, same as any other config
// warning — a profile override that cannot resolve is a degradation on ONE
// uplink, not a protection that is off right now. The fetch_detour half is the
// exception and it is generate's to report, at critical, when that profile is
// the ACTIVE one (FETCH-DETOUR-NOT-APPLIED).
if m != nil {
for _, cw := range model.ValidateProfiles(m.Profiles, m.Resolvers) {
out = append(out, Warning{
Severity: SeverityWarning,
Section: cw.Section,
Name: cw.Name,
Message: cw.Message,
})
}
}
for _, t := range generateWarnings {
// Parse the entity prefix FIRST, so severity can be decided from the section
// (a warning about a blocklist is a protection gap however it is worded).
@@ -944,6 +1014,7 @@ func subscriptionFetchWarnings(m *model.Model) []Warning {
return nil
}
var out []Warning
prof, _ := model.ResolveActiveProfile(m)
for _, s := range m.Subscriptions {
// No URL: subscribe.fetchWithHeader refuses before it builds a request, so
// this one cannot disclose anything by any path. Enabled is deliberately NOT
@@ -951,10 +1022,46 @@ func subscriptionFetchWarnings(m *model.Model) []Warning {
if strings.TrimSpace(s.URL) == "" {
continue
}
// Byte-for-byte the test in Applier.UpdateSubscription and in
// cmd/shaterd's subFetchViaProxy. If this one ever drifts looser than
// those, it reports a leak that is not happening; tighter, and it goes
// quiet on one that is.
// `fetch_via` ABSENT is its own case now, and it is NOT the leak above. The
// operator claimed nothing about this feed; the GENERAL detour applies, and
// if that names something real the fetch travels through it exactly as asked.
// What can still go wrong is the name, and that failure is silent from every
// other angle — generate's FETCH-DETOUR-NOT-APPLIED only fires when the
// configuration also has a remote rule-set to stamp the detour on, and a
// router with no rule-sets at all would have had its subscriptions stop
// refreshing with nothing anywhere saying why.
//
// It is a `warning`, not a critical, for the same reason the unresolvable
// case below is: nothing is disclosed. UpdateSubscription hands the name to
// HTTPClient, which refuses it, and the refresh fails loudly while the cached
// nodes keep working.
if model.ClassifyFetchVia(s.FetchVia) == model.FetchViaInherit {
ov := model.EffectiveFetchDetour(m.Globals, prof)
general := strings.TrimSpace(ov.Value)
if general == "" || strings.EqualFold(general, model.TargetDirect) {
continue // a plain fetch, which is what "not set" has always meant
}
if msg, bad := subFetchDetourFinding(m, subFetchInheritedLead(ov), general, s.Enabled); bad {
out = append(out, Warning{
Severity: SeverityWarning,
Section: subscriptionSection,
Name: s.Name,
Message: msg,
})
}
continue
}
// Byte-for-byte the `proxy` arm of subFetchNeedsEngine — the copy here in
// apply, and its twin in cmd/shaterd (both pinned identical by
// cmd/shaterd/subfetchparity_test.go, which compares them as source). If
// this one ever drifts looser than those, it reports a leak that is not
// happening; tighter, and it goes quiet on one that is.
//
// The name to look for used to be cmd/shaterd's subFetchViaProxy. Schema v3
// gave fetch_via a third state and that two-way predicate could not express
// it; it is now subFetchRouteOf + subFetchNeedsEngine. The INHERIT case is
// handled by the branch above, which continues, so reaching here means
// fetch_via was written out — and only `proxy` is a leak worth reporting.
if !strings.EqualFold(strings.TrimSpace(s.FetchVia), "proxy") {
continue
}
@@ -968,7 +1075,7 @@ func subscriptionFetchWarnings(m *model.Model) []Warning {
})
continue
}
if msg, bad := subFetchDetourFinding(m, detour, s.Enabled); bad {
if msg, bad := subFetchDetourFinding(m, subFetchProxyLead, detour, s.Enabled); bad {
out = append(out, Warning{
Severity: SeverityWarning,
Section: subscriptionSection,
@@ -980,6 +1087,29 @@ func subscriptionFetchWarnings(m *model.Model) []Warning {
return out
}
// The opening clause of an unresolvable-detour finding: WHICH setting holds the
// name that does not resolve. It is a parameter and not a constant because the
// same four messages now serve two different settings, and a message that names
// the wrong one is worse than no message: an operator told "`fetch_via` is
// \"proxy\" and the detour is …" about a subscription whose `fetch_via` is EMPTY
// goes looking for a field that is not there, concludes the warning is wrong, and
// stops reading the list. The alternative — a second copy of the four texts — is
// how two wordings drift into two different claims about one behaviour.
const subFetchProxyLead = "`fetch_via` is \"proxy\" and the detour"
// subFetchInheritedLead is the same clause for a subscription that expressed no
// opinion and got the general detour. It names WHERE the value lives, because the
// subscription's own row — the row the panel badges — is the one place it is not.
func subFetchInheritedLead(ov model.Override) string {
where := "`globals.fetch_detour`"
if name, ok := strings.CutPrefix(ov.From, "profile:"); ok {
where = "the active profile " + strconv.Quote(name) + "'s `fetch_detour`"
}
return "This subscription sets no `fetch_via`, so it uses the general fetch detour from " + where +
" — which applies to every subscription that does not override it, so the fault is there and not " +
"on this row. That detour"
}
// subscriptionBootstrapWarnings names THE FIRST-BOOT DEADLOCK.
//
// # The state
@@ -1040,17 +1170,35 @@ func subscriptionBootstrapWarnings(m *model.Model) []Warning {
}
}
// Half 2: is there any subscription that could be fetched with no outbound?
//
// The question is about the RESOLVED detour, not about `fetch_via` alone. This
// loop used to answer "not proxy => can bootstrap", which stopped being true the
// moment an absent `fetch_via` began inheriting globals.fetch_detour: with
// `globals.fetch_detour='group:sim-bypass'` such a subscription needs the engine
// exactly like an explicit `proxy` one, and the old test would have declared the
// deadlock broken by a subscription that is itself part of it — the detector
// going silent on the state it exists to name.
//
// prof is the ACTIVE profile, because that is what apply.UpdateSubscription will
// resolve against when it actually runs the fetch. A profile that is not active
// cannot rescue or cause this.
prof, _ := model.ResolveActiveProfile(m)
var stuck []string
anyScheduled := false
for _, s := range m.Subscriptions {
if strings.TrimSpace(s.URL) == "" {
continue // cannot fetch at all; not this fault
}
if !strings.EqualFold(strings.TrimSpace(s.FetchVia), "proxy") {
return nil // a direct fetch needs no engine — it can bootstrap the rest
resolved, ok := model.SubscriptionFetchDetour(s, m.Globals, prof)
if !ok {
// `fetch_via` is a value that is not a decision. UpdateSubscription
// refuses it outright, so this subscription bootstraps nothing — but the
// fault is the unreadable value, which model.ValidateSubscriptions names
// on its own channel. Not stuck, not a rescue: skipped.
continue
}
detour := strings.TrimSpace(s.FetchDetour)
if detour == "" || strings.EqualFold(detour, "direct") {
detour := strings.TrimSpace(resolved.Value)
if detour == "" || strings.EqualFold(detour, model.TargetDirect) {
return nil // resolves to the direct outbound: it leaks, but it is not stuck
}
stuck = append(stuck, s.Name)
@@ -1092,7 +1240,8 @@ func subBootstrapDeadlockMessage(stuck []string, scheduled bool) string {
}
return fmt.Sprintf(
"DEADLOCK: this router cannot fetch %s and cannot fix that by itself. Every subscription it has is "+
"pulled THROUGH the tunnel (`fetch_via=proxy` with a detour), the tunnel is built out of nodes that "+
"pulled THROUGH the tunnel (`fetch_via=proxy` with a detour, or no `fetch_via` at all and a "+
"`globals.fetch_detour` that names one), the tunnel is built out of nodes that "+
"only a subscription fetch can supply, and there are none left — no manual node, no egress, and no "+
"cached node on disk (/etc/shater/subs; a wipe or a reflash that did not keep it is the usual "+
"cause). So the fetch waits for the tunnel and the tunnel waits for the fetch. %s If your LAN is "+
@@ -1175,18 +1324,18 @@ func subFetchDirectMessage(detour string, enabled bool) string {
// The prefix list is positive and closed for the same reason the one in
// chainOutboundTag is: an unlisted spelling must fall to the bare-name branch,
// which is what engine.ViaToTag does with it, rather than into a silent "fine".
func subFetchDetourFinding(m *model.Model, detour string, enabled bool) (string, bool) {
func subFetchDetourFinding(m *model.Model, lead, detour string, enabled bool) (string, bool) {
if rest, ok := cutPrefixFold(detour, "group:"); ok {
return subFetchMissing(detour, "group", strings.TrimSpace(rest), hasGroup(m, rest), enabled)
return subFetchMissing(lead, detour, "group", strings.TrimSpace(rest), hasGroup(m, rest), enabled)
}
if rest, ok := cutPrefixFold(detour, "node:"); ok {
return subFetchMissing(detour, "node", strings.TrimSpace(rest), hasNode(m, rest), enabled)
return subFetchMissing(lead, detour, "node", strings.TrimSpace(rest), hasNode(m, rest), enabled)
}
if rest, ok := cutPrefixFold(detour, "egress:"); ok {
return subFetchMissing(detour, "egress", strings.TrimSpace(rest), hasEgress(m, rest), enabled)
return subFetchMissing(lead, detour, "egress", strings.TrimSpace(rest), hasEgress(m, rest), enabled)
}
if rest, ok := cutPrefixFold(detour, chainViaPrefix); ok {
return subFetchMissing(detour, "chain", strings.TrimSpace(rest), hasChain(m, rest), enabled)
return subFetchMissing(lead, detour, "chain", strings.TrimSpace(rest), hasChain(m, rest), enabled)
}
// Bare name. engine.ViaToTag passes it through verbatim, so it is looked up as
@@ -1200,15 +1349,15 @@ func subFetchDetourFinding(m *model.Model, detour string, enabled bool) (string,
// egress-<name> (netplane.EgressOutboundTag) and a chain's hops are tagged
// chain-<name>-h1..hN, so neither is reachable by its bare name.
if hasEgress(m, detour) {
return subFetchWrongSpelling(detour, "egress", "", enabled), true
return subFetchWrongSpelling(lead, detour, "egress", "", enabled), true
}
if hasChain(m, detour) {
return subFetchWrongSpelling(detour, "chain",
return subFetchWrongSpelling(lead, detour, "chain",
" A chain is only built when something else in the configuration routes to it — a fetch "+
"detour does not count — so if the refusal survives the rename, that is what it is "+
"reporting.", enabled), true
}
return subFetchMissing(detour, "node or group", detour, false, enabled)
return subFetchMissing(lead, detour, "node or group", detour, false, enabled)
}
// subFetchRefusedWhat names WHAT the unresolvable detour actually breaks, which
@@ -1227,29 +1376,29 @@ func subFetchRefusedWhat(enabled bool) string {
// subFetchMissing renders the "names nothing" finding, or nothing at all when the
// name was found. present is the caller's lookup result, threaded through here so
// every branch above reports its miss in one voice.
func subFetchMissing(detour, kind, name string, present, enabled bool) (string, bool) {
func subFetchMissing(lead, detour, kind, name string, present, enabled bool) (string, bool) {
if present {
return "", false
}
return fmt.Sprintf(
"`fetch_via` is \"proxy\" and the detour is %q, but this configuration defines no %s named %q. "+
"%s is %q, but this configuration defines no %s named %q. "+
"Nothing is disclosed by this: the fetch is REFUSED rather than quietly retried over the plain "+
"WAN, so what actually happens is that %s and its "+
"node list goes stale, while the nodes already cached keep working. Correct the name, or restore "+
"the %s it used to point at.", detour, kind, name, subFetchRefusedWhat(enabled), kind), true
"the %s it used to point at.", lead, detour, kind, name, subFetchRefusedWhat(enabled), kind), true
}
// subFetchWrongSpelling renders the finding for a bare name that DOES exist in
// the configuration but not as the kind a bare detour is looked up as. tail is an
// optional extra sentence for a kind that has a caveat of its own.
func subFetchWrongSpelling(detour, kind, tail string, enabled bool) string {
func subFetchWrongSpelling(lead, detour, kind, tail string, enabled bool) string {
return fmt.Sprintf(
"`fetch_via` is \"proxy\" and the detour is the bare name %q. A bare detour is looked up as a node "+
"%s is the bare name %q. A bare detour is looked up as a node "+
"or group tag, and this configuration has no node and no group by that name — it has the %s %q, "+
"whose outbound carries a different tag, so the lookup misses. Write %q instead.%s Until then the "+
"fetch is REFUSED rather than quietly sent over the plain WAN, so nothing is disclosed; what fails "+
"is %s, and its node list goes stale.",
detour, kind, detour, kind+":"+detour, tail, subFetchRefusedWhat(enabled))
lead, detour, kind, detour, kind+":"+detour, tail, subFetchRefusedWhat(enabled))
}
func hasGroup(m *model.Model, name string) bool {
+18 -2
View File
@@ -460,7 +460,15 @@ var diagSafeUCI = map[string]map[string]bool{
"active_profile", "geo_provider", "geosite_url", "geoip_url", "geosite_index_url",
"geoip_index_url", "panel_port", "dns_filter", "dns_intercept", "block_doh", "group_health",
"untunnelable", "l3_tunnel", "untunnelable_egress", "stats_backend", "stats_ring_size",
"stats_timeline_minutes", "stats_max_domains", "stats_retention_disabled", "stats_disk_limit_mb"),
"stats_timeline_minutes", "stats_max_domains", "stats_retention_disabled", "stats_disk_limit_mb",
// fetch_detour names an outbound (`direct`, `group:x`, `node:x`, `egress:x`,
// `chain:x`) — topology, not a credential, and the same vocabulary as
// rule.target and resolver.detour which are already here. It is also the
// single option that explains a router whose lists and subscriptions do not
// download at all, which is the fault this bundle is most often collected
// for: masking it left the reader looking at a healthy `direct` config and
// no reason for the failures beside it.
"fetch_detour"),
"inbound": set("name", "enabled", "type", "network", "tproxy_port", "listen", "port", "auth",
"target_addr", "target_port", "target_network", "tcp", "udp"),
"subscription": set("name", "enabled", "update_interval", "fetch_via", "fetch_detour", "format",
@@ -476,8 +484,16 @@ var diagSafeUCI = map[string]map[string]bool{
"rule": set("name", "enabled", "order", "src", "dst_domain", "dst_ip", "dst_ruleset", "dst_port",
"proto", "target", "egress", "kill", "sched_enabled", "sched_day", "sched_start", "sched_end",
"sched_utc_offset"),
// The three DNS/fetch overrides a profile carries are NAMES of `config resolver`
// sections and of an outbound — the same class as globals' own
// resolver_default/resolver_fallback/fetch_detour a few lines up, which have
// always been printed. Masking them was not a policy choice, it was the list
// predating the fields: a bundle from the SIM uplink showed `option
// endpoint_resolver ***` and nothing at all for the other three, so the one
// section that explains why DNS and the list downloads behave differently on
// that uplink arrived unreadable.
"profile": set("name", "enabled", "priority", "match_iface", "enable_rule", "disable_rule",
"endpoint_resolver"),
"endpoint_resolver", "resolver_default", "resolver_fallback", "fetch_detour"),
"resolver": set("name", "type", "detour", "pool"), // address: see diagMaskAddress
"dns_rule": set("order", "match_domain", "match_src", "resolver"),
"blocklist": set("name", "enabled", "source", "url", "path", "category", "entry", "response", "update_interval"),
@@ -0,0 +1,90 @@
//go:build linux
package main
// The four options that explain a router whose downloads do not work, and which
// the diagnostic bundle used to mask.
//
// `shaterd diag` is collected for exactly one kind of question: "why is this
// router not doing what its config says?" On the SIM uplink the answer is usually
// the fetch detour and the profile's DNS overrides — and the bundle printed
// `option endpoint_resolver ***` and nothing at all for the other three, because
// diagSafeUCI predated the fields. The reader was left looking at what appeared to
// be a plain `direct` configuration with unexplained failures beside it.
//
// None of the four is a credential. They name an outbound and two `config
// resolver` sections — the same class as rule.target and resolver.detour, which
// have always been printed.
import (
"strings"
"testing"
"github.com/sagernet/sing-box/shater/model"
)
// TestDiagKeepsTheFetchAndProfileOverrides asserts the four values survive the
// mask, WITH the control that the same document still masks what it must.
//
// The control is not decoration here: this test would pass identically against a
// maskUCIExport that had stopped masking anything at all, which is the one
// regression that turns a shareable bundle into a leak.
func TestDiagKeepsTheFetchAndProfileOverrides(t *testing.T) {
m := &model.Model{
Globals: model.Globals{FetchDetour: "group:sim-bypass"},
Profiles: []model.Profile{{
Name: "mobile-uplink",
Enabled: true,
ResolverDefault: "yandex",
ResolverFallback: "yandex-sec",
EndpointResolver: "yandex",
FetchDetour: "chain:swan-bypass",
}},
// The control's carrier: a subscription URL is a credential (it carries the
// account token) and must NOT survive.
Subscriptions: []model.Subscription{{
Name: "qomar", Enabled: true, URL: "https://provider.example/sub?token=zzsentinel-token",
}},
}
raw := model.RenderUCIExport(m)
// Fixture guard: if the renderer did not emit these options at all, every
// assertion below would be vacuously true.
for _, want := range []string{
"option fetch_detour 'group:sim-bypass'",
"option resolver_default 'yandex'",
"option resolver_fallback 'yandex-sec'",
"option fetch_detour 'chain:swan-bypass'",
} {
if !strings.Contains(raw, want) {
t.Fatalf("fixture: the renderer emitted no %q; the mask cannot be measured\n%s", want, raw)
}
}
masked, secrets := maskUCIExport(raw)
for _, want := range []struct{ what, line string }{
{"the general fetch detour", "option fetch_detour 'group:sim-bypass'"},
{"the profile's default resolver", "option resolver_default 'yandex'"},
{"the profile's fallback resolver", "option resolver_fallback 'yandex-sec'"},
{"the profile's endpoint resolver", "option endpoint_resolver 'yandex'"},
{"the profile's fetch detour", "option fetch_detour 'chain:swan-bypass'"},
} {
if !strings.Contains(masked, want.line) {
t.Errorf("%s was masked out of the bundle. It names an outbound or a `config resolver`, not a "+
"credential, and it is the setting a broken uplink is diagnosed from.\nwanted: %s\ngot:\n%s",
want.what, want.line, masked)
}
}
// THE CONTROL. Same document, same call: the subscription token is gone, and
// was collected for the document-wide scrub.
if strings.Contains(masked, "zzsentinel-token") {
t.Fatalf("CONTROL FAILED: the subscription token survived the mask, so this instrument is not "+
"masking anything and the assertions above prove nothing:\n%s", masked)
}
if len(secrets) == 0 {
t.Fatal("CONTROL FAILED: no refused values were collected, so the log and nft sections of the " +
"bundle would not be scrubbed either")
}
}
+136 -25
View File
@@ -888,13 +888,18 @@ var (
// runSubUpdate refreshes every target subscription and returns the process exit
// code. Each subscription travels the path it is CONFIGURED for:
//
// - fetch_via != proxy — fetched DIRECT, in this process, exactly as before.
// This is the common case and the one that must not regress: a direct
// subscription keeps working with no daemon at all (cold start, first boot).
// - fetched DIRECT, in this process, exactly as before, when nothing asks for an
// engine outbound: an explicit `fetch_via=direct`, or no `fetch_via` at all
// while the general fetch detour is unset/`direct`. This is the common case and
// the one that must not regress: such a subscription keeps working with no
// daemon at all (cold start, first boot).
//
// - fetch_via == proxy — DELEGATED to the running daemon over the control
// socket (`sub update <name>`), which resolves fetch_detour against the
// engine it owns and does its own cache/UCI write and reconcile. This
// - DELEGATED to the running daemon over the control socket (`sub update
// <name>`) when the bytes must come through an engine outbound: an explicit
// `fetch_via=proxy`, or no `fetch_via` while globals.fetch_detour (or the
// active WAN profile's override) names something other than `direct`. The
// daemon resolves the detour against the engine it owns and does its own
// cache/UCI write and reconcile — see subFetchRouteOf for the whole table. This
// process cannot do it: only ONE process may own the engine, so a CLI verb
// has no outbound to dial through — that is why this used to warn and fetch
// DIRECT, which put the feed URL and the owner's real address on the plain
@@ -921,12 +926,30 @@ func runSubUpdate(m *model.Model, targets []*model.Subscription, logger log.Cont
var updated int
var failed bool
// Pass 1 — the DIRECT subscriptions, in this process.
// The active WAN profile is resolved ONCE for the whole run: it is what decides
// the general fetch detour for every subscription that does not override it, and
// re-resolving it per subscription could hand two of them different answers if
// the model were ever mutated mid-run. The warnings are the apply path's to
// report — this verb must not duplicate them into the CLI's output.
var globals model.Globals
var prof *model.Profile
if m != nil {
globals = m.Globals
prof, _ = model.ResolveActiveProfile(m)
}
// Pass 1 — the subscriptions this process may fetch itself, in the clear.
var viaDaemon []int
for i, s := range targets {
if subFetchViaProxy(s) {
switch subFetchRouteOf(s, globals, prof) {
case subFetchDaemon:
viaDaemon = append(viaDaemon, i)
continue
case subFetchRefused:
logger.Error("sub ", s.Name, ": ", errFetchViaUnknown(s.FetchVia))
failed = true
continue
case subFetchHere:
}
body, ferr := subscribe.Fetch(*s, nil)
if ferr != nil {
@@ -975,7 +998,7 @@ func runSubUpdate(m *model.Model, targets []*model.Subscription, logger log.Cont
s := targets[i]
added, derr := subUpdateViaDaemon(s.Name)
if derr != nil {
logger.Error("sub ", s.Name, " (fetch_via=proxy): ", derr)
logger.Error("sub ", s.Name, " (fetched through the engine): ", derr)
failed = true
continue
}
@@ -993,26 +1016,114 @@ func runSubUpdate(m *model.Model, targets []*model.Subscription, logger log.Cont
return 0
}
// subFetchViaProxy reports whether a subscription is configured to be fetched
// THROUGH the tunnel.
// subFetchRoute is where ONE subscription's refresh has to happen. Three values,
// because `fetch_via` has three states and a fourth non-state:
//
// It is a deliberate byte-for-byte copy of the test inside
// apply.(*Applier).UpdateSubscription (shater/apply/apply.go), and the two MUST
// stay identical. This predicate decides whether the fetch is delegated; the one
// in the daemon decides whether the delegated fetch uses a detour. If they ever
// disagree in the direction "CLI says direct, daemon would have said proxy", the
// CLI fetches on the plain WAN — the exact defect this function exists to close.
func subFetchViaProxy(s *model.Subscription) bool {
return strings.EqualFold(strings.TrimSpace(s.FetchVia), "proxy")
// subFetchHere fetch in this process, in the clear. No daemon needed — the
// cold-start / first-boot path must keep working with no engine.
// subFetchDaemon delegate over the control socket: the bytes have to come
// through an engine outbound, and only the daemon owns the engine.
// subFetchRefused `fetch_via` is not a value this option has. Not a decision, so
// no side is picked; the run reports it and exits non-zero.
type subFetchRoute int
const (
subFetchHere subFetchRoute = iota
subFetchDaemon
subFetchRefused
)
// subFetchRouteOf decides which way one subscription's fetch travels, and is the
// CLI half of a two-half contract: this side chooses whether to delegate, and
// apply.(*Applier).UpdateSubscription then chooses whether the delegated fetch
// uses a detour. If they disagree in the direction "CLI says here, daemon would
// have said engine", the CLI puts the feed URL and this router's real address on
// the plain WAN — the exact defect this function exists to close.
//
// It used to be `EqualFold(FetchVia, "proxy")`, and under schema v2 that was the
// whole truth: an absent `fetch_via` meant a direct fetch, full stop. Under v3 it
// means "the GENERAL fetch detour applies" (globals.fetch_detour, or the active WAN
// profile's override), and that setting exists precisely because on the uplink this
// ships to a direct fetch is the one that does not arrive. So the old two-way test
// answered "here, in the clear" for exactly the configuration the operator set up
// to avoid it, while the daemon — reached through the panel's Refresh button on the
// same subscription — pulled it through the tunnel. Two paths, two answers, one of
// them a silent disclosure.
//
// model.SubscriptionFetchDetour is the ONE resolver of that table, shared with
// apply and the panel, and its ok=false is the fourth state: an unrecognised
// `fetch_via` has no safe silent answer (direct discloses, the general detour
// tunnels a feed nobody asked to tunnel), so it is refused by name here exactly as
// the daemon refuses it.
func subFetchRouteOf(s *model.Subscription, g model.Globals, prof *model.Profile) subFetchRoute {
detour, ok := model.SubscriptionFetchDetour(*s, g, prof)
if !ok {
return subFetchRefused
}
if subFetchNeedsEngine(model.ClassifyFetchVia(s.FetchVia), detour.Value) {
return subFetchDaemon
}
return subFetchHere
}
// errNoDaemonForProxyFetch is the refusal a fetch_via=proxy subscription gets
// when no daemon is running. It is an ERROR, never a fallback: the caller's exit
// code is what stops /etc/init.d/shater-cron from stamping the item as freshly
// updated, so the next tick retries instead of waiting out the full interval.
// subFetchNeedsEngine is a deliberate copy of apply.subFetchNeedsEngine
// (shater/apply/apply.go), and the two MUST stay identical — see subFetchRouteOf
// for what a disagreement costs. It is duplicated rather than imported because the
// daemon-side one is unexported and cmd/shaterd must not reach into apply's
// internals for one predicate; TestSubFetchNeedsEngineTable pins the same table
// apply/fetchdetour_severity_test.go pins on the other side, so a change to one
// that is not made to the other fails a test by name rather than leaking quietly.
//
// - `direct` is the operator saying "in the clear", explicitly: no engine, and no
// dependency on one, which is what makes it the escape from the first-boot
// deadlock.
// - `proxy` needs the engine ALWAYS, empty detour included — an empty detour
// resolves to the box's own `direct` outbound and apply/warnings.go grades that
// critical by name; quietly fetching it here instead would change which of the
// two the operator is warned about and leave the disclosure identical.
// - ABSENT inherits the general detour, so it needs the engine exactly when that
// detour names something other than `direct`.
//
// FetchViaUnknown cannot arrive (subFetchRouteOf refuses it first) and is listed so
// a value that is not a decision can never be answered as if it were one.
func subFetchNeedsEngine(via model.FetchViaMode, detour string) bool {
switch via {
case model.FetchViaDirect:
return false
case model.FetchViaProxy:
return true
case model.FetchViaInherit:
d := strings.TrimSpace(detour)
return d != "" && !strings.EqualFold(d, model.TargetDirect)
case model.FetchViaUnknown:
return false
}
return false
}
// errFetchViaUnknown is the refusal an unrecognised `fetch_via` gets, worded to
// match the daemon's (apply.(*Applier).UpdateSubscription) so an operator who hits
// it through the panel and through the CLI reads the same sentence.
func errFetchViaUnknown(via string) error {
return fmt.Errorf("fetch_via %q is not a value this option has — the accepted values are %s, "+
"and leaving it out means \"use the general fetch_detour\". REFUSING to fetch rather than "+
"guess: the guess this used to make was a fetch in the clear",
via, strings.Join(model.FetchViaNames, "/"))
}
// errNoDaemonForProxyFetch is the refusal a subscription whose feed must travel
// through an engine outbound gets when no daemon is running. Two configurations
// reach it — an explicit `fetch_via=proxy`, and no `fetch_via` at all while the
// general fetch detour names something other than `direct` — so the text names
// both rather than only the one that existed under schema v2.
//
// It is an ERROR, never a fallback: the caller's exit code is what stops
// /etc/init.d/shater-cron from stamping the item as freshly updated, so the next
// tick retries instead of waiting out the full interval.
var errNoDaemonForProxyFetch = errors.New(
"fetch_via=proxy, but no daemon is running and only the daemon owns the engine this " +
"fetch must travel through — REFUSING to fetch direct, which would put the feed URL " +
"this subscription's feed has to travel through an engine outbound (fetch_via=proxy, or no " +
"fetch_via with a non-direct fetch_detour in force), but no daemon is running and only the " +
"daemon owns that engine — REFUSING to fetch direct, which would put the feed URL " +
"and this router's real address on the plain WAN. The cached nodes are untouched; " +
"start the daemon (/etc/init.d/shater start) and this retries on the next cron tick")
+102
View File
@@ -0,0 +1,102 @@
// THE THREE PLACES THAT DECIDE WHERE A SUBSCRIPTION FETCH TRAVELS.
//
// `fetch_via` is read by three independent readers, and the cost of a disagreement
// is stated in the source itself (shater/apply/warnings.go): "If this one ever
// drifts looser than those, it reports a leak that is not happening; tighter, and
// it goes quiet on one that is." The CLI's copy is worse than either — a CLI that
// decides "here, in the clear" for a subscription the daemon would have tunnelled
// puts the feed URL and this router's real address on the plain WAN on every cron
// tick, and the only trace is a syslog line an operator can switch off.
//
// The three:
//
// 1. shater/apply/apply.go — subFetchNeedsEngine, the daemon's decision;
// 2. shater/cmd/shaterd/main.go — subFetchNeedsEngine, this package's copy;
// 3. shater/generate/wgdedup.go — subscriptionDetourSeeds, which decides whether
// the fetch keeps an outbound alive. It now asks model.SubscriptionFetchDetour
// directly instead of carrying a fourth copy of the table, so there is nothing
// there to drift.
//
// (1) and (2) cannot share code: apply's is unexported, and cmd/shaterd is package
// main, so neither can import the other's predicate. A COMMENT saying they must
// match already existed, in all three files, and did not stop (1) and (2) from
// coming apart the moment `fetch_via` gained its third state. So this compares the
// two functions AS SOURCE: same signature, same body, comments and formatting
// ignored. Change one and not the other and this fails by name, with both bodies
// printed, instead of the difference shipping as a silent disclosure.
package main
import (
"bytes"
"go/ast"
"go/parser"
"go/printer"
"go/token"
"os"
"path/filepath"
"testing"
)
// daemonSideSource is the daemon's copy, relative to this package's directory.
// `go test` runs with the package directory as the working directory, so this is
// stable; a wrong path fails loudly below rather than skipping.
var daemonSideSource = filepath.Join("..", "..", "apply", "apply.go")
// funcSource returns the signature and body of a top-level (non-method) function,
// printed from the AST. Parsing WITHOUT parser.ParseComments is what makes the
// comparison about code: the two copies document themselves differently on purpose
// (one talks about the daemon, the other about the CLI) and that must not be a
// failure, while a changed condition must be.
func funcSource(t *testing.T, path, name string) (sig, body string) {
t.Helper()
if _, err := os.Stat(path); err != nil {
t.Fatalf("cannot read %s: %v — this tripwire compares two source files and "+
"cannot do its job without both; fix the path rather than skipping it", path, err)
}
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, path, nil, parser.SkipObjectResolution)
if err != nil {
t.Fatalf("parse %s: %v", path, err)
}
for _, decl := range f.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok || fn.Recv != nil || fn.Name == nil || fn.Name.Name != name {
continue
}
var sigBuf, bodyBuf bytes.Buffer
if err := printer.Fprint(&sigBuf, fset, fn.Type); err != nil {
t.Fatalf("print signature of %s in %s: %v", name, path, err)
}
if err := printer.Fprint(&bodyBuf, fset, fn.Body); err != nil {
t.Fatalf("print body of %s in %s: %v", name, path, err)
}
return sigBuf.String(), bodyBuf.String()
}
t.Fatalf("no top-level func %q in %s — the two halves of this contract are found "+
"BY NAME, so renaming one without the other must fail here", name, path)
return "", ""
}
// TestSubFetchNeedsEngineMatchesTheDaemon is the drift tripwire. It is the reason
// the CLI copy is allowed to exist at all.
func TestSubFetchNeedsEngineMatchesTheDaemon(t *testing.T) {
const name = "subFetchNeedsEngine"
ourSig, ourBody := funcSource(t, "main.go", name)
theirSig, theirBody := funcSource(t, daemonSideSource, name)
if ourSig != theirSig {
t.Fatalf("%s has different signatures:\n cmd/shaterd: func%s\n apply: func%s\n"+
"The two decide the two halves of one fetch and must take the same inputs.",
name, ourSig, theirSig)
}
if ourBody != theirBody {
t.Fatalf("%s has DRIFTED between the CLI and the daemon.\n\n"+
"cmd/shaterd/main.go:\n%s\n\napply/apply.go:\n%s\n\n"+
"A CLI that answers \"fetch here, in the clear\" where the daemon would have "+
"answered \"through the engine\" puts the feed URL and this router's real address "+
"on the plain WAN on every cron tick. Make the two identical; if the daemon's rule "+
"genuinely changed, copy it here rather than approximating it.",
name, ourBody, theirBody)
}
}
+225 -19
View File
@@ -42,8 +42,8 @@ import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"os/signal"
"path/filepath"
"strconv"
"strings"
"sync"
@@ -313,9 +313,15 @@ func TestSubUpdateProxyWithoutDaemonRefusesInsteadOfFetchingDirect(t *testing.T)
// there being no daemon: the code chose the direct path.
//
// Both spellings are covered: the explicit "direct" and the empty value
// model/uci.go defaults to.
// model/uci.go defaults to. The empty one is only direct here because nothing sets
// a general fetch detour in this fixture — under schema v3 that is a fact about
// globals, not about the subscription; see TestSubUpdateInheritedDetourIsDelegated.
//
// A value that is NEITHER used to land here too ("whatever-the-panel-writes"). It
// no longer does: an unrecognised fetch_via is refused rather than demoted to a
// fetch in the clear — see TestSubUpdateUnknownFetchViaIsRefused.
func TestSubUpdateDirectStillFetchesDirect(t *testing.T) {
for _, via := range []string{"direct", "", "DIRECT", "whatever-the-panel-writes"} {
for _, via := range []string{"direct", "", "DIRECT"} {
t.Run("fetch_via="+strconv.Quote(via), func(t *testing.T) {
origin := newDirectOrigin(t)
d := startFakeDaemon(t, true, `{"added":7}`)
@@ -408,25 +414,225 @@ func TestSubUpdateOldDaemonIsNotADirectFallback(t *testing.T) {
}
}
// --- the predicate ----------------------------------------------------------
// --- case 6: the general fetch detour, inherited ----------------------------
// TestSubFetchViaProxyTokens pins the token set. It must accept exactly what
// apply.(*Applier).UpdateSubscription accepts — the two predicates decide the two
// halves of the same fetch, and a disagreement in the "CLI says direct" direction
// is a leak.
func TestSubFetchViaProxyTokens(t *testing.T) {
proxy := []string{"proxy", "PROXY", "Proxy", " proxy ", "\tproxy\n"}
direct := []string{"", "direct", "DIRECT", " ", "proxied", "proxy2", "via-proxy", "tunnel"}
for _, v := range proxy {
if !subFetchViaProxy(&model.Subscription{FetchVia: v}) {
t.Errorf("subFetchViaProxy(%q) = false, want true — this value would be "+
"fetched DIRECT here while the daemon would tunnel it", v)
// TestSubUpdateInheritedDetourIsDelegated is the schema-v3 defect, measured.
//
// The subscription says NOTHING about how it is fetched. Under v2 that meant
// "direct" and the CLI fetched it here; under v3 it means "whatever
// globals.fetch_detour says", and globals says `group:sim-bypass`. The daemon —
// the panel's Refresh button on this very row — pulls it through the engine. The
// CLI must do the same, or the same subscription travels two different ways
// depending on who asked for it, and one of those ways is the plain WAN.
//
// The origin is live and would serve the feed instantly; origin=0 is therefore the
// CLI choosing not to dial it, not an endpoint being unavailable. The paired
// positive control is TestSubUpdateDirectStillFetchesDirect, which reads origin=1
// on the same instrument.
func TestSubUpdateInheritedDetourIsDelegated(t *testing.T) {
for _, tc := range []struct {
name string
globals string
profile *model.Profile
}{
{name: "from globals", globals: "group:sim-bypass"},
{name: "from globals, node form", globals: "node:awgout"},
{
// A profile override must be honoured too: on the router the WAN watcher
// pins `mobile-uplink`, and its fetch_detour is the ONLY reason the feed
// is reachable at all. Reading globals alone would answer `direct`.
name: "profile overrides a direct globals",
globals: "direct",
profile: &model.Profile{Name: "mobile-uplink", Enabled: true, FetchDetour: "group:sim-bypass"},
},
} {
t.Run(tc.name, func(t *testing.T) {
origin := newDirectOrigin(t)
d := startFakeDaemon(t, true, `{"added":7}`)
saves, writes := stubPersistence(t)
m := &model.Model{
Globals: model.Globals{FetchDetour: tc.globals},
Subscriptions: []model.Subscription{{Name: "qomar", Enabled: true, URL: origin.url()}},
}
if tc.profile != nil {
m.Globals.ActiveProfile = tc.profile.Name
m.Profiles = []model.Profile{*tc.profile}
}
if code := runOne(m); code != 0 {
t.Fatalf("exit = %d, want 0 (the daemon answered {\"added\":7})", code)
}
if got := origin.count(); got != 0 {
inForce, _ := model.SubscriptionFetchDetour(m.Subscriptions[0], m.Globals, tc.profile)
t.Errorf("the origin was hit %d time(s) DIRECTLY from this process — the general "+
"fetch detour in force is %q (from %s), so this feed must not go out on the plain WAN",
got, inForce.Value, inForce.From)
}
want := ctlSubUpdatePrefix + "qomar"
if got := d.seen(); len(got) != 1 || got[0] != want {
t.Fatalf("control socket saw %q, want [%q] — the fetch was not delegated", got, want)
}
if len(*saves) != 0 || *writes != 0 {
t.Errorf("the CLI wrote cache=%v uci=%d for a DELEGATED subscription", *saves, *writes)
}
})
}
}
// TestSubUpdateInheritedDirectStaysHere is the other half of the same table, and
// the no-regression case that matters most: with the general detour unset or
// `direct`, a subscription with no `fetch_via` must still be fetched by this
// process, with no daemon required. Getting this wrong would make first boot —
// where nothing owns an engine yet — unable to fetch anything at all.
func TestSubUpdateInheritedDirectStaysHere(t *testing.T) {
for _, detour := range []string{"", "direct", " DIRECT "} {
t.Run(strconv.Quote(detour), func(t *testing.T) {
origin := newDirectOrigin(t)
d := startFakeDaemon(t, true, `{"added":7}`)
stubPersistence(t)
m := &model.Model{
Globals: model.Globals{FetchDetour: detour},
Subscriptions: []model.Subscription{{Name: "plainsub", Enabled: true, URL: origin.url()}},
}
if code := runOne(m); code != 0 {
t.Fatalf("exit = %d, want 0", code)
}
if got := origin.count(); got != 1 {
t.Errorf("origin hits = %d, want 1 — a direct general detour must not start "+
"demanding a running daemon", got)
}
if got := d.seen(); len(got) != 0 {
t.Errorf("control socket saw %q; nothing here needs the engine", got)
}
})
}
}
// TestSubUpdateExplicitDirectBeatsTheGeneralDetour: `fetch_via=direct` is the
// operator saying "in the clear, whatever globals says". It is the documented
// escape from the first-boot deadlock (the feed for the nodes the detour is built
// from), so a general detour must NOT capture it.
func TestSubUpdateExplicitDirectBeatsTheGeneralDetour(t *testing.T) {
origin := newDirectOrigin(t)
d := startFakeDaemon(t, true, `{"added":7}`)
stubPersistence(t)
m := &model.Model{
Globals: model.Globals{FetchDetour: "group:sim-bypass"},
Subscriptions: []model.Subscription{{
Name: "bootstrap", Enabled: true, URL: origin.url(), FetchVia: "direct",
}},
}
if code := runOne(m); code != 0 {
t.Fatalf("exit = %d, want 0", code)
}
if got := origin.count(); got != 1 {
t.Errorf("origin hits = %d, want 1 — an explicit fetch_via=direct must stay direct", got)
}
if got := d.seen(); len(got) != 0 {
t.Errorf("control socket saw %q: fetch_via=direct must never need a daemon", got)
}
}
// --- case 7: a fetch_via that is not a value ---------------------------------
// TestSubUpdateUnknownFetchViaIsRefused. A typo used to be swallowed by
// `EqualFold(FetchVia,"proxy")` and fetched in the clear, successfully, with
// nothing to see. apply.(*Applier).UpdateSubscription refuses it by name; so must
// this. Both endpoints are live, so neither zero is an artifact.
func TestSubUpdateUnknownFetchViaIsRefused(t *testing.T) {
for _, via := range []string{"proxied", "proxy2", "via-proxy", "tunnel", "whatever-the-panel-writes"} {
t.Run(via, func(t *testing.T) {
origin := newDirectOrigin(t)
d := startFakeDaemon(t, true, `{"added":7}`)
saves, writes := stubPersistence(t)
m := &model.Model{Subscriptions: []model.Subscription{{
Name: "typo", Enabled: true, URL: origin.url(), FetchVia: via,
}}}
if code := runOne(m); code == 0 {
t.Fatalf("exit = 0 for fetch_via=%q — cron would stamp this as freshly updated", via)
}
if got := origin.count(); got != 0 {
t.Errorf("origin hits = %d: an unrecognised fetch_via was demoted to a fetch "+
"in the clear, which is the defect", got)
}
if got := d.seen(); len(got) != 0 {
t.Errorf("control socket saw %q: a value that is not a decision must not be "+
"turned into one by tunnelling it either", got)
}
if len(*saves) != 0 || *writes != 0 {
t.Errorf("a refused update wrote cache=%v uci=%d", *saves, *writes)
}
msg := errFetchViaUnknown(via).Error()
for _, want := range []string{"REFUSING", "direct", "proxy"} {
if !strings.Contains(msg, want) {
t.Errorf("refusal message does not contain %q: %s", want, msg)
}
}
})
}
}
// --- the predicates ---------------------------------------------------------
// TestSubFetchNeedsEngineTable pins the CLI's copy of the daemon's predicate
// against the SAME table apply/fetchdetour_severity_test.go's TestSubFetchNeedsEngine
// pins on the other side. The two functions are duplicated (apply's is unexported
// and cmd/shaterd must not reach into it), so this is the tripwire: change one and
// not the other, and one of the two tables goes red by name instead of the CLI
// quietly fetching what the daemon would have tunnelled.
func TestSubFetchNeedsEngineTable(t *testing.T) {
cases := []struct {
name string
via model.FetchViaMode
detour string
want bool
}{
{"explicit direct never needs the engine", model.FetchViaDirect, "", false},
{"explicit direct, stale detour beside it, still no engine", model.FetchViaDirect, "group:auto", false},
{"proxy with a detour", model.FetchViaProxy, "group:auto", true},
{"proxy with NO detour still goes through the box", model.FetchViaProxy, "", true},
{"absent + no general detour = plain fetch", model.FetchViaInherit, "", false},
{"absent + general detour 'direct' = plain fetch", model.FetchViaInherit, "direct", false},
{"absent + general detour 'DIRECT' = plain fetch", model.FetchViaInherit, " DIRECT ", false},
{"absent + general detour naming a group = through the box", model.FetchViaInherit, "group:sim-bypass", true},
{"absent + general detour naming a chain = through the box", model.FetchViaInherit, "chain:swan", true},
{"unknown is not answered as proxy", model.FetchViaUnknown, "group:auto", false},
}
for _, tc := range cases {
if got := subFetchNeedsEngine(tc.via, tc.detour); got != tc.want {
t.Errorf("%s: subFetchNeedsEngine(%v, %q) = %v, want %v", tc.name, tc.via, tc.detour, got, tc.want)
}
}
for _, v := range direct {
if subFetchViaProxy(&model.Subscription{FetchVia: v}) {
t.Errorf("subFetchViaProxy(%q) = true, want false — this would demand a "+
"running daemon for a subscription that never needed one", v)
}
// TestSubFetchRouteOfTokens pins the whole three-way decision, including the
// spellings `fetch_via` accepts. The `proxy` row is what the old two-state
// subFetchViaProxy got right; the `inherit` rows are what it could not express;
// the last row is the state it silently answered `direct` for.
func TestSubFetchRouteOfTokens(t *testing.T) {
g := model.Globals{FetchDetour: "group:sim-bypass"}
cases := []struct {
via string
want subFetchRoute
}{
{"proxy", subFetchDaemon}, {"PROXY", subFetchDaemon}, {"Proxy", subFetchDaemon},
{" proxy ", subFetchDaemon}, {"\tproxy\n", subFetchDaemon},
{"direct", subFetchHere}, {"DIRECT", subFetchHere}, {" direct ", subFetchHere},
// Absent: the general detour decides, and here it names a group.
{"", subFetchDaemon}, {" ", subFetchDaemon},
{"proxied", subFetchRefused}, {"proxy2", subFetchRefused},
{"via-proxy", subFetchRefused}, {"tunnel", subFetchRefused},
}
names := map[subFetchRoute]string{subFetchHere: "here", subFetchDaemon: "daemon", subFetchRefused: "refused"}
for _, tc := range cases {
s := &model.Subscription{Name: "s", FetchVia: tc.via, FetchDetour: "group:auto"}
if got := subFetchRouteOf(s, g, nil); got != tc.want {
t.Errorf("subFetchRouteOf(fetch_via=%q) = %s, want %s", tc.via, names[got], names[tc.want])
}
}
}
+36
View File
@@ -99,6 +99,7 @@ func (b *builder) cacheFilePath() string {
// to install a firmware upgrade at all.
if _, statErr := os.Stat(cacheFilePersistent); statErr == nil {
_ = os.Remove(cacheFilePersistent)
forgetCachedRuleSets(cacheFilePersistent)
}
b.warnf("cache: only %d MiB free on %s (need %d MiB), using tmpfs %s instead — remote rule-sets will be re-downloaded after every reboot, so free some space", avail>>20, cacheDirPersistent, cacheMinFreeSpace>>20, cacheFileFallback)
return cacheFileFallback
@@ -106,3 +107,38 @@ func (b *builder) cacheFilePath() string {
return cacheFilePersistent
}
// engineCacheDBPath names the cache DB the ENGINE will read at start, or "" when
// there will be nothing there worth reading. It is the READ-SIDE TWIN of
// cacheFilePath: same states, same order, but no pruning, no deleting and no
// warnings, because it runs while the rule-sets are being built — BEFORE
// cacheFilePath itself runs (generate.go emits Experimental.CacheFile last).
//
// The cold-start check (coldstart_cache.go) needs this and cannot use
// cacheFilePath: calling that twice per generate would warn twice and prune twice.
// The two must agree, so the standard for a presence-check applies — it has to
// cover everything its acting twin does — and cache_test.go pins the four states
// against each other.
//
// The one state where the twins deliberately DIFFER is the oversized DB. There
// cacheFilePath deletes it and keeps the persistent path; anything read out of that
// DB is about to be thrown away, so the honest answer for a reader is "there is no
// cache", not the path. (If the delete then fails, the DB does survive and we lose a
// proof we could have had — that degrades to the pre-existing behaviour, never past
// it.)
func engineCacheDBPath() string {
fi, err := os.Stat(cacheDirPersistent)
if err != nil || !fi.IsDir() {
// cacheFilePath returns the tmpfs path here. That file may well exist and be
// populated — an engine restart without a reboot keeps it — so it is worth
// reading; it is only a REBOOT that empties it.
return cacheFileFallback
}
if st, err := os.Stat(cacheFilePersistent); err == nil && st.Size() > cacheMaxBytes {
return ""
}
if avail, ok := fsAvailBytes(cacheDirPersistent); ok && avail < cacheMinFreeSpace {
return cacheFileFallback
}
return cacheFilePersistent
}
+327
View File
@@ -0,0 +1,327 @@
package generate
// R5, part two — COLD START: a remote rule-set that is already IN THE CACHE must
// not be dropped just because its source is unreachable.
//
// # The hole this closes, measured
//
// The R5 preflight (ruleset.go) asks one question — "can this URL be fetched right
// now?" — and omits the list when the answer is no. Its own comment records the
// limitation and names the fix:
//
// "a brief source outage at generate time temporarily disables the list even
// though a perfectly good cached copy exists ... The cache-seeding fix above
// removes this limitation entirely and should replace the probe when the
// engine/apply owner can take it."
//
// "Brief" was the assumption, and it is false on the router this was written for.
// Behind a SIM uplink whose operator passes a whitelist, raw.githubusercontent.com
// is not briefly unreachable, it is PERMANENTLY unreachable, and the result was a
// permanent
//
// blocklist "apple": RULESET-NOT-APPLIED: ... its source ... is unreachable
// right now, so rule-set "bl-apple-apple" was omitted and matches NOTHING
// DNS-FILTER-NOT-APPLIED: ... NOT ONE of them could be built right now
//
// on a router whose cache holds a perfectly good copy of every one of those lists.
// Nothing was filtered, forever, and no reconcile was ever going to fix it.
//
// # What is checked, and why it is proof rather than a bet
//
// route/rule.RemoteRuleSet.StartContext (rule_set_remote.go:87-103) does exactly
// this before it considers fetching:
//
// savedSet := cacheFile.LoadRuleSet(tag) // bucket "rule_set", key = TAG
// err := loadBytes(savedSet.Content) // srs.Read / json + Upgrade
// if err == nil { lastUpdated = savedSet.LastUpdated }
// if lastUpdated.IsZero() { fetch(...) } // <- the only fatal branch
//
// So the fetch — the branch that fails router.Start and takes the LAN down with it
// — is skipped iff the cache holds an entry for THIS TAG whose content this build
// can decode and whose LastUpdated is non-zero. cachedRuleSetUsable evaluates that
// same predicate on the same bytes with the same reader. It is not a heuristic
// about the cache being "probably warm": a positive answer means the engine has
// already been shown the path it will take.
//
// Two properties make the answer sturdier than it looks:
//
// - The cache is only ever WRITTEN after a successful load. fetch() calls
// loadBytes first and SaveRuleSet only once it returns nil
// (rule_set_remote.go:289-309), so every entry in there was, at the moment it
// was stored, fully constructible by this binary — matchers included. Re-reading
// it here re-verifies the two things that can change underneath that: on-disk
// corruption, and an .srs format version a DOWNGRADED build can no longer read.
// - The key is the TAG from the config being generated. Rename a list and the
// lookup misses, which is the truthful answer: the engine would miss too.
//
// # Why NOT seed the cache instead, as the R5 block proposed
//
// Writing an empty entry to make StartContext skip the fetch was the plan on
// record. It is rejected here, and not for effort:
//
// - It needs a WRITE lock on a bbolt file the engine holds open for its whole
// life, so it can only happen in the gap between closing the old box and
// starting the new one — engine/apply code, not this package.
// - An EMPTY seeded entry is a lie in the dangerous direction. lastUpdated would
// be non-zero with no rules behind it, so the list would be ACTIVE-and-empty
// rather than absent, and an empty allowlist or an empty ip_cidr routing set is
// the "matches everything" collapse (D1/D2) this package spends most of its
// warnings preventing.
//
// Reading is enough, and reading needs no lock the engine will fight over.
//
// # The lock, and the memo that makes one read last
//
// bbolt takes flock() per open file description, so while the engine is up this
// read-only open is REFUSED — including from inside shaterd, which holds the
// exclusive lock on another fd. That is not a problem, it is the shape of the
// scenario: the generate that matters is the FIRST one after boot, which runs
// before any box exists (apply.go: generate -> engine.Apply -> box.New), and there
// the DB is unlocked.
//
// Later reconciles find it locked, so a positive proof is remembered for the
// process lifetime and re-checked only against the DB file still existing. That is
// sound rather than convenient: sing-box replaces a cached rule-set only with
// another one that just loaded, and never deletes one. The single way an entry can
// disappear under us is cacheFilePath discarding the whole DB, which happens in
// this package and calls forgetCachedRuleSets when it does.
//
// # Degradation is unchanged where it has to be
//
// No cache entry, or one that will not decode, still omits the list with the
// original RULESET-NOT-APPLIED warning. An omitted blocklist blocks nothing; an
// omitted routing rule-set loses its matcher and the rule is skipped. Empty still
// means "matches nothing", never "matches everything", and an unusable list still
// cannot stop the engine from starting.
import (
"bytes"
"errors"
"os"
"strings"
"sync"
"time"
"github.com/sagernet/bbolt"
"github.com/sagernet/sing-box/adapter"
"github.com/sagernet/sing-box/common/srs"
C "github.com/sagernet/sing-box/constant"
"github.com/sagernet/sing-box/option"
"github.com/sagernet/sing/common/json"
)
const (
// cachedRuleSetBucket is the bbolt bucket experimental/cachefile keeps remote
// rule-sets in (its bucketRuleSet, unexported). A copied constant is a drift
// hazard, so the test that matters seeds the DB through sing-box's OWN
// CacheFile.SaveRuleSet and reads it back through this file: if the bucket name
// or the SavedBinary encoding ever moves, that test fails rather than this
// check quietly answering "not cached" forever.
cachedRuleSetBucket = "rule_set"
// coldStartOpenTimeout bounds the flock wait. Tiny on purpose: while the engine
// runs, this open is EXPECTED to fail, and it must fail without stalling an
// apply. (cachefile's own open uses 1s x 10 retries — that is a startup cost it
// can afford and a reconcile cannot.)
coldStartOpenTimeout = 50 * time.Millisecond
// coldStartLockedBackoff suppresses further open attempts after one failed:
// when the DB is locked it is locked for every tag, and a config with a dozen
// lists would otherwise pay the timeout a dozen times per reconcile.
coldStartLockedBackoff = 30 * time.Second
// coldStartMissTTL is how long "this tag is not in the cache" is remembered.
// Short, because the engine fills the entry in as soon as one fetch succeeds
// and the next reconcile should see it. Matches ruleSetProbeTTL's role.
coldStartMissTTL = 90 * time.Second
)
// errCacheUnavailable marks "the DB itself could not be opened" — locked, absent
// or unreadable — as opposed to "opened fine, this tag is not in it". Only the
// former arms the backoff.
var errCacheUnavailable = errors.New("cache database is not readable right now")
// cachedRuleSet is the evidence that a remote rule-set can start with no network.
type cachedRuleSet struct {
// lastUpdated is when the engine last fetched this copy. Non-zero by
// construction (a zero value is what makes StartContext fetch), and reported to
// the operator so "serving from cache" carries an age rather than a shrug.
lastUpdated time.Time
}
// cachedRuleSetReader loads one saved rule-set out of a cache DB. A package var so
// the decision logic can be driven without a real bbolt file; the bbolt-backed
// implementation is exercised separately against a DB written by sing-box itself.
var cachedRuleSetReader = boltCachedRuleSetReader
// boltCachedRuleSetReader opens dbPath read-only and returns the saved entry for
// tag. Read-only means bbolt takes a SHARED flock, which also means the open
// succeeds only when no process holds the exclusive one — so a successful open is
// itself the guarantee that nothing is writing while we read.
func boltCachedRuleSetReader(dbPath, tag string) (saved *adapter.SavedBinary, err error) {
db, err := bbolt.Open(dbPath, 0o600, &bbolt.Options{ReadOnly: true, Timeout: coldStartOpenTimeout})
if err != nil {
return nil, errors.Join(errCacheUnavailable, err)
}
defer db.Close()
// A corrupt page makes bbolt panic rather than error (cachefile.view wraps its
// own reads the same way). A damaged cache must degrade to "not cached", not
// take the daemon down.
defer func() {
if r := recover(); r != nil {
saved, err = nil, errors.New("cache database is corrupt")
}
}()
err = db.View(func(tx *bbolt.Tx) error {
bucket := tx.Bucket([]byte(cachedRuleSetBucket))
if bucket == nil {
return os.ErrNotExist
}
raw := bucket.Get([]byte(tag))
if len(raw) == 0 {
return os.ErrNotExist
}
var out adapter.SavedBinary
if err := out.UnmarshalBinary(raw); err != nil {
return err
}
saved = &out
return nil
})
if err != nil {
return nil, err
}
return saved, nil
}
type coldStartProof struct {
set *cachedRuleSet // nil => looked, found nothing usable
when time.Time
}
var (
coldStartMu sync.Mutex
coldStartMemo = map[string]coldStartProof{}
coldStartBlocked time.Time // no open attempts until this instant
)
// cachedRuleSetUsable reports whether sing-box's cache already holds a copy of
// rule-set `tag` that its own StartContext would accept in `format`, i.e. whether
// handing the engine this REMOTE rule-set is safe with the source unreachable.
func cachedRuleSetUsable(tag, format string) (cachedRuleSet, bool) {
dbPath := engineCacheDBPath()
if dbPath == "" {
return cachedRuleSet{}, false
}
key := dbPath + "\x00" + tag
coldStartMu.Lock()
if p, ok := coldStartMemo[key]; ok {
if p.set != nil {
coldStartMu.Unlock()
if _, err := os.Stat(dbPath); err == nil {
return *p.set, true
}
// The DB it was proven from is gone. Nothing else in this package can
// have removed it silently, but a person with a shell can.
forgetCachedRuleSets(dbPath)
return cachedRuleSet{}, false
}
if time.Since(p.when) < coldStartMissTTL {
coldStartMu.Unlock()
return cachedRuleSet{}, false
}
}
if time.Now().Before(coldStartBlocked) {
coldStartMu.Unlock()
return cachedRuleSet{}, false
}
coldStartMu.Unlock()
saved, err := cachedRuleSetReader(dbPath, tag)
if err != nil {
if errors.Is(err, errCacheUnavailable) {
// Locked (the engine is up) or absent. Do not memoise per tag: the
// answer is about the DB, not about this list.
coldStartMu.Lock()
coldStartBlocked = time.Now().Add(coldStartLockedBackoff)
coldStartMu.Unlock()
return cachedRuleSet{}, false
}
return rememberColdStart(key, nil)
}
if !cachedCopyLoadable(saved, format) {
return rememberColdStart(key, nil)
}
return rememberColdStart(key, &cachedRuleSet{lastUpdated: saved.LastUpdated})
}
func rememberColdStart(key string, set *cachedRuleSet) (cachedRuleSet, bool) {
coldStartMu.Lock()
coldStartMemo[key] = coldStartProof{set: set, when: time.Now()}
coldStartMu.Unlock()
if set == nil {
return cachedRuleSet{}, false
}
return *set, true
}
// forgetCachedRuleSets drops every proof read out of dbPath. Called wherever this
// package discards a cache DB (cache.go), because a proof outlives the read that
// produced it and must not outlive the file.
func forgetCachedRuleSets(dbPath string) {
coldStartMu.Lock()
for k := range coldStartMemo {
if strings.HasPrefix(k, dbPath+"\x00") {
delete(coldStartMemo, k)
}
}
coldStartMu.Unlock()
}
// resetColdStartCache clears the memo and the backoff (tests).
func resetColdStartCache() {
coldStartMu.Lock()
coldStartMemo = map[string]coldStartProof{}
coldStartBlocked = time.Time{}
coldStartMu.Unlock()
}
// cachedCopyLoadable decides whether RemoteRuleSet.loadBytes would accept these
// bytes, using loadBytes' own readers on loadBytes' own branches.
//
// LastUpdated is checked first and is not a formality: it is the ONLY thing
// StartContext consults to decide whether to fetch, so a stored copy with a zero
// timestamp is worth nothing here however well it decodes.
//
// The format list is positive and CLOSED. An unrecognised format falls to "not
// usable", which omits the list — the recoverable side. loadBytes' own default
// branch returns "unknown rule-set format", and that error at start is fatal, so
// answering "usable" for a format neither of us knows would be the one mistake
// that costs the LAN.
func cachedCopyLoadable(saved *adapter.SavedBinary, format string) bool {
if saved == nil || saved.LastUpdated.IsZero() {
return false
}
var (
compat option.PlainRuleSetCompat
err error
)
switch format {
case C.RuleSetFormatBinary:
compat, err = srs.Read(bytes.NewReader(saved.Content), false)
case C.RuleSetFormatSource:
compat, err = json.UnmarshalExtended[option.PlainRuleSetCompat](saved.Content)
default:
return false
}
if err != nil {
return false
}
// Upgrade is the second half of loadBytes' decode and has its own failure mode
// (an unknown compat version), so skipping it would leave a fatal case unseen.
if _, err := compat.Upgrade(); err != nil {
return false
}
return true
}
+402
View File
@@ -0,0 +1,402 @@
package generate
import (
"bytes"
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/sagernet/sing-box/adapter"
"github.com/sagernet/sing-box/common/srs"
C "github.com/sagernet/sing-box/constant"
"github.com/sagernet/sing-box/experimental/cachefile"
"github.com/sagernet/sing-box/option"
"github.com/sagernet/sing/common/json/badoption"
"github.com/sagernet/sing/common/logger"
"github.com/sagernet/sing-box/shater/model"
)
// The scenario every test in this file is about, measured on the production
// router: the SIM operator passes a whitelist, raw.githubusercontent.com is not on
// it, and the R5 preflight therefore refuses every geosite/geoip list FOREVER —
// while the cache on /etc/shater holds a good copy of each one. The requirement is
// the owner's, verbatim: the lists must come up from disk after a reboot with no
// working WAN, and only update later.
const coldStartAppleURL = "https://raw.githubusercontent.com/SagerNet/sing-geosite/rule-set/geosite-apple.srs"
// coldStartCompiledSRS builds a real compiled rule-set — the same srs.Write path
// compileDomainList uses — so the tests decode genuine bytes rather than a fake
// that only this package would accept.
func coldStartCompiledSRS(t *testing.T, domains ...string) []byte {
t.Helper()
var buf bytes.Buffer
set := option.PlainRuleSet{Rules: []option.HeadlessRule{{
Type: C.RuleTypeDefault,
DefaultOptions: option.DefaultHeadlessRule{
DomainSuffix: badoption.Listable[string](domains),
},
}}}
if err := srs.Write(&buf, set, C.RuleSetVersionCurrent); err != nil {
t.Fatalf("compile a rule-set for the fixture: %v", err)
}
return buf.Bytes()
}
// coldStartHasRuleSet reports whether the generated config carries a rule-set with
// this tag. dnsfilter_test.go has the same helper, but that file is //go:build
// linux and these tests are not — the cache decision is plain file I/O, so they
// must also run on a developer's machine where the gate does not.
func coldStartHasRuleSet(opts option.Options, tag string) bool {
if opts.Route == nil {
return false
}
for _, rs := range opts.Route.RuleSet {
if rs.Tag == tag {
return true
}
}
return false
}
// withColdStartCache gives this test a private /etc/shater, clears the proof memo
// on both sides, and returns the DB path engineCacheDBPath will choose. The real
// overlay is never read or written from a test (see shater/testguard).
func withColdStartCache(t *testing.T) string {
t.Helper()
dir := filepath.Join(t.TempDir(), "shater")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create the private cache dir: %v", err)
}
persistent, _ := withCachePaths(t, dir)
resetColdStartCache()
t.Cleanup(resetColdStartCache)
if got := engineCacheDBPath(); got != persistent {
t.Fatalf("the fixture did not take: engineCacheDBPath()=%q, want %q "+
"(free space on %s may be below the %d-byte floor)", got, persistent, dir, cacheMinFreeSpace)
}
return persistent
}
// seedCacheDB writes one saved rule-set through sing-box's OWN cache writer.
//
// Using the real writer is the point, not convenience. cachedRuleSetBucket and the
// SavedBinary framing are COPIES of things cachefile keeps unexported, and a copied
// constant drifts silently: the reader would start answering "not cached" for
// everything and every test that hand-rolled the layout would keep passing. Written
// this way, a rename over there fails here.
func seedCacheDB(t *testing.T, dbPath, tag string, content []byte, lastUpdated time.Time) {
t.Helper()
cf := cachefile.New(context.Background(), logger.NOP(), option.CacheFileOptions{Enabled: true, Path: dbPath})
if err := cf.Start(adapter.StartStateInitialize); err != nil {
t.Fatalf("open the cache db for seeding: %v", err)
}
if err := cf.SaveRuleSet(tag, &adapter.SavedBinary{Content: content, LastUpdated: lastUpdated}); err != nil {
cf.Close()
t.Fatalf("save rule-set %q: %v", tag, err)
}
if err := cf.Close(); err != nil {
t.Fatalf("close the cache db (it must not stay locked): %v", err)
}
}
// TestColdStartAppliesCachedRuleSetWhenSourceIsDown is the headline case: cache
// present, network dead, list APPLIED.
//
// It also pins the two halves of that claim that could each be faked on their own —
// the set really is handed to the engine (ok, remote type, the URL intact), and the
// operator is told it came from the cache with an age, rather than being told
// nothing or being told it is fresh.
func TestColdStartAppliesCachedRuleSetWhenSourceIsDown(t *testing.T) {
db := withColdStartCache(t)
fetched := time.Now().Add(-36 * time.Hour).Truncate(time.Second)
seedCacheDB(t, db, "bl-apple-apple", coldStartCompiledSRS(t, "apple.com", "icloud.com"), fetched)
withRuleSetProbe(t, func(string) bool { return false }) // the SIM blocks it
b := newBuilder(&model.Model{})
set, ok := b.remoteRuleSet("bl-apple-apple", coldStartAppleURL, "24h", `blocklist "apple"`)
if !ok {
t.Fatalf("a list with a loadable copy in the cache must still be applied when its source is down; warnings: %v", b.warnings)
}
if set.Type != C.RuleSetTypeRemote || set.RemoteOptions.URL != coldStartAppleURL {
t.Fatalf("the engine must still own fetch/update/status for this list, got %+v", set)
}
if set.Format != C.RuleSetFormatBinary {
t.Fatalf("format %q: the cached copy was decoded as binary, so the engine must be told binary", set.Format)
}
if !warnsHaveSub(b.warnings, "RULESET-FROM-CACHE") {
t.Fatalf("applying a stale cached copy silently is a lie about freshness; warnings: %v", b.warnings)
}
if !warnsHaveSub(b.warnings, fetched.UTC().Format(time.RFC3339)) {
t.Fatalf("the warning must carry the age of the copy being served; warnings: %v", b.warnings)
}
if warnsHaveSub(b.warnings, "RULESET-NOT-APPLIED") {
t.Fatalf("the list WAS applied, so the not-applied warning is a false alarm; warnings: %v", b.warnings)
}
}
// TestColdStartOmitsRuleSetWithNoCache is the pair, and the invariant that must
// survive the fix: with no cached copy the list is still OMITTED — never handed to
// the engine on a hope — with the original, greppable warning. An omitted blocklist
// blocks nothing; a list handed over unfetchable and uncached aborts box.Start and
// takes the whole LAN down.
func TestColdStartOmitsRuleSetWithNoCache(t *testing.T) {
db := withColdStartCache(t)
if _, err := os.Stat(db); err == nil {
t.Fatalf("fixture: %s must not exist for the empty-cache case", db)
}
withRuleSetProbe(t, func(string) bool { return false })
b := newBuilder(&model.Model{})
set, ok := b.remoteRuleSet("bl-apple-apple", coldStartAppleURL, "24h", `blocklist "apple"`)
if ok {
t.Fatalf("with no cached copy the list must be omitted, got %+v", set)
}
if !warnsHaveSub(b.warnings, "RULESET-NOT-APPLIED") {
t.Fatalf("the omission must keep its greppable prefix; warnings: %v", b.warnings)
}
for _, want := range []string{"matches NOTHING", "no usable copy in the cache"} {
if !warnsHaveSub(b.warnings, want) {
t.Fatalf("warning must still say %q; warnings: %v", want, b.warnings)
}
}
if warnsHaveSub(b.warnings, "RULESET-FROM-CACHE") {
t.Fatalf("nothing was served from the cache; warnings: %v", b.warnings)
}
}
// TestColdStartRefusesUnusableCachedCopies covers the three ways a cache entry is
// present but worthless. Each one, if accepted, ends the same way: StartContext
// fails to load it, lastUpdated stays zero, it fetches, the fetch fails, and
// box.Start returns an error with the LAN fail-closed behind it. So each must
// degrade to the omission above, not to an optimistic yes.
func TestColdStartRefusesUnusableCachedCopies(t *testing.T) {
good := coldStartCompiledSRS(t, "apple.com")
cases := []struct {
name string
content []byte
lastUpdated time.Time
why string
}{
{
name: "corrupt content", content: []byte("not a rule-set at all"), lastUpdated: time.Now(),
why: "srs.Read refuses it, so loadBytes refuses it and the engine refetches — fatally",
},
{
name: "truncated content", content: good[:len(good)/2], lastUpdated: time.Now(),
why: "a half-written .srs decodes its header and then fails in the zlib stream",
},
{
name: "never updated", content: good, lastUpdated: time.Time{},
why: "StartContext consults ONLY lastUpdated to decide whether to fetch; zero means it fetches",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
db := withColdStartCache(t)
seedCacheDB(t, db, "bl-apple-apple", tc.content, tc.lastUpdated)
withRuleSetProbe(t, func(string) bool { return false })
b := newBuilder(&model.Model{})
if _, ok := b.remoteRuleSet("bl-apple-apple", coldStartAppleURL, "24h", `blocklist "apple"`); ok {
t.Fatalf("this cached copy must NOT be accepted: %s", tc.why)
}
if !warnsHaveSub(b.warnings, "RULESET-NOT-APPLIED") {
t.Fatalf("expected the omission warning, got %v", b.warnings)
}
})
}
}
// TestColdStartDNSFilterComesUpOffline is the router's symptom, end to end at the
// generate level: dns_filter on, one geosite blocklist, source unreachable. Before
// the fix this produced "RULESET-NOT-APPLIED ... matches NOTHING" followed by
// "DNS-FILTER-NOT-APPLIED ... NOT ONE of them could be built", i.e. no filtering at
// all on a router whose cache held the list. The control is in the same test: with
// the cache emptied, both warnings must come back — otherwise this only proves that
// the filter is built, not that the CACHE is what built it.
func TestColdStartDNSFilterComesUpOffline(t *testing.T) {
appleModel := func() *model.Model {
g := model.DefaultGlobals()
g.DNSFilter = true
g.ResolverDefault = "cf"
return &model.Model{
Globals: g,
Inbounds: []model.Inbound{
{Name: "lan", Enabled: true, Type: "tproxy", TproxyPort: 12380, TCP: true, UDP: true},
},
Resolvers: []model.Resolver{
{Name: "cf", Type: "doh", Address: "https://1.1.1.1/dns-query", Detour: "direct"},
},
Blocklists: []model.Blocklist{
{Name: "apple", Enabled: true, Source: "geosite", Categories: []string{"apple"}, Response: "nxdomain"},
},
}
}
t.Run("cache warm", func(t *testing.T) {
db := withColdStartCache(t)
seedCacheDB(t, db, "bl-apple-apple", coldStartCompiledSRS(t, "apple.com"), time.Now().Add(-time.Hour))
withRuleSetProbe(t, func(string) bool { return false })
opts, warns, err := GenerateWithWarnings(appleModel())
if err != nil {
t.Fatalf("generate: %v", err)
}
if !coldStartHasRuleSet(opts, "bl-apple-apple") {
t.Fatalf("the blocklist must be in the config, built from the cache; warnings: %v", warns)
}
if warnsHaveSub(warns, "DNS-FILTER-NOT-APPLIED") {
t.Fatalf("the DNS filter WAS built, so this warning is false; warnings: %v", warns)
}
})
t.Run("control: cache cold", func(t *testing.T) {
withColdStartCache(t) // the DB is not created at all
withRuleSetProbe(t, func(string) bool { return false })
opts, warns, err := GenerateWithWarnings(appleModel())
if err != nil {
t.Fatalf("generate: %v", err)
}
if coldStartHasRuleSet(opts, "bl-apple-apple") {
t.Fatalf("with nothing in the cache and no network the list must NOT be emitted")
}
if !warnsHaveSub(warns, "DNS-FILTER-NOT-APPLIED") {
t.Fatalf("control: the offline-and-uncached router must still say the filter is not applied; warnings: %v", warns)
}
})
}
// TestColdStartReaderMatchesTheEngineWriter is the drift pin for the two things
// this package copies out of experimental/cachefile: the bucket name and the
// SavedBinary framing. It is deliberately a direct reader test rather than an
// assertion about behaviour, so a rename over there reports itself here instead of
// silently turning every cold start back into an omission.
func TestColdStartReaderMatchesTheEngineWriter(t *testing.T) {
db := withColdStartCache(t)
content := coldStartCompiledSRS(t, "apple.com")
when := time.Now().Add(-2 * time.Hour).Truncate(time.Second)
seedCacheDB(t, db, "bl-apple-apple", content, when)
saved, err := boltCachedRuleSetReader(db, "bl-apple-apple")
if err != nil {
t.Fatalf("reading back what sing-box's own writer wrote failed: %v — "+
"cachedRuleSetBucket (%q) or adapter.SavedBinary has moved", err, cachedRuleSetBucket)
}
if !bytes.Equal(saved.Content, content) {
t.Fatalf("content round-trip differs: %d bytes back, %d written", len(saved.Content), len(content))
}
if !saved.LastUpdated.Equal(when) {
t.Fatalf("LastUpdated round-trip: got %s, want %s", saved.LastUpdated, when)
}
if _, err := boltCachedRuleSetReader(db, "bl-nobody-asked-for-this"); err == nil {
t.Fatal("a tag that was never cached must not report a hit")
}
// A DB that is not there at all is "unavailable", not "this tag is missing" —
// the difference arms the backoff instead of memoising a per-tag miss.
missing := filepath.Join(t.TempDir(), "no-such-cache.db")
if _, err := boltCachedRuleSetReader(missing, "bl-apple-apple"); err == nil {
t.Fatal("opening a non-existent DB must fail")
} else if !strings.Contains(err.Error(), "not readable right now") {
t.Fatalf("a missing DB must be reported as unavailable, got %v", err)
}
}
// TestEngineCacheDBPathTwinsCacheFilePath holds the read-side twin to the standard
// this repo learned the hard way: a presence check must cover everything its acting
// twin does, or the fast path becomes a trap. engineCacheDBPath decides which DB to
// READ, cacheFilePath decides which DB the engine WRITES, and they must not be able
// to disagree about which file that is.
func TestEngineCacheDBPathTwinsCacheFilePath(t *testing.T) {
t.Run("dir present: both choose the persistent DB", func(t *testing.T) {
dir := filepath.Join(t.TempDir(), "shater")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("mkdir: %v", err)
}
persistent, _ := withCachePaths(t, dir)
if got := newBuilder(&model.Model{Globals: model.DefaultGlobals()}).cacheFilePath(); got != persistent {
t.Skipf("this machine's free space puts the writer on tmpfs (%s); the twin has nothing to compare", got)
}
if got := engineCacheDBPath(); got != persistent {
t.Fatalf("reader chose %q, writer chose %q — the cold-start check would consult a file the engine never reads", got, persistent)
}
})
t.Run("dir missing: both choose tmpfs", func(t *testing.T) {
dir := filepath.Join(t.TempDir(), "absent")
_, fallback := withCachePaths(t, dir)
if got := newBuilder(&model.Model{Globals: model.DefaultGlobals()}).cacheFilePath(); got != fallback {
t.Fatalf("writer: want %q, got %q", fallback, got)
}
if got := engineCacheDBPath(); got != fallback {
t.Fatalf("reader: want %q, got %q", fallback, got)
}
})
t.Run("oversized DB: the reader reports no cache", func(t *testing.T) {
dir := filepath.Join(t.TempDir(), "shater")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("mkdir: %v", err)
}
persistent, _ := withCachePaths(t, dir)
if err := os.WriteFile(persistent, make([]byte, cacheMaxBytes+1), 0o600); err != nil {
t.Fatalf("seed an oversized db: %v", err)
}
// cacheFilePath is about to DELETE this DB, so anything read out of it is
// about to stop existing. "" is the honest answer; the persistent path is not.
if got := engineCacheDBPath(); got != "" {
t.Fatalf("reader returned %q for a DB the writer is about to discard", got)
}
})
}
// TestColdStartProofSurvivesTheEngineTakingTheLock is the reason a proof is
// remembered at all. The generate that matters runs before any box exists; every
// reconcile after it finds the DB flocked by the engine and cannot read a thing. If
// the proof did not outlive that first read, the list would be applied once and
// dropped again a minute later — which is the original defect wearing a hat.
func TestColdStartProofSurvivesTheEngineTakingTheLock(t *testing.T) {
db := withColdStartCache(t)
seedCacheDB(t, db, "bl-apple-apple", coldStartCompiledSRS(t, "apple.com"), time.Now().Add(-time.Hour))
withRuleSetProbe(t, func(string) bool { return false })
b := newBuilder(&model.Model{})
if _, ok := b.remoteRuleSet("bl-apple-apple", coldStartAppleURL, "24h", `blocklist "apple"`); !ok {
t.Fatalf("cold start must apply the cached list; warnings: %v", b.warnings)
}
// Now the engine holds it: every further read fails, exactly as on a running
// router. The stub stands in for the flock, whose behaviour is the OS's and not
// something a unit test can arrange portably.
prev := cachedRuleSetReader
cachedRuleSetReader = func(string, string) (*adapter.SavedBinary, error) {
return nil, errCacheUnavailable
}
t.Cleanup(func() { cachedRuleSetReader = prev })
b2 := newBuilder(&model.Model{})
if _, ok := b2.remoteRuleSet("bl-apple-apple", coldStartAppleURL, "24h", `blocklist "apple"`); !ok {
t.Fatalf("the proof must survive the engine locking the DB; warnings: %v", b2.warnings)
}
// ...but it must NOT survive the DB being removed. A proof that outlives the
// file it was read from is the fail-open half of this feature, and cache.go
// removes that file on its own in two branches.
if err := os.Remove(db); err != nil {
t.Fatalf("remove the db: %v", err)
}
b3 := newBuilder(&model.Model{})
if _, ok := b3.remoteRuleSet("bl-apple-apple", coldStartAppleURL, "24h", `blocklist "apple"`); ok {
t.Fatal("with the cache DB deleted there is nothing to start from, so the list must be omitted again")
}
}
+168 -13
View File
@@ -75,7 +75,20 @@ func (b *builder) buildDNS() *option.DNSOptions {
// skipped resolver here makes the engine reject the config outright — the
// fail-closed outcome every other path in this package avoids. servers is in
// model order, so servers[0] is the first resolver that survived.
final := b.m.Globals.ResolverDefault
//
// The name itself comes from resolverNameInForce, not straight out of globals:
// the active WAN profile may override `resolver_default`, and a profile that
// names a resolver this configuration cannot use rolls back to the globals value
// (loudly) rather than taking DNS down on the one uplink where nobody is looking.
final, _ := b.resolverNameInForce(profileResolverField{
field: "resolver_default",
globalsValue: b.m.Globals.ResolverDefault,
unusableWhy: "no such resolver, or it was skipped for a bad address or type",
// No "…, and the next warning names it": when BOTH sides are empty the caller
// stays silent (it only warns for a non-empty `final`), so pointing at a
// warning that may not exist would be a promise about output nobody checked.
noGlobalsConsequence: "the first resolver that did build becomes the default instead",
}, b.effectiveResolverDefault(), serverTags)
if !serverTags[final] {
if strings.TrimSpace(final) != "" {
b.warnf("resolver_default %q was not built (unknown, or skipped for a bad address/type); falling back to resolver %q as the default.", final, servers[0].Tag)
@@ -218,6 +231,130 @@ func (b *builder) buildDNS() *option.DNSOptions {
}
}
// profileOverrideNotAppliedTag is the operator's grep handle in `logread` for a
// profile override that named something this configuration does not have and is
// therefore NOT the value in force. Same shape and same purpose as the
// RULESET-NOT-APPLIED / DNS-FILTER-NOT-APPLIED / DEVICE-FILTER-NOT-APPLIED tags
// already emitted by ruleset.go, dnsfilter.go and devices.go.
//
// It is also how apply/warnings.go can grade the finding structurally: that file
// returns SeverityCritical outright for a tag in its notAppliedTags list. This tag
// is NOT in that list yet — apply is another tree — so today the grade comes from
// the criticalMarkers entry "inert", which the messages below say because it is
// literally what happened to the override, not to satisfy a classifier.
const profileOverrideNotAppliedTag = "PROFILE-OVERRIDE-NOT-APPLIED"
// profileResolverField describes ONE of the resolver-name scalars a WAN profile
// may override, for resolverNameInForce. Every field here is something the two
// callers say differently, which is why none of them is baked into the helper:
//
// unusableWhy what "unusable" can mean at this call site. buildDNS asks
// about BUILT servers (a resolver can exist and still have
// been skipped for a bad address); endpointResolver runs
// before anything is built and asks only about existence.
// noGlobalsConsequence what happens when globals has nothing usable either. The
// callers degrade differently — the default falls to the
// first resolver that built, the fallback disappears
// outright, the endpoint resolver is left unset — and one
// shared sentence could only be vague about the single thing
// the operator needs to know.
type profileResolverField struct {
field string
globalsValue string
unusableWhy string
noGlobalsConsequence string
}
// resolverNameInForce turns "what model says is in force" into "what this
// generator can actually use", applying the rollback rule for a profile override
// that names a resolver this configuration does not have.
//
// # The rule
//
// The profile override is DROPPED and the globals value takes effect. Not the
// other way round, and not a refusal to apply: a profile applies to exactly one
// uplink, so a typo in the SIM profile's `resolver_default` would otherwise take
// name resolution down on the SIM uplink — which is the uplink you are on when the
// ethernet is out, i.e. the one where you can least afford it and can least easily
// look. Refusing the whole config over it would be worse still: with the kill
// switch closed, "refuse to start" means the whole LAN is offline over a name that
// is wrong on ONE uplink.
//
// # Why it is never silent
//
// A profile override is invisible by construction — it only applies while that
// profile is active — so nothing about the running router reveals the typo until
// the uplink changes, at which point the config that broke DNS looks exactly like
// the config that worked. The warning therefore names all four things the operator
// needs: the profile, the field, the value that was rejected, and what is in force
// instead. It is tagged (see profileOverrideNotAppliedTag) so it is greppable and
// so apply can grade it critical.
//
// # Return
//
// name is what the caller must use. fromProfile reports whether that name is STILL
// the profile's, so a caller that attributes the value in its own diagnostics
// (endpointResolver's `src`) keeps saying "active profile" only while it is true.
//
// known is the set of names this call site can use — built DNS server tags for
// buildDNS, declared `config resolver` names for endpointResolver. Membership is
// tested with the caller's own spelling of the name, so the verdict here and the
// lookup that follows it cannot disagree.
func (b *builder) resolverNameInForce(f profileResolverField, ov model.Override, known map[string]bool) (name string, fromProfile bool) {
v := strings.TrimSpace(ov.Value)
// Two states, both named. Positive and closed: model.Override.From is either
// FromGlobals or "profile:<name>", and there is no third thing to sweep into a
// default. When the value came from globals there is nothing to roll back TO —
// globals IS the fallback — and the caller reports an unusable one itself.
prof := b.activeProfile
if prof == nil || ov.From == model.FromGlobals {
return v, false
}
if v == "" || known[v] {
return v, true // the profile set nothing, or set something usable
}
g := strings.TrimSpace(f.globalsValue)
switch {
case known[g]:
b.warnf("%s: profile %q: %s %q is not usable here (%s), so the override is inert and the globals value %q "+
"is what resolves while this profile is active. The configuration was still applied: a profile covers "+
"ONE uplink, and refusing to start over a name that is wrong on one uplink would take the router down "+
"on every other one — with the kill switch closed that is the whole LAN. Fix the name in profile %q, "+
"or add the `config resolver` it means.",
profileOverrideNotAppliedTag, prof.Name, f.field, v, f.unusableWhy, g, prof.Name)
case g == "":
b.warnf("%s: profile %q: %s %q is not usable here (%s), so the override is inert — and globals sets no %s "+
"either, so %s. The configuration was still applied: a profile covers ONE uplink, and refusing to "+
"start over a name that is wrong on one uplink would take the router down on every other one — with "+
"the kill switch closed that is the whole LAN. Fix the name in profile %q, or add the `config "+
"resolver` it means.",
profileOverrideNotAppliedTag, prof.Name, f.field, v, f.unusableWhy, f.field, f.noGlobalsConsequence, prof.Name)
default:
b.warnf("%s: profile %q: %s %q is not usable here (%s), so the override is inert — and the globals value "+
"%q it falls back to is not usable either, so %s. The configuration was still applied: a profile "+
"covers ONE uplink, and refusing to start over a name that is wrong on one uplink would take the "+
"router down on every other one — with the kill switch closed that is the whole LAN. Fix the name in "+
"profile %q, or add the `config resolver` it means.",
profileOverrideNotAppliedTag, prof.Name, f.field, v, f.unusableWhy, g, f.noGlobalsConsequence, prof.Name)
}
return g, false
}
// declaredResolverNames is the set of `config resolver` names the model declares,
// keyed by the SAME spelling endpointResolver's own lookup compares against (the
// raw Name, untrimmed) — so "this resolver exists" means one thing in both places.
// It is deliberately NOT the set of BUILT servers: it is consulted during
// buildRoute, before any DNS server has been built.
func (b *builder) declaredResolverNames() map[string]bool {
names := make(map[string]bool, len(b.m.Resolvers))
for i := range b.m.Resolvers {
names[b.m.Resolvers[i].Name] = true
}
return names
}
// endpointResolverTagPrefix names the synthetic bootstrap-direct DNS server the
// endpoint resolver is cloned into, so its tag never collides with the user's own
// `config resolver` server tag it was cloned from.
@@ -232,7 +369,11 @@ const endpointResolverTagPrefix = "shater-endpoint-dns-"
// # Tag source priority
//
// The active WAN profile's EndpointResolver override wins over Globals.EndpointResolver.
// Both name an existing `config resolver` by name.
// Both name an existing `config resolver` by name. That priority is NOT decided here:
// it comes from model.EffectiveEndpointResolver, the one place the rule lives, shared
// with apply and with the panel that has to print what is in force. A profile override
// naming a resolver this configuration does not have rolls back to the globals value
// with a loud, tagged warning — see resolverNameInForce.
//
// # Bootstrap-direct
//
@@ -257,16 +398,23 @@ func (b *builder) endpointResolver() string {
}
b.endpointResolverComputed = true
// Profile overrides live on the builder only after applyProfiles; it is
// idempotent, so calling it here makes endpointResolver safe to invoke from either
// buildRoute or buildDNS regardless of order.
b.applyProfiles()
name := b.profileEndpointResolver
src := "active profile"
if name == "" {
name = strings.TrimSpace(b.m.Globals.EndpointResolver)
src = "globals.endpoint_resolver"
// effectiveEndpointResolver runs applyProfiles for us, and that is idempotent, so
// endpointResolver is safe to invoke from either buildRoute or buildDNS regardless
// of order.
//
// The `known` set here is the DECLARED resolvers, not the built DNS servers: this
// runs during buildRoute, before a single server exists. That is also the exact
// predicate the res lookup below uses, so the rollback and the lookup can never
// disagree about what "there is such a resolver" means.
name, fromProfile := b.resolverNameInForce(profileResolverField{
field: "endpoint_resolver",
globalsValue: b.m.Globals.EndpointResolver,
unusableWhy: "no such resolver",
noGlobalsConsequence: "proxy server domains are resolved by the engine's built-in resolver and route.default_domain_resolver is left unset",
}, b.effectiveEndpointResolver(), b.declaredResolverNames())
src := "globals.endpoint_resolver"
if fromProfile {
src = "active profile"
}
if name == "" {
return "" // feature off — no default_domain_resolver, engine uses its built-in
@@ -512,7 +660,14 @@ func (b *builder) warnBootstrapInTheClear(res model.Resolver) {
// Returns nil (and the config is byte-identical to one without the feature) when
// no fallback is configured, when it is unusable, or when it would be a no-op.
func (b *builder) resolverFallbackRules(final string, serverTags map[string]bool) []option.DNSRule {
fb := strings.TrimSpace(b.m.Globals.ResolverFallback)
// Same door as resolver_default: the active profile may override the fallback on
// its own, and an unusable override rolls back to globals with a loud warning.
fb, _ := b.resolverNameInForce(profileResolverField{
field: "resolver_fallback",
globalsValue: b.m.Globals.ResolverFallback,
unusableWhy: "no such resolver, or it was skipped for a bad address or type",
noGlobalsConsequence: "there is no DNS failover at all — when the default resolver stops answering, queries simply fail",
}, b.effectiveResolverFallback(), serverTags)
if fb == "" {
return nil
}
+120 -6
View File
@@ -31,6 +31,8 @@ import (
C "github.com/sagernet/sing-box/constant"
"github.com/sagernet/sing-box/option"
"github.com/sagernet/sing/common/json/badoption"
"github.com/sagernet/sing-box/shater/model"
)
// filterAnswerTTL is the TTL stamped on a "zero"-response predefined answer
@@ -47,12 +49,124 @@ const wildcardOwner = "*."
// (url-source) filter rule-set when the model does not specify one.
const defaultRuleSetUpdateInterval = "24h"
// filterFetchDetour is the outbound a remote filter rule-set fetches through.
// "direct" (always present as a baseline outbound) is chosen so a list fetch
// cannot blackhole when the proxy is down or the kill-switch Final is "block";
// it also avoids a chicken-and-egg where the list needed to route the proxy is
// itself fetched via the proxy. (A future task may make this configurable.)
const filterFetchDetour = tagDirect
// fetchDetourNotAppliedTag is the SCREAMING-KEBAB prefix on the one warning this
// file emits about the fetch detour. It exists for the same reason
// RULESET-NOT-APPLIED does: apply/warnings.go grades severity off the TAG rather
// than off prose (see its notAppliedTags), and this condition has to be critical —
// the operator configured every list/geo download to travel through a tunnel and
// it is travelling over the plain WAN instead. Rename it here and
// apply/fetchdetour_severity_test.go fails by name.
const fetchDetourNotAppliedTag = "FETCH-DETOUR-NOT-APPLIED"
// filterFetchDetour is the outbound a remote filter/routing rule-set is fetched
// through — the engine's `http_client{detour}` on every option.RemoteRuleSet this
// package emits (ruleset.go remoteRuleSetAs).
//
// It used to be the constant `tagDirect`, on the reasoning that a list fetch must
// not blackhole when the proxy is down and that a list needed to route the proxy
// must not be fetched through the proxy. Both halves of that are still true and
// neither is what the router actually runs into: on the SIM uplink this ships to,
// the operator passes a whitelist, so `direct` is not the safe path — it is the
// path where raw.githubusercontent.com is refused FOREVER and the `apple`
// blocklist never loads at all. The value is therefore configurable
// (globals.fetch_detour, overridable per WAN profile), and the old constant is the
// behaviour you get by leaving it unset.
//
// # What is resolved here, and what is NOT
//
// model.EffectiveFetchDetour answers "which value is in force" (profile over
// globals, per field). This function answers the half model cannot: whether the
// named node/group/egress/chain is something THIS generate pass actually emits.
// The list of accepted shapes is closed and positive; anything that does not
// resolve falls back to `direct` and says so at critical, because a silent
// substitution here is indistinguishable from the setting working.
//
// # The limitation this does NOT remove, stated rather than implied
//
// The R5 preflight (ruleset.go remoteRuleSetUsable) probes the source with a plain
// DIRECT http.Client, because generate has no way to dial through the running box.
// With a non-direct detour that probe no longer measures the path the engine will
// take, in both directions:
//
// - probe reachable + detour broken at start => RemoteRuleSet.StartContext still
// fails fatally, unless the cold-start cache already holds the list
// (coldstart_cache.go, which is what covers the steady state);
// - probe unreachable + detour would have worked + nothing cached => the list is
// still omitted, so a list that has NEVER been fetched cannot bootstrap over
// the detour alone.
//
// Making the preflight detour-aware means dialling through the engine from inside
// generate, which is engine/apply territory; it is deliberately not attempted
// here, and it is not claimed to have been.
func (b *builder) filterFetchDetour() string {
// applyProfiles resolves (and reports) the active profile once, and buildRoute
// calls it before anything materialises a rule-set. Asking again is cheap; the
// duplicate warnings it re-emits are not, so they are dropped.
var prof *model.Profile
b.withoutNewWarnings(func() { prof = b.resolveActiveProfile() })
ov := model.EffectiveFetchDetour(b.m.Globals, prof)
want := strings.TrimSpace(ov.Value)
if want == "" {
// Not set anywhere. `direct` is the documented default, not a fallback, so
// there is nothing to report.
return tagDirect
}
if strings.EqualFold(want, tagBlock) {
b.warnFetchDetourNotApplied(ov, want,
"`block` is not a route: it is the outbound that discards whatever it is handed, so every "+
"list, geo and rule-set download aimed through it would fail — and a remote rule-set whose "+
"first fetch fails aborts engine start, which with a closed kill switch takes the LAN down. "+
"It is refused here rather than obeyed")
return tagDirect
}
tag, ok := b.resolveTarget(want)
if !ok {
b.warnFetchDetourNotApplied(ov, want,
"nothing this configuration builds answers to that name — no enabled node, no group with "+
"usable members (an empty group is not emitted at all, so it cannot be named), no interface "+
"egress and no chain that assembles")
return tagDirect
}
return tag
}
// warnFetchDetourNotApplied states, once per generate pass, that the configured
// fetch detour is not the one in force and what is running instead.
//
// The entity clause (`profile "x": ` / `globals "fetch_detour": `) is not
// decoration: apply/warnings.go recovers Section and Name from exactly that shape
// (taggedEntityRe), which is what lets the panel badge the profile that carries
// the bad value instead of printing a paragraph under "generate".
func (b *builder) warnFetchDetourNotApplied(ov model.Override, want, why string) {
entity := `globals "fetch_detour"`
if name, ok := strings.CutPrefix(ov.From, "profile:"); ok {
entity = fmt.Sprintf("profile %q", name)
}
b.warnOnce("%s: %s: fetch_detour %q is NOT in force — %s. Every blocklist, allowlist, ruleset, "+
"geoip and geosite download is going out %q instead: over the plain WAN, with this router's real "+
"address, which is the disclosure fetch_detour is set to prevent — and on an uplink that refuses "+
"those sources they will not load at all. Correct the name, or remove the option if %q is what "+
"you meant.",
fetchDetourNotAppliedTag, entity, want, why, tagDirect, tagDirect)
}
// warnOnce is warnf that never repeats an identical sentence.
//
// filterFetchDetour is consulted once per remote rule-set — a dozen times on a
// real config — and one mistyped detour must produce one critical, not a dozen. It
// deliberately deduplicates on the FINISHED text rather than on a flag, because
// there is nowhere to keep a flag: builder's fields are owned elsewhere this pass,
// and a per-call memo would have to live there.
func (b *builder) warnOnce(format string, args ...any) {
msg := fmt.Sprintf(format, args...)
for _, w := range b.warnings {
if w == msg {
return
}
}
b.warnings = append(b.warnings, msg)
}
// dnsFilterActive reports whether the D15 filter is switched on AND at least one
// blocklist/allowlist is enabled — used to decide whether to warn when the DNS
+9
View File
@@ -532,6 +532,15 @@ func TestDNSFilterHostileInputStillValidates(t *testing.T) {
// missing. Verified off-VM too: with the remote set forced into the config the same
// scenario fails with "initialize rule-set[0]: initial rule-set: rs-...".
func TestOfflineBootStartsEngine(t *testing.T) {
// A COLD cache is this test's premise — "power came back" means the router has
// not fetched anything yet. It has to be arranged explicitly now that generate
// consults sing-box's cache before omitting an unreachable list
// (coldstart_cache.go): TestMain gives the whole binary ONE private cache DB, and
// an earlier test in this file starts a real engine that caches a rule-set under
// this very tag (bl-remote-ads), which the cold-start check would then serve from
// — correctly, but that is the OTHER scenario. Without this line the assertions
// below depend on which tests ran first.
withColdStartCache(t)
withRuleSetProbe(t, unreachableProbe)
g := model.DefaultGlobals()
+329
View File
@@ -0,0 +1,329 @@
package generate
// The configurable fetch detour (globals.fetch_detour, overridable per WAN
// profile) as the engine actually receives it: the `http_client{detour}` of every
// remote rule-set this package emits.
//
// WHY THESE TESTS ARE SHAPED AROUND CONTROLS. The value under test is a single
// string on a nested option struct, and "it says direct" is the answer for BOTH
// "the operator configured direct" and "the whole feature is dead". Every case
// below therefore comes with its opposite: a configured detour that must ARRIVE,
// beside an unusable one that must roll back AND be named. A test that only
// checked the rollback would pass over a filterFetchDetour that returned tagDirect
// unconditionally — which is exactly the constant this change removes.
import (
"os"
"strings"
"testing"
"time"
"github.com/sagernet/sing-box/adapter"
C "github.com/sagernet/sing-box/constant"
"github.com/sagernet/sing-box/option"
"github.com/sagernet/sing-box/shater/model"
)
// fetchDetourSRSURL ends in .srs on purpose: ruleSetURLIsEngineNative decides
// remote-vs-locally-compiled by URL EXTENSION alone, and only the REMOTE path
// carries an http_client{detour} for the engine to dial through. Trim the suffix
// and the list becomes a text list generate downloads itself, which is a different
// fetch path with a different (and, today, un-detourable) client.
const fetchDetourSRSURL = "https://lists.example/ads.srs"
// listFetchDetourModel is one enabled node, one group over it, and one remote
// blocklist — the smallest configuration in which a detour has something real to
// name and something real to be stamped on.
//
// The group is `manual` over the node rather than subscription-backed so the
// fixture needs no sub cache; buildGroups emits the tag either way, which is all
// resolveTarget consults.
func listFetchDetourModel(globalsDetour string, profiles ...model.Profile) *model.Model {
g := model.DefaultGlobals()
g.DNSFilter = true
g.ResolverDefault = "cf"
g.FetchDetour = globalsDetour
if len(profiles) > 0 {
// Pin the profile explicitly. Auto-select skips iface-conditioned profiles
// (shaterd's WAN watcher owns that), so a fixture that relied on it would be
// testing the selector rather than the override.
g.ActiveProfile = profiles[0].Name
}
return &model.Model{
Globals: g,
Profiles: profiles,
Resolvers: []model.Resolver{{Name: "cf", Type: "udp", Address: "1.1.1.1", Detour: "direct"}},
Nodes: []model.Node{{
Name: "tokyo", Enabled: true,
URI: "vless://11111111-1111-1111-1111-111111111111@example.com:443?security=tls&sni=example.com#tokyo",
}},
Groups: []model.Group{{Name: "auto", Source: "manual", Strategy: "single", Nodes: []string{"tokyo"}}},
Blocklists: []model.Blocklist{
{Name: "ads", Enabled: true, Source: "url", URL: fetchDetourSRSURL, UpdateInterval: "24h"},
},
}
}
// remoteDetourOf returns the detour stamped on the emitted remote rule-set, or
// fails: an absent rule-set would make every assertion below vacuously true.
func remoteDetourOf(t *testing.T, opts option.Options, tag string) string {
t.Helper()
if opts.Route == nil {
t.Fatalf("no route emitted at all")
}
for _, rs := range opts.Route.RuleSet {
if rs.Tag != tag {
continue
}
if rs.Type != C.RuleSetTypeRemote {
t.Fatalf("rule-set %q type = %q, want remote — only the remote path carries a fetch detour", tag, rs.Type)
}
hc := rs.RemoteOptions.HTTPClient
if hc == nil {
t.Fatalf("rule-set %q has no http_client at all, so it dials with the engine's default", tag)
}
return hc.Detour
}
t.Fatalf("rule-set %q was not emitted; rule-sets=%+v", tag, opts.Route.RuleSet)
return ""
}
// generateFetchDetour runs the real generator with the probe stubbed reachable —
// the tests here are about which OUTBOUND the download is aimed at, not about
// whether the source answers.
func generateFetchDetour(t *testing.T, m *model.Model) (option.Options, []string) {
t.Helper()
withRuleSetProbe(t, func(string) bool { return true })
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
return opts, warns
}
// fetchDetourWarnings returns just the FETCH-DETOUR-NOT-APPLIED lines.
func fetchDetourWarnings(warns []string) []string {
var out []string
for _, w := range warns {
if strings.HasPrefix(w, fetchDetourNotAppliedTag+":") {
out = append(out, w)
}
}
return out
}
// TestFetchDetourUnsetIsDirect is the BASELINE CONTROL, and it is the one that
// says the change is backwards compatible: with nothing configured the emitted
// detour is byte-identical to the constant this replaced.
func TestFetchDetourUnsetIsDirect(t *testing.T) {
opts, warns := generateFetchDetour(t, listFetchDetourModel(""))
if got := remoteDetourOf(t, opts, "bl-ads"); got != tagDirect {
t.Fatalf("detour = %q, want %q — an unconfigured fetch_detour must behave exactly as the old constant did", got, tagDirect)
}
if got := fetchDetourWarnings(warns); len(got) != 0 {
t.Fatalf("not configuring an OPTIONAL override is not a fault; warnings: %v", got)
}
}
// TestFetchDetourFromGlobalsReachesTheEngine is the positive case the whole change
// exists for: on the SIM uplink the operator points every list download at a group
// of nodes the operator's whitelist does pass, and the engine has to be TOLD.
func TestFetchDetourFromGlobalsReachesTheEngine(t *testing.T) {
opts, warns := generateFetchDetour(t, listFetchDetourModel("group:auto"))
if got := remoteDetourOf(t, opts, "bl-ads"); got != "auto" {
t.Fatalf("detour = %q, want %q — the configured group must be the outbound the engine downloads through; warnings=%v",
got, "auto", warns)
}
if got := fetchDetourWarnings(warns); len(got) != 0 {
t.Fatalf("a detour that resolves must be silent; warnings: %v", got)
}
}
// TestFetchDetourProfileOverridesGlobals: the field-level inheritance, end to end.
// Globals says `direct` (the ethernet answer) and the ACTIVE profile says the SIM
// group — and the profile wins, which is the entire reason the option is
// per-profile rather than global.
func TestFetchDetourProfileOverridesGlobals(t *testing.T) {
m := listFetchDetourModel(tagDirect, model.Profile{
Name: "mobile-uplink", Enabled: true, FetchDetour: "node:tokyo",
})
opts, warns := generateFetchDetour(t, m)
if got := remoteDetourOf(t, opts, "bl-ads"); got != "tokyo" {
t.Fatalf("detour = %q, want %q — the active profile's override must beat globals; warnings=%v",
got, "tokyo", warns)
}
}
// TestFetchDetourProfileWithoutOverrideInheritsGlobals is the CONTROL for the test
// above: the same active profile, not setting the field, must not blank the
// globals value. Inheritance is per FIELD, and a profile that overrides something
// else entirely must leave this one alone.
func TestFetchDetourProfileWithoutOverrideInheritsGlobals(t *testing.T) {
m := listFetchDetourModel("group:auto", model.Profile{
Name: "mobile-uplink", Enabled: true, EndpointResolver: "cf",
})
opts, warns := generateFetchDetour(t, m)
if got := remoteDetourOf(t, opts, "bl-ads"); got != "auto" {
t.Fatalf("detour = %q, want %q — a profile that sets no fetch_detour inherits globals'; warnings=%v",
got, "auto", warns)
}
}
// TestFetchDetourUnresolvableRollsBackToDirectAndSaysSo is the owner's rule: an
// unusable value falls back to `direct` and the substitution is ANNOUNCED, with
// the profile, the field's value and what is in force instead all named.
//
// The three assertions are independent on purpose. A rollback with no warning is
// the silent substitution this project bans; a warning with no rollback would mean
// the engine downloads through nothing; and a warning that does not name the
// profile is unactionable, because a profile only misbehaves on its own uplink.
func TestFetchDetourUnresolvableRollsBackToDirectAndSaysSo(t *testing.T) {
m := listFetchDetourModel(tagDirect, model.Profile{
Name: "mobile-uplink", Enabled: true, FetchDetour: "group:sim-bypass",
})
opts, warns := generateFetchDetour(t, m)
if got := remoteDetourOf(t, opts, "bl-ads"); got != tagDirect {
t.Fatalf("detour = %q, want %q — an unresolvable detour must roll back to the outbound that always exists", got, tagDirect)
}
hits := fetchDetourWarnings(warns)
if len(hits) != 1 {
t.Fatalf("want exactly 1 %s warning, got %d: %v", fetchDetourNotAppliedTag, len(hits), hits)
}
for _, want := range []string{
`profile "mobile-uplink"`, // WHOSE value it was
`"group:sim-bypass"`, // the value that was refused
`"direct"`, // what is in force instead
"real", // and what that costs
} {
if !strings.Contains(hits[0], want) {
t.Fatalf("the warning must contain %q:\n %s", want, hits[0])
}
}
}
// TestFetchDetourFromGlobalsIsAttributedToGlobals: the same fault written in
// globals must not be blamed on a profile. The entity prefix is what
// apply/warnings.go parses into Section+Name, so a wrong one sends the panel's
// badge to the wrong page.
func TestFetchDetourFromGlobalsIsAttributedToGlobals(t *testing.T) {
_, warns := generateFetchDetour(t, listFetchDetourModel("group:ghost"))
hits := fetchDetourWarnings(warns)
if len(hits) != 1 {
t.Fatalf("want exactly 1 %s warning, got %d: %v", fetchDetourNotAppliedTag, len(hits), hits)
}
if !strings.Contains(hits[0], `globals "fetch_detour"`) {
t.Fatalf("a globals-level fault must be attributed to globals:\n %s", hits[0])
}
if strings.Contains(hits[0], "profile ") {
t.Fatalf("no profile set this value; the warning must not name one:\n %s", hits[0])
}
}
// TestFetchDetourBlockIsRefused: `block` passes model's SHAPE check (it is one of
// the two bare words) and resolveTarget resolves it happily to the block outbound
// — so nothing upstream stops it, and a rule-set aimed at it can never be
// downloaded. Worse than useless: a remote rule-set whose first fetch fails aborts
// engine start, so obeying this value would take the LAN down on a closed kill
// switch. It is refused HERE, by name.
func TestFetchDetourBlockIsRefused(t *testing.T) {
opts, warns := generateFetchDetour(t, listFetchDetourModel(tagBlock))
if got := remoteDetourOf(t, opts, "bl-ads"); got != tagDirect {
t.Fatalf("detour = %q, want %q — `block` discards the download and must not be obeyed", got, tagDirect)
}
hits := fetchDetourWarnings(warns)
if len(hits) != 1 {
t.Fatalf("want exactly 1 %s warning, got %d: %v", fetchDetourNotAppliedTag, len(hits), hits)
}
if !strings.Contains(hits[0], "`block` is not a route") {
t.Fatalf("the warning must say why block is refused rather than reading as a missing name:\n %s", hits[0])
}
}
// TestFetchDetourWarnsOncePerPass: filterFetchDetour is consulted once per remote
// rule-set. Four lists sharing one mistyped detour is ONE fault, and four copies
// of the same critical in the panel's banner is how the banner stops being read.
func TestFetchDetourWarnsOncePerPass(t *testing.T) {
m := listFetchDetourModel("group:ghost")
for _, name := range []string{"trackers", "malware", "porn"} {
m.Blocklists = append(m.Blocklists, model.Blocklist{
Name: name, Enabled: true, Source: "url",
URL: "https://lists.example/" + name + ".srs", UpdateInterval: "24h",
})
}
opts, warns := generateFetchDetour(t, m)
// The control: all four lists really were emitted, so the single warning is not
// single because three lists silently vanished.
for _, tag := range []string{"bl-ads", "bl-trackers", "bl-malware", "bl-porn"} {
if got := remoteDetourOf(t, opts, tag); got != tagDirect {
t.Fatalf("%s detour = %q, want %q", tag, got, tagDirect)
}
}
if hits := fetchDetourWarnings(warns); len(hits) != 1 {
t.Fatalf("one bad value must produce one warning, got %d: %v", len(hits), hits)
}
}
// TestFetchDetourReachesRoutingRuleSetsToo: the DNS-filter lists and the routing
// `config ruleset`s are two independent callers of the same builder
// (buildFilterRuleSetRaw and buildRoutingRuleSets), and the whole point of putting
// the resolution in remoteRuleSetAs is that neither can be wired up while the
// other is forgotten. A geoip routing ruleset is the second caller, and on this
// router it is the one carrying the country lists.
func TestFetchDetourReachesRoutingRuleSetsToo(t *testing.T) {
m := listFetchDetourModel("group:auto")
m.Rulesets = []model.Ruleset{{Name: "ru", Source: "geoip", Categories: []string{"ru"}}}
m.Rules = []model.Rule{{
Name: "ru-direct", Enabled: true, DstRuleset: []string{"ru"}, Target: tagDirect,
}}
opts, warns := generateFetchDetour(t, m)
if got := remoteDetourOf(t, opts, "rs-ru-ru"); got != "auto" {
t.Fatalf("routing rule-set detour = %q, want %q — the routing path must not keep the old constant; warnings=%v",
got, "auto", warns)
}
}
// TestFetchDetourCachedListAlsoSaysAppliedFromTheCache binds the marker
// apply/warnings.go grades on to the text this package actually emits.
//
// It is the producer half of the 4b fix: apply's degradedProtectionMarkers now
// contains "APPLIED FROM THE CACHE", and a marker with no producer is fiction —
// this project has already been bitten by five such markers matching no living
// text. Reword the cold-start branch in ruleset.go and this fails by name, before
// the panel goes quietly back to painting a working list red.
func TestFetchDetourCachedListAlsoSaysAppliedFromTheCache(t *testing.T) {
withCachePaths(t, t.TempDir())
resetColdStartCache()
prev := cachedRuleSetReader
cachedRuleSetReader = func(_, _ string) (*adapter.SavedBinary, error) {
return &adapter.SavedBinary{
Content: compiledSRS(t, []string{"ads.example"}),
LastUpdated: time.Now().Add(-36 * time.Hour).Truncate(time.Second),
}, nil
}
t.Cleanup(func() {
cachedRuleSetReader = prev
resetColdStartCache()
})
withRuleSetProbe(t, func(string) bool { return false }) // the SIM blocks the source
b := newBuilder(&model.Model{})
if _, ok := b.remoteRuleSet("bl-ads", fetchDetourSRSURL, "24h", `blocklist "ads"`); !ok {
t.Fatalf("a list with a loadable cached copy must still be applied; warnings: %v", b.warnings)
}
if !warnsHaveSub(b.warnings, "APPLIED FROM THE CACHE") {
t.Fatalf("apply/warnings.go grades severity on this exact phrase; warnings: %v", b.warnings)
}
// The control: the same instrument, no cached copy, must produce the OTHER text
// — otherwise "the marker is present" would be true of every outcome.
cachedRuleSetReader = func(_, _ string) (*adapter.SavedBinary, error) { return nil, os.ErrNotExist }
resetColdStartCache()
b2 := newBuilder(&model.Model{})
if _, ok := b2.remoteRuleSet("bl-ads", fetchDetourSRSURL, "24h", `blocklist "ads"`); ok {
t.Fatalf("with no cached copy the list must be omitted; warnings: %v", b2.warnings)
}
if warnsHaveSub(b2.warnings, "APPLIED FROM THE CACHE") {
t.Fatalf("nothing was served from the cache; the phrase must not appear: %v", b2.warnings)
}
}
@@ -0,0 +1,108 @@
//go:build linux
// The RUNTIME half of fetchdetour_wgdedup_test.go.
//
// The portable half asks whether the tag a rule-set downloads through still exists
// in the emitted option.Options. That is the right question, but it is asked of a
// struct. This one asks the SAME question of a real box, with the runtime's own
// resolver: box.New builds the OutboundManager, and dialer.InitializeDetour runs
// exactly the lookup dialer.DetourDialer.init performs on the first download —
// `outbound detour not found: <tag>` is its error, verbatim the one that fails
// RemoteRuleSet.StartContext, fails router.Start (FastFail), fails box.Start, and
// with the kill switch closed leaves the LAN with no way out.
//
// Why box.New and not box.Start: starting this config would make the engine
// actually FETCH the geosite list through a WireGuard peer at 203.0.113.10, which
// no test host can reach — so Start would fail identically whether the tag resolves
// or not, and an instrument that reports failure in both states measures nothing.
// The detour lookup is resolved against the outbound manager and is complete after
// New; the fetch that follows it is a different question, already covered by
// TestDNSFilterRemoteBlocklistHTTPClient over a local origin.
//
// Linux-only for the reason the whole *_linux_test.go half of this package is: the
// loop-guard routing_mark that generate emits is validated by box.New only here.
package generate
import (
"context"
"testing"
box "github.com/sagernet/sing-box"
"github.com/sagernet/sing-box/adapter"
"github.com/sagernet/sing-box/common/dialer"
"github.com/sagernet/sing-box/experimental/deprecated"
"github.com/sagernet/sing-box/shater/registry"
"github.com/sagernet/sing/service"
)
// TestFetchDetourRuleSetDownloadResolvesInTheBox drives the production shape —
// fetch_detour naming a WireGuard node that is also a chain hop — through a real
// box.New and then resolves every remote rule-set's download detour the way the
// engine will.
//
// The POSITIVE CONTROL is inside the same test: before asking about the real
// detour, the identical instrument is pointed at a tag that certainly does not
// exist, and it must report the miss. Without that, "no dangling detour found"
// would be indistinguishable from a resolver that cannot see a miss at all.
func TestFetchDetourRuleSetDownloadResolvesInTheBox(t *testing.T) {
for _, egress := range []string{"", "w1"} {
name := "chain shares the node's uplink"
if egress != "" {
name = "chain enters over its own egress"
}
t.Run(name, func(t *testing.T) {
opts, warns, err := GenerateWithWarnings(fetchDetourWGModel("node:wg1", egress))
if err != nil {
t.Fatalf("Generate: %v", err)
}
detours := remoteRuleSetDetours(t, opts)
if egress == "" {
// The merge case: the download must still be dialled through the
// WireGuard device, not quietly demoted to `direct`. Asserted here as
// well as portably, because this is the run that proves the tag the
// engine resolves and the tag the config carries are the same tag.
wg := wgEndpointTags(opts)
for rsTag, detour := range detours {
if !wg[detour] {
t.Fatalf("rule-set %q downloads through %q, not through the WireGuard device fetch_detour named (wireguard endpoints: %v)", rsTag, detour, wg)
}
}
}
ctx := context.Background()
var mgr deprecated.Manager = &recordingDeprecated{}
ctx = service.ContextWith(ctx, mgr)
ctx = registry.Context(ctx)
b, err := box.New(box.Options{Context: ctx, Options: withoutL3Ingress(t, opts)})
if err != nil {
t.Fatalf("box.New: %v\nwarnings: %v", err, warns)
}
defer func() { _ = b.Close() }()
om := service.FromContext[adapter.OutboundManager](ctx)
if om == nil {
t.Fatalf("no outbound manager in the box context — the instrument below cannot answer anything")
}
resolve := func(tag string) error {
return dialer.InitializeDetour(dialer.NewDetour(om, tag, true))
}
// Positive control: the same resolver, the same box, a tag nobody emitted.
const ghost = "no-such-outbound-tag"
if err := resolve(ghost); err == nil {
t.Fatalf("the detour resolver accepted %q, a tag no outbound carries — it cannot detect a miss, so the assertions below would pass no matter what", ghost)
}
for rsTag, detour := range detours {
if err := resolve(detour); err != nil {
t.Fatalf("rule-set %q downloads through detour %q and the RUNNING box cannot resolve it: %v\n"+
"That error is returned from RemoteRuleSet.StartContext on the first fetch; the router starts "+
"rule-sets with FastFail, so it is box.Start failing — and with the kill switch closed that is "+
"the whole LAN offline.\noutbounds=%v endpoints=%v\nwarnings=%v",
rsTag, detour, err, outboundTags(opts), endpointTags(opts), warns)
}
}
})
}
}
+376
View File
@@ -0,0 +1,376 @@
package generate
// THE FETCH DETOUR IS A REFERENCE, AND THE DEDUPLICATOR HAS TO SEE IT.
//
// # The defect
//
// wgdedup.go decides which WireGuard endpoints are still referenced by walking
// option.Options from every tag traffic can enter the graph at. Its rule-set arm
// read exactly one field:
//
// rt.RuleSet[i].RemoteOptions.DownloadDetour
//
// which is the DEPRECATED spelling, and the one this generator NEVER writes:
// ruleset.go remoteRuleSetAs emits `http_client{detour}` and leaves DownloadDetour
// empty (dnsfilter_test.go asserts exactly that, and warns if a deprecation notice
// ever appears). So the walk saw no rule-set download reference at all.
//
// That was harmless while the value was the hard-wired constant `direct` — a seed
// naming `direct` cannot keep anything alive that this pass would otherwise delete.
// It stopped being harmless when the detour became configurable
// (globals.fetch_detour, overridable per WAN profile), because `fetch_detour=node:
// <wg>` makes that node's BASE endpoint load-bearing: it is the outbound every
// blocklist, ruleset, geoip and geosite download dials through.
//
// # The consequence, which is what these tests measure
//
// Unseen, the base endpoint looks unreferenced. If the same node is also a chain
// hop — the ordinary shape, and the exact shape the production router runs — the
// pass has two endpoints for one private key, keeps the chain copy and deletes the
// base one. remapTags then rewrites every reference it KNOWS about; the rule-set's
// http_client detour is not one of them, so it is left naming a tag that no longer
// exists in the config.
//
// At runtime that is not a cosmetic dangle. RemoteRuleSet.StartContext builds its
// transport, and dialer.DetourDialer.init resolves the tag against the running
// box's OutboundManager: a miss is `outbound detour not found: <tag>`, the fetch
// fails, StartContext returns an error, the router starts rule-sets with FastFail
// — so box.Start fails, and with the kill switch closed a failed start is the whole
// LAN with no way out. See the R5 block in ruleset.go for the same failure arriving
// from the other direction.
//
// # Why the assertion is "the config is still connected", not "the seed is there"
//
// Asserting that the tag lands in the seed set would pass on a build that seeds it
// and then forgets to REMAP it, which is the same outage. So the test asks the only
// question that matters: after the pass, does the tag the rule-set dials through
// still exist among the outbounds/endpoints the engine will be given. The Linux
// half (fetchdetour_wgdedup_linux_test.go) asks the running box the same question
// with the runtime's own resolver.
import (
"testing"
C "github.com/sagernet/sing-box/constant"
"github.com/sagernet/sing-box/option"
"github.com/sagernet/sing-box/shater/model"
)
// fetchDetourWGModel is the production shape in miniature: one AmneziaWG-style
// WireGuard node that is BOTH a chain hop and the fetch detour every remote list
// downloads through, plus a geosite ruleset to be that download.
//
// entryEgress != "" gives the chain its own uplink, which makes the chain copy of
// the node a materially DIFFERENT device from the base one (different dialer) —
// the branch where the pass has to pick a loser rather than merge.
func fetchDetourWGModel(fetchDetour, entryEgress string) *model.Model {
g := model.DefaultGlobals()
g.FetchDetour = fetchDetour
hops := []string{"node:wg1", "node:ss1"}
m := &model.Model{
Globals: g,
Nodes: []model.Node{
{Name: "wg1", Enabled: true, URI: wgDedupURI(wgDedupKey(41), wgDedupKey(51), "203.0.113.10", 51820)},
{Name: "ss1", Enabled: true, URI: "ss://aes-256-gcm:secret@203.0.113.2:8388#ss1"},
},
Rulesets: []model.Ruleset{
{Name: "yt", Source: "geosite", Categories: []string{"youtube"}},
},
Rules: []model.Rule{
{Name: "via-chain", Enabled: true, Order: 10, DstPort: "443", Target: "chain:c"},
{Name: "yt-direct", Enabled: true, Order: 20, DstRuleset: []string{"yt"}, Target: "direct"},
},
}
if entryEgress != "" {
m.Egresses = []model.Egress{{Name: entryEgress, Type: "interface", Interface: "eth1"}}
hops = append([]string{"egress:" + entryEgress}, hops...)
}
m.Chains = []model.Chain{{Name: "c", Hops: hops}}
return m
}
// emittedTags is every outbound/endpoint tag the engine will be handed — i.e. every
// tag a Detour is allowed to name. It is the option-layer stand-in for the lookup
// dialer.DetourDialer.init performs against the running box's OutboundManager.
func emittedTags(opts option.Options) map[string]bool {
tags := make(map[string]bool, len(opts.Outbounds)+len(opts.Endpoints))
for i := range opts.Outbounds {
tags[opts.Outbounds[i].Tag] = true
}
for i := range opts.Endpoints {
tags[opts.Endpoints[i].Tag] = true
}
return tags
}
// remoteRuleSetDetours maps each emitted remote rule-set tag to the outbound tag it
// downloads through, reading the modern http_client{detour} the generator writes.
func remoteRuleSetDetours(t *testing.T, opts option.Options) map[string]string {
t.Helper()
if opts.Route == nil {
t.Fatalf("no route options emitted")
}
out := map[string]string{}
for i := range opts.Route.RuleSet {
rs := opts.Route.RuleSet[i]
if rs.Type != C.RuleSetTypeRemote {
continue
}
if rs.RemoteOptions.HTTPClient == nil {
t.Fatalf("remote rule-set %q carries no http_client at all; this test is measuring the wrong field", rs.Tag)
}
out[rs.Tag] = rs.RemoteOptions.HTTPClient.Detour
}
if len(out) == 0 {
t.Fatalf("no remote rule-set was emitted, so nothing here is being measured; rule-sets=%+v", opts.Route.RuleSet)
}
return out
}
// wgEndpointTags is the set of emitted tags that really are WireGuard devices —
// what "the download travels through the tunnel the operator named" means when
// checked rather than assumed.
func wgEndpointTags(opts option.Options) map[string]bool {
out := map[string]bool{}
for i := range opts.Endpoints {
if _, ok := wgDeviceKey(opts.Endpoints[i]); ok {
out[opts.Endpoints[i].Tag] = true
}
}
return out
}
// TestFetchDetourWGNodeStaysConnectedAfterDedup is the defect, measured as its
// consequence.
//
// The configuration is the production one: `fetch_detour=node:wg1` where wg1 is
// ALSO a chain hop. The chain here has no entry egress, so the hop copy and the
// base endpoint build the SAME device — one device written down twice, which the
// pass merges. Nothing about this config is wrong, and after the pass it must
// still be TRUE, in both of the ways it can stop being true:
//
// 1. the download detour must name a tag that EXISTS. Deleting the copy it named
// without repointing it is the dangling reference that fails box.Start.
// 2. that tag must still be the WIREGUARD DEVICE. Repointing it at `direct`
// would leave a perfectly valid config in which every list, geo and ruleset
// download goes out on the plain WAN with this router's real address — which
// is the whole thing fetch_detour exists to prevent, and the failure is
// invisible because the config still starts and the lists still load.
//
// Both are asserted because the two possible half-fixes fail exactly one each: a
// seed with no rewrite dangles (1), a rewrite with no seed silently degrades (2).
func TestFetchDetourWGNodeStaysConnectedAfterDedup(t *testing.T) {
opts, warns, err := GenerateWithWarnings(fetchDetourWGModel("node:wg1", ""))
if err != nil {
t.Fatalf("Generate: %v", err)
}
tags := emittedTags(opts)
wg := wgEndpointTags(opts)
for rsTag, detour := range remoteRuleSetDetours(t, opts) {
if !tags[detour] {
t.Fatalf("rule-set %q downloads through detour %q, and NO outbound or endpoint of that name is emitted.\n"+
"This is the outage: dialer.DetourDialer.init answers \"outbound detour not found: %s\", "+
"RemoteRuleSet.StartContext returns that error, the router starts rule-sets with FastFail, "+
"so box.Start fails — and with the kill switch closed the whole LAN is dark.\n"+
"emitted outbounds=%v endpoints=%v\nwarnings=%v",
rsTag, detour, detour, outboundTags(opts), endpointTags(opts), warns)
}
if !wg[detour] {
t.Fatalf("rule-set %q downloads through %q, which is not the WireGuard device fetch_detour=node:wg1 named "+
"(wireguard endpoints emitted: %v).\n"+
"The config is still valid and the engine will still start — and every blocklist, ruleset, geoip and "+
"geosite download now leaves on the plain WAN with this router's real address, which is the disclosure "+
"the option was set to prevent.\nwarnings=%v",
rsTag, detour, wg, warns)
}
}
}
// TestFetchDetourWGNodeKeepsExactlyOneDevice guards the other direction. Seeing the
// reference must not resurrect the duplicate it was blind to: one private key still
// has to leave exactly ONE device behind, or the merge that keeps the config
// connected has bought it at the price of the eviction loop wgdedup.go exists to
// prevent (two devices, one key, neither passing traffic).
func TestFetchDetourWGNodeKeepsExactlyOneDevice(t *testing.T) {
opts, _, err := GenerateWithWarnings(fetchDetourWGModel("node:wg1", ""))
if err != nil {
t.Fatalf("Generate: %v", err)
}
keys := map[string]int{}
for i := range opts.Endpoints {
if key, ok := wgDeviceKey(opts.Endpoints[i]); ok {
keys[key]++
}
}
for _, n := range keys {
if n > 1 {
t.Fatalf("one private key materialised %d times (endpoints=%v): two devices per key evict each other at the peer and NEITHER tunnel passes traffic", n, endpointTags(opts))
}
}
if len(keys) != 1 {
t.Fatalf("want exactly one wireguard device, got %d (endpoints=%v)", len(keys), endpointTags(opts))
}
}
// TestFetchDetourDirectNeedsNoSeed is the control that keeps the fix from being
// vacuous: with fetch_detour left at its default the download detour is `direct`,
// which names nothing this pass could preserve, and the base endpoint of a
// chain-only node must STILL be deleted as the duplicate it is. A seed that kept
// everything alive would pass the test above and quietly restore the two-devices
// bug for every config that does not use the option.
func TestFetchDetourDirectNeedsNoSeed(t *testing.T) {
opts, _, err := GenerateWithWarnings(fetchDetourWGModel("", ""))
if err != nil {
t.Fatalf("Generate: %v", err)
}
if got := endpointTags(opts); len(got) != 1 || got[0] != "chain-c-h1" {
t.Fatalf("endpoints = %v, want exactly [chain-c-h1]: with no fetch detour configured the base endpoint is genuinely unreferenced", got)
}
for rsTag, detour := range remoteRuleSetDetours(t, opts) {
if detour != tagDirect {
t.Fatalf("rule-set %q downloads through %q, want %q when nothing is configured", rsTag, detour, tagDirect)
}
}
}
// TestFetchDetourDownloadIsNeverPointedAtBlock covers the one case where `block` —
// this pass's fail-closed answer everywhere else — is the answer that takes the LAN
// down rather than protects it.
//
// When the copy the fetch detour resolved to loses the device conflict, remapTags
// would point the rule-set's download at `block`. A download through `block` cannot
// succeed, an uncached remote rule-set whose first fetch fails aborts engine start,
// and a failed start with a closed kill switch is the whole LAN offline. So the
// download degrades to `direct` — the path the R5 preflight already measured as
// reachable, which is why it is safe to promise a start — and says so at critical,
// because it IS the disclosure fetch_detour was set to prevent.
func TestFetchDetourDownloadIsNeverPointedAtBlock(t *testing.T) {
// Two chains over two different egresses: the base endpoint (what
// fetch_detour=node:wg1 resolves to) is a third materialisation and loses.
m := fetchDetourWGModel("node:wg1", "w1")
m.Egresses = append(m.Egresses, model.Egress{Name: "w2", Type: "interface", Interface: "eth2"})
m.Chains = append(m.Chains, model.Chain{Name: "d", Hops: []string{"egress:w2", "node:wg1"}})
m.Rules = append(m.Rules, model.Rule{Name: "via-d", Enabled: true, Order: 30, DstPort: "8443", Target: "chain:d"})
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("Generate: %v", err)
}
tags := emittedTags(opts)
for rsTag, detour := range remoteRuleSetDetours(t, opts) {
if detour == tagBlock {
t.Fatalf("rule-set %q downloads through %q: that fetch cannot succeed, and an uncached remote rule-set whose first fetch fails aborts engine start — the LAN goes dark to protect a list", rsTag, tagBlock)
}
if !tags[detour] {
t.Fatalf("rule-set %q downloads through %q, which no outbound or endpoint answers to", rsTag, detour)
}
}
// And it must not be silent about having done it.
if got := warningsContaining(warns, fetchDetourNotAppliedTag); len(got) == 0 {
t.Fatalf("the downgrade to %q was silent; an operator whose downloads now carry this router's real address has to be told. warnings=%v",
tagDirect, warns)
}
}
// --- the third reader: a subscription that says nothing ----------------------
// TestSubscriptionWithNoFetchViaPinsTheGeneralDetour covers the OTHER old
// two-state reader in wgdedup.go, subscriptionDetourSeeds.
//
// A subscription with no `fetch_via` at all is pulled through the GENERAL fetch
// detour, and apply.(*Applier).UpdateSubscription resolves that string against the
// RUNNING box — a reference this pass can neither see in option.Options nor
// rewrite. So the tag has to be seeded AND pinned, or the base endpoint is merged
// into the node's chain copy and the next refresh fails with "unknown outbound
// tag".
//
// The fixture deliberately emits NO remote rule-set, so the rule-set download seed
// cannot cover for a missing subscription seed: `wg1` survives here only because
// the subscription itself is counted.
func TestSubscriptionWithNoFetchViaPinsTheGeneralDetour(t *testing.T) {
m := fetchDetourModel(true, "")
m.Globals.FetchDetour = "node:wg1"
m.Subscriptions = []model.Subscription{{
// No FetchVia: the whole point. Under schema v2 this meant "direct"; under
// v3 it means "whatever globals.fetch_detour says", which here is wg1.
Name: "sub0", Enabled: true, URL: "https://example.net/sub",
}}
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("Generate: %v", err)
}
if got := endpointTags(opts); len(got) != 1 || got[0] != "wg1" {
t.Fatalf("endpoints = %v, want exactly [wg1].\n"+
"The subscription carries no fetch_via, so globals.fetch_detour=node:wg1 is what it is "+
"fetched through, and apply resolves that name against the running box. Merging it away "+
"leaves the tag unresolvable and every refresh of this subscription fails with "+
"\"unknown outbound tag\".\nwarnings=%v", got, warns)
}
// The chain must still work — it detours through the survivor.
if d := anyDetour(t, opts, "chain-c-h2"); d != "wg1" {
t.Fatalf("chain-c-h2 detour = %q, want wg1 (the merged survivor)", d)
}
}
// TestSubscriptionExplicitDirectIsStillNotAReference is the control that stops the
// fix above from being "seed everything". An explicit `fetch_via=direct` beats the
// general detour — it is the documented escape from the first-boot deadlock — so
// such a subscription dials no outbound and must keep nothing alive, even with
// globals.fetch_detour naming the node.
//
// The unrecognised value rides along: apply REFUSES to fetch it at all, so it is
// not a reference either, and treating it as one would preserve an endpoint on the
// strength of a typo.
func TestSubscriptionExplicitDirectIsStillNotAReference(t *testing.T) {
for _, via := range []string{"direct", "DIRECT", "proxied"} {
t.Run(via, func(t *testing.T) {
m := fetchDetourModel(true, "")
m.Globals.FetchDetour = "node:wg1"
m.Subscriptions = []model.Subscription{{
Name: "sub0", Enabled: true, URL: "https://example.net/sub", FetchVia: via,
}}
opts, _, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("Generate: %v", err)
}
if got := endpointTags(opts); len(got) != 1 || got[0] != "chain-c-h1" {
t.Fatalf("endpoints = %v, want exactly [chain-c-h1]: fetch_via=%q dials no engine outbound, so it must not keep the base endpoint alive", got, via)
}
})
}
}
// TestOptionSeedsReadsTheFieldTheGeneratorWrites is the narrow unit tripwire under
// the behavioural tests above. It states the fact that made the defect possible —
// the generator writes http_client{detour} and never download_detour — so a future
// change that moves the value back to the deprecated field fails HERE, with a
// message that says which field to look at, instead of failing as a dead LAN.
func TestOptionSeedsReadsTheFieldTheGeneratorWrites(t *testing.T) {
opts := &option.Options{Route: &option.RouteOptions{
RuleSet: []option.RuleSet{{
Type: C.RuleSetTypeRemote,
Tag: "rs-x",
RemoteOptions: option.RemoteRuleSet{
URL: "https://example.net/x.srs",
HTTPClient: &option.HTTPClientOptions{DialerOptions: option.DialerOptions{Detour: "wg1"}},
},
}},
}}
var found bool
for _, seed := range optionSeeds(opts) {
if seed.tag == "wg1" {
found = true
}
}
if !found {
t.Fatalf("optionSeeds did not report the http_client{detour} of a remote rule-set; seeds=%v", optionSeeds(opts))
}
replace := map[string]string{"wg1": "chain-c-h1"}
remapTags(opts, replace)
if got := opts.Route.RuleSet[0].RemoteOptions.HTTPClient.Detour; got != "chain-c-h1" {
t.Fatalf("remapTags left the http_client detour at %q; a reference the walk counts must be a reference the rewrite can repoint, or the merge leaves it dangling", got)
}
}
+14 -3
View File
@@ -200,9 +200,20 @@ type builder struct {
// Profiles. applyProfiles fills these once, guarded by effectiveComputed;
// buildRoute consumes them. For a model with no profiles effectiveRules is an
// exact copy of b.m.Rules (identical generate output).
effectiveRules []model.Rule // active-profile enable/disable applied
profileEndpointResolver string // active profile's EndpointResolver override ("" => none)
effectiveComputed bool // applyProfiles has run
effectiveRules []model.Rule // active-profile enable/disable applied
// activeProfile is the WAN profile in force, or nil when none is.
//
// It is kept WHOLE. This field used to be `profileEndpointResolver string` — one
// scalar copied out of the profile, with the "profile beats globals" decision
// then re-made by hand at the point of use. That shape does not survive a second
// override, let alone a third: each consumer would carry its own copy of the
// inheritance rule, and copies of a rule are free to disagree. They did — on the
// production router the panel drew `endpoint_resolver = local` from globals while
// the engine, under the active profile, was resolving through `yandex`. So the
// profile is held as-is and every override is resolved by model.Effective*, the
// one place that rule now lives (shared with apply and the panel).
activeProfile *model.Profile
effectiveComputed bool // applyProfiles has run
// Endpoint resolver (route.default_domain_resolver): the bootstrap-direct DNS
// server used ONLY to resolve proxy outbounds' server DOMAINS. Computed once by
+41 -5
View File
@@ -1,8 +1,6 @@
package generate
import (
"strings"
"github.com/sagernet/sing-box/shater/model"
)
@@ -37,14 +35,52 @@ func (b *builder) applyProfiles() {
for _, w := range owarns {
b.warnf("%s", w.Error())
}
// Per-profile endpoint-resolver override (by the active WAN profile). Consumed
// by endpointResolver() with priority OVER Globals.EndpointResolver.
b.profileEndpointResolver = strings.TrimSpace(prof.EndpointResolver)
// The active profile is kept whole; the DNS scalars it may override are
// resolved on demand through the accessors below, never copied out one field
// at a time. See builder.activeProfile (generate.go) for what the copies cost.
b.activeProfile = prof
}
b.effectiveRules = eff
}
// The three DNS scalars a WAN profile may override are answered HERE and nowhere
// else in this package, each by a one-line delegation to model.
//
// That is the whole point of them. The inheritance rule is one sentence — a
// non-blank profile value wins, per field, otherwise globals — and it is written
// once, in model/fetchdetour.go, because the two other consumers (apply, which
// builds the fetch client, and the panel, which has to PRINT what is in force)
// must give the same answer. They did not: the panel showed the operator
// `endpoint_resolver = local` out of globals while the engine, under the active
// profile, was resolving through `yandex`. A second copy of the rule inside
// generate would put the generator back in that same position.
//
// Each accessor calls applyProfiles first. That is idempotent, so they are safe
// to call from buildRoute or buildDNS in either order, and safe to call twice.
// effectiveResolverDefault is the resolver consulted FIRST — globals', or the
// active profile's override of it.
func (b *builder) effectiveResolverDefault() model.Override {
b.applyProfiles()
return model.EffectiveResolverDefault(b.m.Globals, b.activeProfile)
}
// effectiveResolverFallback is the resolver the failover chain retries against.
// Independent of effectiveResolverDefault by construction: inheritance is per
// field, so a profile that overrides only the fallback keeps globals' default.
func (b *builder) effectiveResolverFallback() model.Override {
b.applyProfiles()
return model.EffectiveResolverFallback(b.m.Globals, b.activeProfile)
}
// effectiveEndpointResolver is the bootstrap resolver for proxy SERVER DOMAINS
// (route.default_domain_resolver).
func (b *builder) effectiveEndpointResolver() model.Override {
b.applyProfiles()
return model.EffectiveEndpointResolver(b.m.Globals, b.activeProfile)
}
// resolveActiveProfile delegates to model.ResolveActiveProfile — the SINGLE
// source of truth for "which profile is active", shared with the netplane
// divert plan (nftPlanRules) so the engine's route rules and the nft
@@ -0,0 +1,416 @@
package generate
import (
"encoding/json"
"reflect"
"strings"
"testing"
C "github.com/sagernet/sing-box/constant"
"github.com/sagernet/sing-box/option"
"github.com/sagernet/sing-box/shater/model"
)
// A WAN profile may override THREE DNS scalars — resolver_default,
// resolver_fallback and endpoint_resolver — and this file is about what the
// GENERATED config does with them.
//
// Why it needs its own instrument at all: a profile override is invisible by
// construction. It only applies while that profile is active, so the SIM
// profile's `resolver_default` is unobservable until the ethernet cable comes
// out — at which point DNS stops and the config that broke it looks exactly like
// the config that worked. That is not hypothetical; it is what was measured on
// the production router on 2026-07-27, where the panel showed
// `endpoint_resolver = local` from globals while the engine, under the active
// profile, was resolving through `yandex`.
//
// Every test here drives the REAL generator end to end and reads the emitted
// option.Options. None of them asserts on model.Effective* directly — that is
// model's own test's job, and asserting it here would prove only that the
// generator can call a function, not that its output changed.
// overrideTestResolvers is the resolver set every model in this file uses.
//
// THE FIRST ENTRY IS A SENTINEL AND IS NEVER NAMED BY ANY TEST HERE. buildDNS has
// a last-resort path — when `final` names no server that was built, it falls
// through to servers[0], "the first resolver that survived", which has nothing to
// do with what anybody configured. While the first entry was also the value under
// test, that path produced the SAME answer as a correct resolution and the
// assertions could not tell them apart: a mutation that dropped the profile
// rollback entirely passed two of these tests. With a sentinel in front, any
// arrival at servers[0] shows up by name.
func overrideTestResolvers() []model.Resolver {
return []model.Resolver{
{Name: "servers0-sentinel", Type: "udp", Address: "192.0.2.1", Detour: "direct"},
{Name: "quad9", Type: "udp", Address: "9.9.9.9", Detour: "direct"},
{Name: "cloudflare", Type: "udp", Address: "1.1.1.1", Detour: "direct"},
{Name: "yandex", Type: "udp", Address: "77.88.8.8", Detour: "direct"},
{Name: "yandex-sec", Type: "udp", Address: "77.88.8.88", Detour: "direct"},
}
}
// mobileProfileModel is the shape these tests are about: the resolver set above,
// one profile pinned active, and whatever overrides the caller sets on it.
func mobileProfileModel(prof model.Profile, g model.Globals) *model.Model {
g.ActiveProfile = prof.Name
prof.Enabled = true
return &model.Model{
Globals: g,
Resolvers: overrideTestResolvers(),
Profiles: []model.Profile{prof},
}
}
// emittedDNSFinal returns the emitted dns.final, failing when there is no DNS plane.
func emittedDNSFinal(t *testing.T, opts option.Options) string {
t.Helper()
if opts.DNS == nil {
t.Fatalf("no DNS plane was emitted at all")
}
return opts.DNS.Final
}
// emittedFailoverServers returns the servers named by the three-rule failover chain
// resolverFallbackRules emits, in order ([1] primary, [2] fallback). An empty
// slice means no failover chain was emitted.
func emittedFailoverServers(opts option.Options) []string {
if opts.DNS == nil {
return nil
}
var out []string
for _, r := range opts.DNS.Rules {
if r.Type != C.RuleTypeDefault {
continue
}
if r.DefaultOptions.Action != C.RuleActionTypeEvaluate {
continue
}
out = append(out, r.DefaultOptions.RouteOptions.Server)
}
return out
}
// emittedDomainResolver returns route.default_domain_resolver's server tag, or ""
// when the field is unset. For diagnostics only — a test that WANTS the field set
// asserts on it directly, so that an unset one fails as "" rather than nil-panics.
func emittedDomainResolver(opts option.Options) string {
if opts.Route == nil || opts.Route.DefaultDomainResolver == nil {
return ""
}
return opts.Route.DefaultDomainResolver.Server
}
// configJSON renders a config for a failure message. option.Options carries typed
// pointers whose %+v is a list of addresses, which is exactly no help when the
// question is "what moved?".
func configJSON(t *testing.T, opts option.Options) string {
t.Helper()
b, err := json.Marshal(opts)
if err != nil {
return "<unmarshalable: " + err.Error() + ">"
}
return string(b)
}
// TestProfileResolverDefaultWins: the active profile's resolver_default is the
// engine's dns.final, not globals'. This is the whole point of the feature — on
// the SIM uplink quad9/cloudflare are unreachable (the operator's ISP refuses
// :443 to both, measured), so the profile has to be able to move the default.
func TestProfileResolverDefaultWins(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", ResolverDefault: "yandex"},
model.Globals{ResolverDefault: "quad9", ResolverFallback: "cloudflare"},
)
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if got := emittedDNSFinal(t, opts); got != "yandex" {
t.Fatalf("dns.final = %q, want the active profile's override %q (globals says %q); warns=%v",
got, "yandex", "quad9", warns)
}
// Per-FIELD inheritance: the profile said nothing about the fallback, so
// globals' fallback is still the one in the failover chain.
if got, want := emittedFailoverServers(opts), []string{"yandex", "cloudflare"}; !reflect.DeepEqual(got, want) {
t.Fatalf("failover chain = %v, want %v — the fallback is inherited from globals per field", got, want)
}
}
// TestProfileResolverFallbackWinsAlone: a profile that overrides ONLY the
// fallback keeps globals' default. Inheritance is per field; a profile is not
// an all-or-nothing block.
func TestProfileResolverFallbackWinsAlone(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", ResolverFallback: "yandex-sec"},
model.Globals{ResolverDefault: "quad9", ResolverFallback: "cloudflare"},
)
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if got := emittedDNSFinal(t, opts); got != "quad9" {
t.Fatalf("dns.final = %q, want globals' %q — the profile overrode only the fallback; warns=%v",
got, "quad9", warns)
}
if got, want := emittedFailoverServers(opts), []string{"quad9", "yandex-sec"}; !reflect.DeepEqual(got, want) {
t.Fatalf("failover chain = %v, want %v", got, want)
}
}
// TestProfileEndpointResolverWinsOverGlobals is the third scalar, asserted on the
// generated config rather than on the builder field it used to be copied into.
// (TestEndpointResolverProfileOverridesGlobals in endpoint_resolver_test.go
// already covers this; it is repeated here so that this file, which owns the
// rollback rule, also owns the positive control for it — a rollback test that
// passes because the override never worked would prove nothing.)
func TestProfileEndpointResolverWinsOverGlobals(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", EndpointResolver: "yandex"},
model.Globals{ResolverDefault: "quad9", EndpointResolver: "cloudflare"},
)
opts, _, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if opts.Route == nil || opts.Route.DefaultDomainResolver == nil {
t.Fatalf("route.default_domain_resolver must be set; route=%+v", opts.Route)
}
if got, want := opts.Route.DefaultDomainResolver.Server, endpointServerTag("yandex"); got != want {
t.Fatalf("default_domain_resolver.server = %q, want the profile's %q", got, want)
}
}
// assertRollbackWarning checks the one message shape the owner asked for: it must
// name the PROFILE, the FIELD, the REJECTED value and WHAT IS IN FORCE INSTEAD,
// and it must say that applying was not stopped. Anything less and the operator
// gets a fault they cannot act on for the one setting they cannot observe.
func assertRollbackWarning(t *testing.T, warns []string, profile, field, bad, inForce string) {
t.Helper()
var hit string
for _, w := range warns {
if strings.Contains(w, profileOverrideNotAppliedTag) && strings.Contains(w, field) {
hit = w
break
}
}
if hit == "" {
t.Fatalf("no %s warning for %s; got %v", profileOverrideNotAppliedTag, field, warns)
}
// Printed, not just matched. These sentences are the entire user interface for a
// fault the operator cannot otherwise observe, and a substring assertion says
// nothing about whether the whole thing reads as English. `go test -v` puts them
// in front of a human; the gate's green-run filter drops t.Log lines, so this
// costs nothing there.
t.Logf("operator-facing text:\n %s", hit)
for _, want := range []string{
`profile "` + profile + `"`, // whose override it was
field, // which field
`"` + bad + `"`, // the value that was rejected
inForce, // what is in force instead
"inert", // the override does nothing (and apply grades this critical)
"still applied", // and it did not stop the config
} {
if !strings.Contains(hit, want) {
t.Fatalf("the rollback warning must contain %q; got:\n %s", want, hit)
}
}
}
// TestProfileResolverDefaultUnknownRollsBackToGlobals is the owner's decision,
// verbatim: a profile that names a resolver this configuration does not have does
// NOT get to take DNS down on its own uplink. The globals value takes effect, and
// the substitution is announced.
//
// The distinction this asserts is a real fork in the code, not a formality: the
// weaker outcome is to leave the unusable name in place, let buildDNS notice no
// server was built for it and fall through to servers[0]. That produces a working
// default too — just not the operator's one. overrideTestResolvers' sentinel is what makes
// the two visibly different.
func TestProfileResolverDefaultUnknownRollsBackToGlobals(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", ResolverDefault: "yandex-typo"},
model.Globals{ResolverDefault: "quad9"},
)
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if got := emittedDNSFinal(t, opts); got != "quad9" {
t.Fatalf("dns.final = %q, want the globals value %q — an unusable profile override rolls back to what "+
"globals configured, it does not fall through to whichever resolver happened to build first (%q); warns=%v",
got, "quad9", "servers0-sentinel", warns)
}
assertRollbackWarning(t, warns, "mobile-uplink", "resolver_default", "yandex-typo", `"quad9"`)
}
// TestProfileResolverFallbackUnknownRollsBackToGlobals: same rule, second field.
// The failover chain must be the one globals configured, not silently absent.
func TestProfileResolverFallbackUnknownRollsBackToGlobals(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", ResolverFallback: "yandex-sec-typo"},
model.Globals{ResolverDefault: "quad9", ResolverFallback: "cloudflare"},
)
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if got, want := emittedFailoverServers(opts), []string{"quad9", "cloudflare"}; !reflect.DeepEqual(got, want) {
t.Fatalf("failover chain = %v, want %v — the globals fallback must survive an unusable profile override; warns=%v",
got, want, warns)
}
assertRollbackWarning(t, warns, "mobile-uplink", "resolver_fallback", "yandex-sec-typo", `"cloudflare"`)
}
// TestProfileEndpointResolverUnknownRollsBackToGlobals: third field. The
// bootstrap-direct clone must be built from GLOBALS' resolver, not left unset —
// an unset default_domain_resolver on a two-transport plane sends node hostnames
// down the client DNS rule chain, which is the bootstrap loop dns.go's
// implicitBootstrapResolver exists to prevent.
func TestProfileEndpointResolverUnknownRollsBackToGlobals(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", EndpointResolver: "ghost"},
model.Globals{ResolverDefault: "quad9", EndpointResolver: "cloudflare"},
)
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if opts.Route == nil || opts.Route.DefaultDomainResolver == nil {
t.Fatalf("default_domain_resolver must be set from the globals value; route=%+v, warns=%v", opts.Route, warns)
}
if got, want := opts.Route.DefaultDomainResolver.Server, endpointServerTag("cloudflare"); got != want {
t.Fatalf("default_domain_resolver.server = %q, want the globals value's clone %q", got, want)
}
assertRollbackWarning(t, warns, "mobile-uplink", "endpoint_resolver", "ghost", `"cloudflare"`)
}
// TestProfileResolverUnknownAndGlobalsUnsetSaysSo is the third branch of the
// rollback: globals has nothing to roll back TO. The message must then carry the
// CONSEQUENCE, because there is no substitute value to name — and the consequence
// differs per field, which is why the helper takes it from the caller.
func TestProfileResolverUnknownAndGlobalsUnsetSaysSo(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", ResolverFallback: "yandex-sec-typo"},
model.Globals{ResolverDefault: "quad9"}, // no globals fallback at all
)
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if got := emittedFailoverServers(opts); len(got) != 0 {
t.Fatalf("no failover chain may be emitted when neither side names a usable fallback, got %v", got)
}
assertRollbackWarning(t, warns, "mobile-uplink", "resolver_fallback", "yandex-sec-typo",
"there is no DNS failover at all")
}
// TestProfileAndGlobalsBothUnusableNamesBoth is the last of the three rollback
// branches: the globals value the override falls back to is broken too. The
// message must not pretend the substitution worked — it names BOTH bad values and
// the consequence, because "the globals value takes effect" would be false here
// and a warning that is false about the recovery is worse than no warning.
func TestProfileAndGlobalsBothUnusableNamesBoth(t *testing.T) {
m := mobileProfileModel(
model.Profile{Name: "mobile-uplink", ResolverDefault: "yandex-typo"},
model.Globals{ResolverDefault: "quad9-typo"},
)
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
// Nothing usable on either side, so buildDNS's own last resort applies — and it
// is REPORTED by the pre-existing warning, not silently taken.
if got := emittedDNSFinal(t, opts); got != "servers0-sentinel" {
t.Fatalf("dns.final = %q, want the first resolver that built (%q)", got, "servers0-sentinel")
}
assertRollbackWarning(t, warns, "mobile-uplink", "resolver_default", "yandex-typo",
"the globals value \"quad9-typo\" it falls back to is not usable either")
if !warnsHaveSub(warns, `resolver_default "quad9-typo" was not built`) {
t.Fatalf("the globals value must ALSO be reported as unbuilt — the rollback warning promises no more "+
"than that it fell back; got %v", warns)
}
}
// TestProfileWithNoDNSOverridesChangesNothing is the regression guard on the half
// of this change nobody asked for: every configuration that does NOT use a
// profile DNS override must generate exactly what it generated before.
//
// The instrument is a whole-config comparison against the same model with the
// profile removed, not a spot check on dns.final — a rewrite that resolved the
// default correctly and quietly moved a rule, a server or the fallback chain
// would pass a spot check. reflect.DeepEqual over option.Options is the only
// assertion that cannot be satisfied by getting the interesting field right.
func TestProfileWithNoDNSOverridesChangesNothing(t *testing.T) {
g := model.DefaultGlobals()
g.ResolverDefault = "quad9"
g.ResolverFallback = "cloudflare"
g.EndpointResolver = "yandex"
withProfile := mobileProfileModel(model.Profile{Name: "mobile-uplink"}, g)
bare := *withProfile
bare.Profiles = nil
bare.Globals.ActiveProfile = ""
got, gotWarns, err := GenerateWithWarnings(withProfile)
if err != nil {
t.Fatalf("GenerateWithWarnings (with profile): %v", err)
}
want, wantWarns, err := GenerateWithWarnings(&bare)
if err != nil {
t.Fatalf("GenerateWithWarnings (no profile): %v", err)
}
if !reflect.DeepEqual(got, want) {
// %+v on option.Options is pointer soup and says nothing about WHAT moved, so
// the three DNS scalars this change touches are spelled out first; the JSON
// dump behind them is what catches a difference anywhere else.
t.Fatalf("a profile that overrides no DNS scalar must produce an identical config.\n"+
" dns.final : %q (with profile) vs %q (without)\n"+
" failover chain : %v vs %v\n"+
" default_domain_resolver : %q vs %q\n"+
" full config with profile: %s\n"+
" full config without : %s",
emittedDNSFinal(t, got), emittedDNSFinal(t, want),
emittedFailoverServers(got), emittedFailoverServers(want),
emittedDomainResolver(got), emittedDomainResolver(want),
configJSON(t, got), configJSON(t, want))
}
if !reflect.DeepEqual(gotWarns, wantWarns) {
t.Fatalf("...and identical warnings\n with profile: %v\n without : %v", gotWarns, wantWarns)
}
}
// TestNoProfileAtAllUsesGlobals is the control for every rollback test above: on
// a model with no profiles the three scalars come from globals and nothing is
// warned about. Without it, an implementation that ignored profiles entirely
// (returning globals always) would pass the "unchanged behaviour" test and every
// rollback test, and fail only the three positive ones — so this is what makes
// those three mean something.
func TestNoProfileAtAllUsesGlobals(t *testing.T) {
m := &model.Model{
Globals: model.Globals{ResolverDefault: "quad9", ResolverFallback: "cloudflare", EndpointResolver: "yandex"},
Resolvers: overrideTestResolvers(), // servers[0] is the sentinel, never any of the three
}
opts, warns, err := GenerateWithWarnings(m)
if err != nil {
t.Fatalf("GenerateWithWarnings: %v", err)
}
if got := emittedDNSFinal(t, opts); got != "quad9" {
t.Fatalf("dns.final = %q, want globals' %q", got, "quad9")
}
if got, want := emittedFailoverServers(opts), []string{"quad9", "cloudflare"}; !reflect.DeepEqual(got, want) {
t.Fatalf("failover chain = %v, want %v", got, want)
}
if opts.Route == nil || opts.Route.DefaultDomainResolver == nil {
t.Fatalf("default_domain_resolver must be set from globals; route=%+v", opts.Route)
}
if got, want := opts.Route.DefaultDomainResolver.Server, endpointServerTag("yandex"); got != want {
t.Fatalf("default_domain_resolver.server = %q, want %q", got, want)
}
for _, w := range warns {
if strings.Contains(w, profileOverrideNotAppliedTag) {
t.Fatalf("no profile-override warning may be emitted for a model with no profiles: %q", w)
}
}
}
+69 -24
View File
@@ -975,8 +975,8 @@ func compileDomainList(domains []string, path string) error {
return nil
}
// remoteRuleSet builds a remote (.srs binary) rule-set fetched through the direct
// outbound (filterFetchDetour). Shared by every url/geosite/geoip source across
// remoteRuleSet builds a remote (.srs binary) rule-set fetched through the
// configured fetch detour (filterFetchDetour). Shared by every url/geosite/geoip source across
// the routing rule-sets AND the DNS-filter lists so the http_client{detour}, the
// binary format and the update-interval handling stay identical in one place. url
// must already be non-empty (the caller validates the source-specific inputs); an
@@ -988,19 +988,55 @@ func compileDomainList(domains []string, path string) error {
// to the engine. See the R5 block above for why such a rule-set would otherwise
// prevent the engine from starting at all.
func (b *builder) remoteRuleSet(tag, url, updateInterval, diag string) (option.RuleSet, bool) {
// The format the engine will be given. Both callers that override it afterwards
// derive it the same way (dnsfilter.go's url branch verbatim; the geo branches
// always fetch .srs), so this reproduces their answer rather than guessing — and
// it has to be right, because the cold-start check below decodes the CACHED copy
// with exactly this format's reader. The one caller that can legitimately
// disagree — a `url` ruleset with an explicit `format` — calls remoteRuleSetAs.
format, native := ruleSetURLIsEngineNative(url)
if !native {
format = C.RuleSetFormatBinary
}
return b.remoteRuleSetAs(tag, url, updateInterval, diag, format)
}
// remoteRuleSetAs is remoteRuleSet with the engine format stated outright, for the
// one caller whose config can override what the URL extension implies.
func (b *builder) remoteRuleSetAs(tag, url, updateInterval, diag, format string) (option.RuleSet, bool) {
// Resolved BEFORE the preflight, not at the struct literal below, and the order
// is the whole point: a mistyped fetch_detour must be reported even on the
// reconciles where every list is omitted for being unreachable. Otherwise the
// one warning that explains WHY the downloads fail would be suppressed by the
// failure it explains — visible only once the problem had already fixed itself.
// filterFetchDetour reports at most once per generate pass (dnsfilter.go).
detour := b.filterFetchDetour()
if v := remoteRuleSetUsable(url); !v.ok {
// Requirement: an operator must be able to tell "configured but NOT APPLIED
// right now" from "not configured at all". The omitted set produces no row in
// GET /api/ruleset/status (that endpoint projects the engine's ACTIVE
// rule-sets), so this warning is currently the only signal — hence the stable,
// greppable RULESET-NOT-APPLIED prefix, and hence it names the list, the tag
// and the reason. See the panel follow-up noted in the audit report.
b.warnf("RULESET-NOT-APPLIED: %s is configured but NOT ACTIVE: %s, "+
"so rule-set %q was omitted and matches NOTHING until it loads (a blocklist blocks nothing; a routing rule is skipped). "+
"Handing an unusable list to the engine would abort engine start and take the LAN down instead. "+
"Retried automatically on the next reconcile (~1 min) — no action needed unless this persists.",
diag, v.reason, tag)
return option.RuleSet{}, false
// COLD START. Unreachable is not the same as unusable: if sing-box's cache
// already holds a copy this build can load, StartContext will never touch the
// network for this list, so handing it over cannot fail box.Start — the whole
// reason the preflight exists. See coldstart_cache.go for what is proven.
if cached, ok := cachedRuleSetUsable(tag, format); ok {
b.warnf("RULESET-FROM-CACHE: %s could not be checked against its source (%s), "+
"but the cache holds a copy of rule-set %q that the engine can load (fetched %s), so it is APPLIED FROM THE CACHE — "+
"this is what lets a reboot with no working WAN still come up with its lists. "+
"That copy will not change until the source is reachable again, so the list is as old as the timestamp says.",
diag, v.reason, tag, cached.lastUpdated.UTC().Format(time.RFC3339))
} else {
// Requirement: an operator must be able to tell "configured but NOT APPLIED
// right now" from "not configured at all". The omitted set produces no row in
// GET /api/ruleset/status (that endpoint projects the engine's ACTIVE
// rule-sets), so this warning is currently the only signal — hence the stable,
// greppable RULESET-NOT-APPLIED prefix, and hence it names the list, the tag
// and the reason. See the panel follow-up noted in the audit report.
b.warnf("RULESET-NOT-APPLIED: %s is configured but NOT ACTIVE: %s, "+
"so rule-set %q was omitted and matches NOTHING until it loads (a blocklist blocks nothing; a routing rule is skipped). "+
"There is no usable copy in the cache either, so there is nothing to fall back to. "+
"Handing an unusable list to the engine would abort engine start and take the LAN down instead. "+
"Retried automatically on the next reconcile (~1 min) — no action needed unless this persists.",
diag, v.reason, tag)
return option.RuleSet{}, false
}
}
iv := strings.TrimSpace(updateInterval)
if iv == "" {
@@ -1014,17 +1050,24 @@ func (b *builder) remoteRuleSet(tag, url, updateInterval, diag string) (option.R
return option.RuleSet{
Type: C.RuleSetTypeRemote,
Tag: tag,
// Convention: remote lists are compiled .srs (binary), fetched through the
// always-present direct outbound (never the proxy/kill-switch) — a list
// needed to route the proxy must not depend on the proxy.
Format: C.RuleSetFormatBinary,
// Convention: remote lists are compiled .srs (binary) unless the URL says
// otherwise, fetched through the always-present direct outbound (never the
// proxy/kill-switch) — a list needed to route the proxy must not depend on
// the proxy. Stating the format here rather than leaving it to the caller to
// patch afterwards is what lets the cold-start check above decode the cached
// copy with the reader the engine will actually use.
Format: format,
RemoteOptions: option.RemoteRuleSet{
URL: url,
UpdateInterval: dur,
// http_client{detour} — NOT the deprecated download_detour, which makes
// box.New emit a deprecation notice.
// box.New emit a deprecation notice. The value is the CONFIGURED fetch
// detour (globals.fetch_detour, overridable per WAN profile), resolved and
// validated by filterFetchDetour (dnsfilter.go); it is `direct` when
// nothing is configured, which is what this field held unconditionally
// before the option existed.
HTTPClient: &option.HTTPClientOptions{
DialerOptions: option.DialerOptions{Detour: filterFetchDetour},
DialerOptions: option.DialerOptions{Detour: detour},
},
},
}, true
@@ -1225,13 +1268,15 @@ func (b *builder) buildRoutingRuleSetRaw(rs model.Ruleset) ([]option.RuleSet, []
// The URL serves something the engine can parse itself, so it stays REMOTE
// and sing-box keeps owning fetch/cache/update/status.
b.warnRuleSetTypeIgnored(diag, "url", rs.Type)
set, ok := b.remoteRuleSet(tag, rs.URL, rs.UpdateInterval, diag)
// Previously hard-coded to binary, which made a .json rule-set URL fail its
// initial fetch — and a failed initial fetch aborts engine start (R5). The
// format is handed DOWN rather than patched onto the result, so the
// cold-start cache check reads the cached copy with the same reader the
// engine will (an explicit `format` here can disagree with the extension).
set, ok := b.remoteRuleSetAs(tag, rs.URL, rs.UpdateInterval, diag, format)
if !ok {
return nil, nil, false // unreachable; warned, retried next reconcile
}
// Previously hard-coded to binary, which made a .json rule-set URL fail its
// initial fetch — and a failed initial fetch aborts engine start (R5).
set.Format = format
return []option.RuleSet{set}, []string{tag}, true
}
// Not engine-native: a plain-text list (hosts file / one domain per line),
+130 -4
View File
@@ -9,6 +9,7 @@ import (
C "github.com/sagernet/sing-box/constant"
"github.com/sagernet/sing-box/option"
"github.com/sagernet/sing-box/shater/model"
"github.com/sagernet/sing-box/shater/netplane"
"github.com/sagernet/sing-box/shater/parse"
)
@@ -292,6 +293,7 @@ func (b *builder) dedupWireGuardEndpoints(opts *option.Options) {
// the plain WAN with the router's real address.
remapTags(opts, replace)
removeRemappedEndpoints(opts, replace)
b.unblockRuleSetDownloads(opts)
// Warn LAST, against the config that actually resulted. Whether losing a copy
// takes the default route down is not decidable from the copy alone — the default
@@ -306,6 +308,54 @@ func (b *builder) dedupWireGuardEndpoints(opts *option.Options) {
}
}
// unblockRuleSetDownloads is the ONE place `block` is the wrong fail-closed answer,
// and it runs after the rewrite because only then is it known which downloads got
// pointed at it.
//
// Everywhere else in this pass a reference whose tunnel was deleted is sent to
// `block`: traffic that lost its tunnel must stop, not fall out onto the plain WAN.
// A remote rule-set DOWNLOAD is not traffic. RemoteRuleSet.StartContext fetches an
// uncached list at engine start and returns an error when the fetch fails, the
// router starts its rule-sets with FastFail, so that error is box.Start failing —
// and with the kill switch closed a failed start is the entire LAN offline (the R5
// block in ruleset.go is about nothing else). Dialling a download through `block`
// makes that failure CERTAIN.
//
// `direct` is not a guess at a better route, it is the one path already measured:
// the list is only in this config because remoteRuleSetUsable fetched its header
// over a plain DIRECT client and got an answer (or because the cold-start cache
// already holds a loadable copy, in which case start does not fetch at all). So the
// fallback is the path the preflight validated, which is what makes it safe to
// promise the engine will still start.
//
// It is loud, because it IS a disclosure: the operator set fetch_detour precisely so
// these downloads would not carry this router's address, and they now do.
func (b *builder) unblockRuleSetDownloads(opts *option.Options) {
if opts.Route == nil {
return
}
for i := range opts.Route.RuleSet {
rs := &opts.Route.RuleSet[i]
if rs.Type != C.RuleSetTypeRemote || rs.RemoteOptions.HTTPClient == nil {
continue
}
// `block` can only be here because the rewrite above put it here:
// filterFetchDetour (dnsfilter.go) refuses a configured `block` outright.
if !strings.EqualFold(rs.RemoteOptions.HTTPClient.Detour, tagBlock) {
continue
}
rs.RemoteOptions.HTTPClient.Detour = tagDirect
b.warnf("%s: rule-set %q was configured to download through the fetch detour, but the "+
"outbound that detour resolved to was just removed as a duplicate WireGuard device (see the "+
"warning above). Its download is going out %q instead — over the plain WAN, with this "+
"router's real address, which is the disclosure fetch_detour is set to prevent. It is NOT "+
"pointed at %q, because a remote rule-set whose first fetch fails aborts engine start, and "+
"with a closed kill switch that takes the whole LAN down. Fix the WireGuard duplicate named "+
"above, or point fetch_detour at a node that is not also materialised twice.",
fetchDetourNotAppliedTag, rs.Tag, tagDirect, tagBlock)
}
}
// defaultRouteBlocked reports whether the default route — route.Final, where every
// packet that matched no rule goes — now ends at `block`, i.e. whether the LAN has
// lost its way out. A group counts as usable while ANY member is usable, which is
@@ -627,19 +677,54 @@ func optionTagWeights(opts *option.Options, extraSeeds ...string) map[string]tag
// by name and never checks it, so the panel can fetch a disabled one and the detour
// must resolve when it does.
//
// # Three states, and why this asks the model instead of testing for "proxy"
//
// This used to be `EqualFold(FetchVia, "proxy")` — which was the whole truth while
// `fetch_via` had two states. Under schema v3 it has three, and the third is the
// one that produces a reference here: `fetch_via` ABSENT means the GENERAL fetch
// detour applies (globals.fetch_detour, or the active profile's override). A
// subscription that says nothing at all is therefore pulled through
// `globals.fetch_detour=node:<wg>` — a live reference to that node's BASE endpoint,
// resolved at RUNTIME against the running box, and one this pass cannot rewrite.
// The old test answered "direct, no reference" for it, so the base endpoint looked
// unused, was merged away into the node's chain copy, and the very next
// subscription refresh failed with "unknown outbound tag".
//
// model.SubscriptionFetchDetour is asked rather than the rule re-implemented,
// because it is the SAME function apply.(*Applier).UpdateSubscription calls to pick
// the detour it dials. Answering the question with the runtime's own resolver is
// what makes the seed provably the tag the runtime will look up, instead of a
// fourth copy of a table that has already drifted once. ok=false is an
// unrecognised `fetch_via`: apply refuses to fetch such a subscription at all, so
// it references nothing and seeds nothing.
//
// Non-node forms come along for free rather than being filtered out: one mapping
// (viaOutboundTag) covers them all, and seeding `egress-<x>` or a group tag costs
// nothing — a tag that does not exist is ignored by the walk. Filtering would be
// extra code that could only make the result less correct (a WG node that is a
// member of a group used ONLY as a fetch detour would lose its endpoint).
// member of a group used ONLY as a fetch detour would lose its endpoint). `direct`
// is the one form dropped, and only because it can keep nothing alive: it is always
// emitted and reaches nothing this pass could delete.
func (b *builder) subscriptionDetourSeeds() []string {
// The active profile decides the general detour. Resolved with warnings
// suppressed for the same reason filterFetchDetour does it: buildRoute has
// already reported them once and a duplicate paragraph per subscription is not
// a diagnostic.
var prof *model.Profile
b.withoutNewWarnings(func() { prof = b.resolveActiveProfile() })
var seeds []string
for i := range b.m.Subscriptions {
sub := b.m.Subscriptions[i]
if !strings.EqualFold(strings.TrimSpace(sub.FetchVia), "proxy") {
continue // a direct fetch dials no outbound at all
ov, ok := model.SubscriptionFetchDetour(sub, b.m.Globals, prof)
if !ok {
continue // not a decision; apply refuses to fetch it rather than guess
}
seeds = append(seeds, viaOutboundTag(sub.FetchDetour))
tag := viaOutboundTag(ov.Value)
if tag == tagDirect {
continue // a direct fetch dials no configured outbound at all
}
seeds = append(seeds, tag)
}
return seeds
}
@@ -734,6 +819,7 @@ func optionSeeds(opts *option.Options) []optionSeed {
for i := range rt.RuleSet {
if rt.RuleSet[i].Type == C.RuleSetTypeRemote {
add(rt.RuleSet[i].RemoteOptions.DownloadDetour)
add(remoteRuleSetHTTPDetour(rt.RuleSet[i]))
}
}
if rt.GeoIP != nil {
@@ -774,6 +860,45 @@ func walkOptionReachable(tag string, graph map[string]optionNode, reachable map[
walkOptionReachable(node.detour, graph, reachable)
}
// remoteRuleSetHTTPDetour returns the outbound tag a remote rule-set downloads
// through under the MODERN option — `http_client{detour}` — or "" when it carries
// none.
//
// It is a second reader beside RemoteOptions.DownloadDetour rather than a
// replacement for it because the two are different fields of the same struct and
// this pass has to see whichever one the generator wrote. It wrote the deprecated
// one never and the modern one always: ruleset.go remoteRuleSetAs emits
// `HTTPClient: &option.HTTPClientOptions{DialerOptions{Detour: detour}}` and leaves
// DownloadDetour empty — dnsfilter_test.go pins exactly that. So while this file
// read only DownloadDetour it was reading the field that is always "", i.e. it saw
// NO rule-set download reference at all.
//
// That was inert for as long as the value was the hard-wired constant `direct`: a
// seed naming `direct` reaches nothing this pass can delete. It stopped being inert
// the moment the detour became configurable (globals.fetch_detour / the profile
// override). `fetch_detour=node:<wg>` makes the node's BASE endpoint load-bearing;
// unseen, it looked unreferenced, was deleted as a duplicate of the node's chain
// copy, and the rule-set was left pointing at a tag that no longer exists.
// box.Start then fails on the first fetch with "outbound detour not found", and
// with the kill switch closed a failed start is the whole LAN offline. See
// TestFetchDetourWGNodeStaysConnectedAfterDedup.
func remoteRuleSetHTTPDetour(rs option.RuleSet) string {
if rs.RemoteOptions.HTTPClient == nil {
return ""
}
return rs.RemoteOptions.HTTPClient.Detour
}
// remapRemoteRuleSetHTTPDetour is the write half of remoteRuleSetHTTPDetour and
// must stay in step with it: a reference the seed walk counted is a reference this
// pass has to be able to repoint, or the merge it just made leaves a dangling tag.
func remapRemoteRuleSetHTTPDetour(rs *option.RuleSet, replace map[string]string) {
if rs.RemoteOptions.HTTPClient == nil {
return
}
remapField(&rs.RemoteOptions.HTTPClient.Detour, replace)
}
// ruleActionOutbound returns the outbound tag a route rule's action sends traffic
// to, or "" for the actions that route nowhere (reject/sniff/dns/hijack-dns/…).
// An empty Action is the route action (see option.RuleAction.UnmarshalJSON), so it
@@ -860,6 +985,7 @@ func remapTags(opts *option.Options, replace map[string]string) {
for i := range rt.RuleSet {
if rt.RuleSet[i].Type == C.RuleSetTypeRemote {
remapField(&rt.RuleSet[i].RemoteOptions.DownloadDetour, replace)
remapRemoteRuleSetHTTPDetour(&rt.RuleSet[i], replace)
}
}
if rt.GeoIP != nil {
-1
View File
@@ -36,4 +36,3 @@ func ParseDuration(s string) (time.Duration, bool) {
}
return d, true
}
-1
View File
@@ -35,4 +35,3 @@ func TestParseDuration(t *testing.T) {
}
}
}
+276
View File
@@ -0,0 +1,276 @@
package model
// The DNS/fetch overrides a profile may carry, and the one place their
// inheritance is resolved.
//
// This lives in model for the same reason profile.go and schedule.go do: the two
// packages that must agree — generate (which builds the engine's DNS servers and
// the download_detour of every remote rule-set) and apply (which builds the
// HTTP client a subscription fetch dials through) — cannot share a resolver any
// other way without one importing the other. While each side derived "what is in
// force" for itself, the panel derived a THIRD answer, and it was the wrong one:
// on the production router it drew `endpoint_resolver = local` out of globals
// while the engine, under the active profile, was using `yandex`.
//
// Everything here is SYNTACTIC. Whether a named resolver or a named detour
// target actually exists is the generator's verdict — model is a stdlib-only
// leaf and knows nothing about emitted tags.
import (
"fmt"
"strings"
)
// FetchViaMode classifies a subscription's `fetch_via`. The set is CLOSED and
// POSITIVE and there is no "everything else" member that behaves like one of the
// real ones: an unrecognised value gets FetchViaUnknown, its own branch, so a
// caller cannot silently treat a typo as a decision. That matters here more than
// in most places, because the value it used to be silently treated as was
// `direct` — a fetch that goes out over the plain WAN with this router's real
// address, and succeeds, so nothing about the outcome reveals the typo.
type FetchViaMode int
const (
// FetchViaInherit is `fetch_via` ABSENT (""): this subscription expresses no
// opinion and the general fetch detour (globals, or the active profile's
// override) applies.
FetchViaInherit FetchViaMode = iota
// FetchViaDirect is an explicit `direct`: fetch this feed in the clear no
// matter what the general setting says.
FetchViaDirect
// FetchViaProxy is an explicit `proxy`: fetch this feed through the
// subscription's OWN FetchDetour.
FetchViaProxy
// FetchViaUnknown is anything else. It is not a mode; it is a config defect
// that must be named (ValidateSubscriptions does) and must not be quietly
// mapped onto one of the three above.
FetchViaUnknown
)
// FetchVia spellings. These are the WHOLE accepted vocabulary of the option.
const (
FetchViaValueDirect = "direct"
FetchViaValueProxy = "proxy"
)
// FetchViaNames is the closed, positive list of values `option fetch_via` accepts
// (absent is the third state and has no spelling). Used by ValidateSubscriptions
// to say what the valid values ARE rather than only that this one is not.
var FetchViaNames = []string{FetchViaValueDirect, FetchViaValueProxy}
// ClassifyFetchVia maps a raw `fetch_via` value to its mode. Case- and
// space-insensitive, matching the comparisons the rest of the tree already makes
// (apply.subscriptionFetchWarnings, cmd/shaterd.subFetchViaProxy).
func ClassifyFetchVia(v string) FetchViaMode {
switch strings.ToLower(strings.TrimSpace(v)) {
case "":
return FetchViaInherit
case FetchViaValueDirect:
return FetchViaDirect
case FetchViaValueProxy:
return FetchViaProxy
default:
return FetchViaUnknown
}
}
// Override is one resolved override: the value in force plus WHERE it came from.
//
// From exists because the panel has to print it. `Endpoint resolver (globals)`
// showing `local` next to a line reading "in force right now: yandex (profile
// mobile-uplink)" is the shape the owner asked for, and it needs both halves
// from one call — a panel that recomputes the winner itself is how the three
// answers diverged in the first place.
type Override struct {
// Value is what is in force. "" means nothing is set anywhere, which is a
// legitimate answer (no resolver override, no fetch detour) and not an error.
Value string
// From names the section the value came from: FromGlobals, or "profile:<name>".
// It is FromGlobals when nothing is set anywhere — globals is where an
// unset value would be written.
From string
}
// FromGlobals is Override.From when the globals value (or the absence of any
// value) is what is in force.
const FromGlobals = "globals"
// The prefixes Override.From uses for the two sections that can supply a value.
// Exported because a consumer that wants the NAME back out — the panel prints
// "(profile mobile-uplink)" — would otherwise have to hard-code the separator or
// carry the *Profile alongside, and a hard-coded ":" in three files is how the
// two sides drift apart. Prefer the accessors below to slicing the string.
const (
FromProfilePrefix = "profile:"
FromSubscriptionPrefix = "subscription:"
)
// fromProfile renders Override.From for a profile-supplied value.
func fromProfile(name string) string { return FromProfilePrefix + name }
// ProfileName returns the profile that supplied this value, and whether one did.
// ok=false means the value came from globals (or from a subscription).
func (o Override) ProfileName() (string, bool) {
return strings.CutPrefix(o.From, FromProfilePrefix)
}
// SubscriptionName returns the subscription that supplied this value, and
// whether one did.
func (o Override) SubscriptionName() (string, bool) {
return strings.CutPrefix(o.From, FromSubscriptionPrefix)
}
// IsFromGlobals reports whether the value in force is the globals one — i.e.
// nothing overrode it. Named so a caller does not compare From to a literal.
func (o Override) IsFromGlobals() bool { return o.From == FromGlobals }
// overrideOf is the ONE inheritance rule, applied per field: a non-blank profile
// value wins, otherwise globals. prof may be nil (no active profile).
//
// Blank-not-set is deliberate and matches the renderer: strOpt omits an empty
// value, so a profile can only ever carry "" by not having the option at all. A
// profile therefore has no way to say "override this back to nothing", which is
// the right trade — the alternative is an operator who cannot tell an override
// they cleared from one they never wrote.
func overrideOf(globalsValue, profileValue, profileName string) Override {
if v := strings.TrimSpace(profileValue); v != "" {
return Override{Value: v, From: fromProfile(profileName)}
}
return Override{Value: strings.TrimSpace(globalsValue), From: FromGlobals}
}
// EffectiveResolverDefault resolves the resolver consulted FIRST: the active
// profile's `resolver_default` when it sets one, else globals'.
func EffectiveResolverDefault(g Globals, prof *Profile) Override {
if prof == nil {
return Override{Value: strings.TrimSpace(g.ResolverDefault), From: FromGlobals}
}
return overrideOf(g.ResolverDefault, prof.ResolverDefault, prof.Name)
}
// EffectiveResolverFallback resolves the resolver consulted LAST. Independent of
// EffectiveResolverDefault by construction: inheritance is per field, so a
// profile that sets only the fallback keeps globals' default.
func EffectiveResolverFallback(g Globals, prof *Profile) Override {
if prof == nil {
return Override{Value: strings.TrimSpace(g.ResolverFallback), From: FromGlobals}
}
return overrideOf(g.ResolverFallback, prof.ResolverFallback, prof.Name)
}
// EffectiveEndpointResolver resolves the bootstrap resolver used for proxy SERVER
// DOMAINS. It reads the field that already existed on both sections; it is here
// so all three DNS scalars are answered by one mechanism instead of the profile
// one being open-coded wherever it was needed.
func EffectiveEndpointResolver(g Globals, prof *Profile) Override {
if prof == nil {
return Override{Value: strings.TrimSpace(g.EndpointResolver), From: FromGlobals}
}
return overrideOf(g.EndpointResolver, prof.EndpointResolver, prof.Name)
}
// EffectiveFetchDetour resolves the GENERAL fetch detour — the one that applies
// to blocklist, allowlist, ruleset, geoip and geosite downloads, and to every
// subscription that does not override it.
func EffectiveFetchDetour(g Globals, prof *Profile) Override {
if prof == nil {
return Override{Value: strings.TrimSpace(g.FetchDetour), From: FromGlobals}
}
return overrideOf(g.FetchDetour, prof.FetchDetour, prof.Name)
}
// FromSubscription renders Override.From for a value a subscription supplied.
func FromSubscription(name string) string { return FromSubscriptionPrefix + name }
// SubscriptionFetchDetour resolves where ONE subscription's feed is fetched
// through, folding the subscription's override into the general setting:
//
// fetch_via absent -> the general detour (globals / active profile)
// fetch_via=direct -> "direct", explicitly, whatever the general setting is
// fetch_via=proxy -> this subscription's own fetch_detour
// anything else -> FetchViaUnknown; ok=false and the caller must refuse or
// report rather than pick a side
//
// ok=false is the whole reason this returns two values. There is no safe silent
// answer for an unrecognised `fetch_via`: choosing `direct` discloses the feed URL
// and this router's address to the provider and the ISP — the exact thing
// `fetch_via=proxy` is set to prevent — and choosing the general detour would put
// a feed through a tunnel the operator never asked for. Both are decisions, so
// the model refuses to make either and hands the caller a named defect instead.
func SubscriptionFetchDetour(s Subscription, g Globals, prof *Profile) (Override, bool) {
switch ClassifyFetchVia(s.FetchVia) {
case FetchViaInherit:
return EffectiveFetchDetour(g, prof), true
case FetchViaDirect:
return Override{Value: TargetDirect, From: FromSubscription(s.Name)}, true
case FetchViaProxy:
return Override{Value: strings.TrimSpace(s.FetchDetour), From: FromSubscription(s.Name)}, true
case FetchViaUnknown:
return Override{}, false
}
// Unreachable: ClassifyFetchVia returns one of the four above, and the switch
// covers all four by name. Present so a future member cannot fall out of a
// function that has to return something.
return Override{}, false
}
// Fetch-detour target vocabulary. `direct` and `block` stand alone; the other
// four are `<kind>:<name>` prefixes. This mirrors generate.resolveTarget, which
// is the single resolution point for rule targets, device targets, DNS resolver
// detours and egress targets — the fetch detour deliberately speaks the SAME
// language rather than inventing a fifth one.
const (
TargetDirect = "direct"
TargetBlock = "block"
)
// FetchDetourKinds is the closed, positive set of `<kind>:` prefixes a fetch
// detour may carry.
var FetchDetourKinds = []string{"node", "group", "egress", "chain"}
// FetchDetourFormWarning judges the SHAPE of a fetch-detour value and returns a
// complete operator-facing sentence when the shape is wrong, or "" when it is
// acceptable. Existence of the named node/group/egress/chain is NOT checked here
// — that is the generator's verdict, and model cannot see emitted tags.
//
// It exists because a mistyped PREFIX does not fail loudly downstream, it
// degrades: generate.resolveTarget's last branch treats any unrecognised
// `kind:name` as a BARE NAME and looks it up as a node, then a group. So
// `grup:auto` does not report "no such kind"; it reports nothing at all until the
// lookup misses, and a bare name that happens to collide with a real node would
// have silently routed the fetch somewhere nobody chose.
//
// The empty value is acceptable and means "not set" — every caller of this
// function is asking about an OPTIONAL override.
func FetchDetourFormWarning(field, value string) string {
v := strings.TrimSpace(value)
if v == "" {
return ""
}
if strings.EqualFold(v, TargetDirect) || strings.EqualFold(v, TargetBlock) {
return ""
}
kind, name := SplitTarget(v)
if !strings.Contains(v, ":") {
// A bare name is legal (it resolves if a NODE or GROUP carries that tag)
// but it cannot name an egress or a chain, whose outbounds are tagged
// `egress-X` and `chain-X-h1..hN`. Say so; do not reject it.
return fmt.Sprintf("%s is the bare name %q. It is looked up as a NODE and then as a GROUP, "+
"and nothing else: an egress of that name does not answer to it (its outbound is tagged "+
"egress-%s) and neither does a chain (chain-%s-h1..hN). Write `node:%s`, `group:%s`, "+
"`egress:%s` or `chain:%s` to say which one you mean.",
field, v, name, name, v, v, v, v)
}
if !inSet(strings.ToLower(kind), FetchDetourKinds) {
return fmt.Sprintf("%s is %q, and %q is not one of the kinds this option accepts (%s, or the "+
"bare words %s/%s). A value with an unrecognised prefix is NOT refused downstream — it is "+
"demoted to a bare name and looked up as a node and then a group, so it either matches "+
"nothing at all or matches something you did not name.",
field, v, kind, strings.Join(FetchDetourKinds, ":/")+":", TargetDirect, TargetBlock)
}
if strings.TrimSpace(name) == "" {
return fmt.Sprintf("%s is %q — the kind is there but it names nothing. Nothing is resolved and "+
"the fetch falls back to whatever the caller does with an unresolvable detour.", field, v)
}
return ""
}
+531
View File
@@ -0,0 +1,531 @@
package model
// Coverage for the profile DNS/fetch overrides and for the third state of
// `fetch_via` — the one that did not exist before schema v3.
//
// The pair that matters here is (absent, "direct"). They were the SAME value in
// the model until now, so every test below that distinguishes them is testing the
// change itself, and every test that renders an EMPTY override is testing that
// the distinction survives a save: an override written back as `option x ''`
// would read as "overridden with nothing" on the next boot and blank the globals
// value the profile was meant to leave alone.
import (
"reflect"
"strconv"
"strings"
"testing"
)
// --- the four new fields, through UCI and back ------------------------------
// TestNewOverridesRoundTrip: a model with ALL FOUR new options set survives
// render -> parse byte-for-byte in the fields' own values, and each option lands
// in the section the contract names (globals vs profile), spelled as `option`
// and not `list`.
func TestNewOverridesRoundTrip(t *testing.T) {
m := &Model{
Globals: Globals{FetchDetour: "group:sim-bypass"},
Profiles: []Profile{{
Name: "mobile-uplink",
Enabled: true,
ResolverDefault: "yandex",
ResolverFallback: "yandex-sec",
EndpointResolver: "yandex",
FetchDetour: "chain:swan-bypass",
}},
}
text := RenderUCIExport(m)
for _, want := range []string{
"\toption fetch_detour 'group:sim-bypass'",
"\toption resolver_default 'yandex'",
"\toption resolver_fallback 'yandex-sec'",
"\toption fetch_detour 'chain:swan-bypass'",
} {
if !strings.Contains(text, want) {
t.Fatalf("render is missing %q — the field is in the model and not in the file, so it "+
"cannot survive a save\n%s", want, text)
}
}
// `list` here would be silently dropped on read (uciSection keeps Options and
// Lists in separate maps and every one of these is read with s.opt), which is
// how a value can be in the config, visible in `uci show`, and inert.
for _, k := range []string{"fetch_detour", "resolver_default", "resolver_fallback"} {
if strings.Contains(text, "\tlist "+k+" ") {
t.Fatalf("%s is rendered as a `list`; it is read as an `option` and would be dropped\n%s", k, text)
}
}
got, err := ParseUCIExport(text)
if err != nil {
t.Fatalf("parse: %v", err)
}
if got.Globals.FetchDetour != "group:sim-bypass" {
t.Fatalf("globals.fetch_detour = %q, want group:sim-bypass", got.Globals.FetchDetour)
}
if len(got.Profiles) != 1 {
t.Fatalf("profiles = %+v", got.Profiles)
}
p := got.Profiles[0]
if p.ResolverDefault != "yandex" || p.ResolverFallback != "yandex-sec" ||
p.EndpointResolver != "yandex" || p.FetchDetour != "chain:swan-bypass" {
t.Fatalf("profile overrides did not round-trip: %+v", p)
}
}
// TestEmptyOverridesLeaveNoOption is the other half, and the load-bearing one:
// an override that is NOT set must leave no option behind. `option x ”` reads
// back as an override whose value is the empty string, which the renderer would
// then keep emitting — so "inherit from globals" would decay into "override with
// nothing" on the first save, permanently.
func TestEmptyOverridesLeaveNoOption(t *testing.T) {
m := &Model{
Globals: Globals{},
Profiles: []Profile{{Name: "ethernet-uplink", Enabled: true}},
}
text := RenderUCIExport(m)
for _, k := range []string{"fetch_detour", "resolver_default", "resolver_fallback", "endpoint_resolver"} {
if strings.Contains(text, "option "+k) {
t.Fatalf("an UNSET %s was emitted anyway; \"not set\" has become \"set to nothing\"\n%s", k, text)
}
}
got, err := ParseUCIExport(text)
if err != nil {
t.Fatalf("parse: %v", err)
}
if got.Globals.FetchDetour != "" {
t.Fatalf("globals.fetch_detour = %q, want empty", got.Globals.FetchDetour)
}
p := got.Profiles[0]
if p.ResolverDefault != "" || p.ResolverFallback != "" || p.FetchDetour != "" || p.EndpointResolver != "" {
t.Fatalf("an unset override came back non-empty: %+v", p)
}
// And the whole model must be identical, not merely these fields.
if !reflect.DeepEqual(m.Profiles, got.Profiles) {
t.Fatalf("profiles changed across render+parse:\nwant %+v\ngot %+v", m.Profiles, got.Profiles)
}
}
// --- unset vs explicit direct ------------------------------------------------
// TestFetchViaUnsetIsNotDirect is the change in one assertion: a subscription
// with no `fetch_via` and a subscription with `fetch_via='direct'` must come out
// of the parser as DIFFERENT models, and each must render back to what it was.
//
// Before schema v3 the parser answered "direct" to both (optOr's default), so the
// two rows below were indistinguishable the moment the config was read — and with
// a general fetch detour to inherit they belong in different places.
func TestFetchViaUnsetIsNotDirect(t *testing.T) {
const text = `package shater
config subscription
option name 'inherits'
option url 'https://p.example/a'
config subscription
option name 'explicit'
option url 'https://p.example/b'
option fetch_via 'direct'
`
m, err := ParseUCIExport(text)
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(m.Subscriptions) != 2 {
t.Fatalf("subs = %+v", m.Subscriptions)
}
if got := m.Subscriptions[0].FetchVia; got != "" {
t.Fatalf("a subscription with NO fetch_via parsed as %q; \"not set\" and \"direct\" are "+
"different states and the parser has merged them again", got)
}
if got := m.Subscriptions[1].FetchVia; got != "direct" {
t.Fatalf("explicit fetch_via = %q, want direct", got)
}
if ClassifyFetchVia(m.Subscriptions[0].FetchVia) != FetchViaInherit {
t.Fatal("an absent fetch_via must classify as FetchViaInherit")
}
if ClassifyFetchVia(m.Subscriptions[1].FetchVia) != FetchViaDirect {
t.Fatal("an explicit fetch_via='direct' must classify as FetchViaDirect")
}
// Both states must survive the save, which is where an over-eager renderer
// would put the difference back.
back, err := ParseUCIExport(RenderUCIExport(m))
if err != nil {
t.Fatalf("re-parse: %v", err)
}
if back.Subscriptions[0].FetchVia != "" || back.Subscriptions[1].FetchVia != "direct" {
t.Fatalf("the unset/direct distinction did not survive render+parse: %q / %q",
back.Subscriptions[0].FetchVia, back.Subscriptions[1].FetchVia)
}
}
// TestSubscriptionFetchDetourResolution pins the whole override table in one
// place: which of (subscription, profile, globals) supplies the answer, and
// where the answer says it came from.
func TestSubscriptionFetchDetourResolution(t *testing.T) {
g := Globals{FetchDetour: "group:general"}
prof := &Profile{Name: "mobile-uplink", FetchDetour: "group:sim-bypass"}
cases := []struct {
name string
sub Subscription
prof *Profile
wantValue string
wantFrom string
wantResolv bool
}{
{
name: "unset inherits globals",
sub: Subscription{Name: "qomar"},
wantValue: "group:general",
wantFrom: FromGlobals,
wantResolv: true,
},
{
name: "unset inherits the ACTIVE PROFILE over globals",
sub: Subscription{Name: "qomar"},
prof: prof,
wantValue: "group:sim-bypass",
wantFrom: "profile:mobile-uplink",
wantResolv: true,
},
{
name: "explicit direct beats the general setting",
sub: Subscription{Name: "qomar", FetchVia: "direct"},
prof: prof,
wantValue: "direct",
wantFrom: "subscription:qomar",
wantResolv: true,
},
{
name: "explicit proxy uses the subscription's own detour",
sub: Subscription{Name: "qomar", FetchVia: "proxy", FetchDetour: "node:tokyo"},
prof: prof,
wantValue: "node:tokyo",
wantFrom: "subscription:qomar",
wantResolv: true,
},
{
name: "an unrecognised fetch_via resolves to NOTHING",
sub: Subscription{Name: "qomar", FetchVia: "prxoy"},
prof: prof,
wantResolv: false,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, ok := SubscriptionFetchDetour(tc.sub, g, tc.prof)
if ok != tc.wantResolv {
t.Fatalf("ok = %v, want %v (got %+v)", ok, tc.wantResolv, got)
}
if !tc.wantResolv {
return
}
if got.Value != tc.wantValue || got.From != tc.wantFrom {
t.Fatalf("= {%q from %q}, want {%q from %q}", got.Value, got.From, tc.wantValue, tc.wantFrom)
}
})
}
}
// TestProfileInheritanceIsPerField: a profile that overrides ONE of the three DNS
// scalars must leave the other two on the globals value. Per-section inheritance
// ("a profile that names any resolver owns all of them") would drag a field the
// operator never wrote into the override.
func TestProfileInheritanceIsPerField(t *testing.T) {
g := Globals{ResolverDefault: "quad9", ResolverFallback: "cloudflare", EndpointResolver: "local"}
prof := &Profile{Name: "mobile-uplink", ResolverFallback: "yandex-sec"}
if got := EffectiveResolverDefault(g, prof); got.Value != "quad9" || got.From != FromGlobals {
t.Fatalf("resolver_default = {%q from %q}, want {quad9 from globals}: the profile set only "+
"the FALLBACK and must not have taken the default with it", got.Value, got.From)
}
if got := EffectiveResolverFallback(g, prof); got.Value != "yandex-sec" || got.From != "profile:mobile-uplink" {
t.Fatalf("resolver_fallback = {%q from %q}, want {yandex-sec from profile:mobile-uplink}", got.Value, got.From)
}
if got := EffectiveEndpointResolver(g, prof); got.Value != "local" || got.From != FromGlobals {
t.Fatalf("endpoint_resolver = {%q from %q}, want {local from globals}", got.Value, got.From)
}
// No active profile at all: globals, and said to be globals.
if got := EffectiveResolverFallback(g, nil); got.Value != "cloudflare" || got.From != FromGlobals {
t.Fatalf("with no active profile: {%q from %q}, want {cloudflare from globals}", got.Value, got.From)
}
}
// TestClassifyFetchViaIsClosed walks the whole accepted vocabulary and pins that
// everything outside it lands in FetchViaUnknown rather than being folded into
// one of the three real modes.
func TestClassifyFetchViaIsClosed(t *testing.T) {
known := map[string]FetchViaMode{
"": FetchViaInherit,
" ": FetchViaInherit,
"direct": FetchViaDirect,
"DIRECT": FetchViaDirect,
" proxy ": FetchViaProxy,
"Proxy": FetchViaProxy,
}
for v, want := range known {
if got := ClassifyFetchVia(v); got != want {
t.Errorf("ClassifyFetchVia(%q) = %v, want %v", v, got, want)
}
}
// The values that used to be silently read as "direct" — a plain-WAN fetch
// with this router's real address, which SUCCEEDS, so nothing revealed them.
for _, v := range []string{"prxoy", "yes", "1", "tunnel", "proxy!", "dir", "block"} {
if got := ClassifyFetchVia(v); got != FetchViaUnknown {
t.Errorf("ClassifyFetchVia(%q) = %v, want FetchViaUnknown — an unrecognised value must "+
"not be folded into a real mode", v, got)
}
}
}
// TestValidateSubscriptionsNamesUnknownFetchVia: the unknown branch is not just
// classified, it is REPORTED. A branch nobody prints is a silent skip.
func TestValidateSubscriptionsNamesUnknownFetchVia(t *testing.T) {
got := ValidateSubscriptions([]Subscription{
{Name: "typo", FetchVia: "prxoy"},
{Name: "inherits"},
{Name: "clear", FetchVia: "direct"},
{Name: "tunnelled", FetchVia: "proxy", FetchDetour: "group:auto"},
})
if len(got) != 1 {
t.Fatalf("want exactly one warning (the typo), got %d: %+v", len(got), got)
}
if got[0].Name != "typo" || !strings.Contains(got[0].Message, "prxoy") {
t.Fatalf("the warning must name the subscription AND the value: %+v", got[0])
}
}
// TestFetchDetourFormWarning: the shape check is a CLOSED positive list of
// prefixes. A mistyped prefix is the interesting case, because downstream it is
// not refused — resolveTarget demotes any unrecognised `kind:name` to a bare
// name and looks it up as a node and then a group.
func TestFetchDetourFormWarning(t *testing.T) {
silent := []string{"", " ", "direct", "DIRECT", "block", "node:tokyo", "group:auto",
"egress:wg0", "chain:swan", "Group:Auto"}
for _, v := range silent {
if msg := FetchDetourFormWarning("fetch_detour", v); msg != "" {
t.Errorf("FetchDetourFormWarning(%q) = %q, want silence", v, msg)
}
}
loud := map[string]string{
"grup:auto": "grup", // a mistyped prefix
"group:": "group:", // a kind naming nothing
"sim-bypass": "bare", // a bare name cannot reach an egress or a chain
}
for v, want := range loud {
msg := FetchDetourFormWarning("fetch_detour", v)
if msg == "" {
t.Errorf("FetchDetourFormWarning(%q) was silent; this value does not resolve to what it looks like", v)
continue
}
if !strings.Contains(msg, want) {
t.Errorf("FetchDetourFormWarning(%q) = %q, must mention %q", v, msg, want)
}
}
}
// TestValidateProfilesNamesAMissingResolver: a profile override naming a resolver
// that does not exist is invisible until that profile's uplink is the live one,
// which is exactly why it has to be reported at config time.
func TestValidateProfilesNamesAMissingResolver(t *testing.T) {
resolvers := []Resolver{{Name: "quad9"}, {Name: "yandex"}}
got := ValidateProfiles([]Profile{
{Name: "mobile-uplink", ResolverDefault: "yandex-typo", ResolverFallback: "yandex"},
{Name: "ethernet-uplink", ResolverDefault: "quad9"},
{Name: "no-overrides"},
}, resolvers)
if len(got) != 1 {
t.Fatalf("want exactly one warning, got %d: %+v", len(got), got)
}
if got[0].Name != "mobile-uplink" ||
!strings.Contains(got[0].Message, "yandex-typo") ||
!strings.Contains(got[0].Message, "resolver_default") {
t.Fatalf("the warning must name the profile, the FIELD and the value: %+v", got[0])
}
}
// --- the v2 -> v3 migration --------------------------------------------------
// v2Config is a config as this build's predecessor left it: schema 2, and three
// subscriptions whose fetch is de-facto direct in three different spellings.
const v2Config = `package shater
config globals 'globals'
option schema_version '2'
config subscription
option name 'silent'
option url 'https://p.example/a'
config subscription
option name 'explicit'
option url 'https://p.example/b'
option fetch_via 'direct'
config subscription
option name 'tunnelled'
option url 'https://p.example/c'
option fetch_via 'proxy'
option fetch_detour 'group:auto'
config subscription
option name 'junk'
option url 'https://p.example/d'
option fetch_via 'yes'
`
// TestMigrate2to3RecordsTheExistingDirectFetch: every subscription that fetches
// directly today says so explicitly afterwards, and the one that goes through the
// tunnel is untouched. Nothing about where any feed is fetched changes.
func TestMigrate2to3RecordsTheExistingDirectFetch(t *testing.T) {
var notes []string
old := migrateNotef
migrateNotef = func(f string, a ...any) { notes = append(notes, f) }
defer func() { migrateNotef = old }()
f := newFakeUCI(v2Config)
if err := migrateWith(f); err != nil {
t.Fatalf("migrate: %v", err)
}
if v, _ := f.Get("shater.globals.schema_version"); v != strconv.Itoa(CurrentSchemaVersion) {
t.Fatalf("schema_version = %q, want %d", v, CurrentSchemaVersion)
}
want := map[int]string{0: "direct", 1: "direct", 2: "proxy", 3: "direct"}
for idx, w := range want {
got, ok := f.Get("shater.@subscription[" + strconv.Itoa(idx) + "].fetch_via")
if !ok {
t.Fatalf("subscription[%d] has no fetch_via after the migration — its absence now means "+
"\"inherit the general detour\", which is NOT where it fetched before", idx)
}
if got != w {
t.Fatalf("subscription[%d].fetch_via = %q, want %q", idx, got, w)
}
}
// The junk value is rewritten, and the rewrite is announced.
if len(notes) != 1 {
t.Fatalf("an unrecognised fetch_via must be named when it is normalised; notes = %v", notes)
}
}
// TestMigrate2to3PreservesInherit: a config ALREADY at v3 must not have its
// deliberate blanks filled in. This is the guard that keeps the migration from
// undoing the feature every time it runs.
func TestMigrate2to3PreservesInherit(t *testing.T) {
f := newFakeUCI(`package shater
config globals 'globals'
option schema_version '3'
config subscription
option name 'inherits'
option url 'https://p.example/a'
`)
if err := migrateWith(f); err != nil {
t.Fatalf("migrate: %v", err)
}
if v, ok := f.Get("shater.@subscription[0].fetch_via"); ok {
t.Fatalf("a v3 config's deliberate \"inherit\" was overwritten with %q", v)
}
}
// TestMigrate2to3IsIdempotent: running the full chain again over its own output
// changes nothing.
func TestMigrate2to3IsIdempotent(t *testing.T) {
f := newFakeUCI(v2Config)
if err := migrateWith(f); err != nil {
t.Fatalf("first migrate: %v", err)
}
before, _ := f.Export("shater")
if err := migrateWith(f); err != nil {
t.Fatalf("second migrate: %v", err)
}
after, _ := f.Export("shater")
if before != after {
t.Fatalf("second run changed the config:\n--- before\n%s\n--- after\n%s", before, after)
}
}
// TestMigrate2to3ThenParseKeepsEveryFeedWhereItWas is the end-to-end statement of
// the promise: take a v2 config, migrate it, read it with THIS build's parser,
// and every subscription resolves to the same fetch path it had under the old
// one — where "the same" is judged against the old rule (proxy, or direct).
func TestMigrate2to3ThenParseKeepsEveryFeedWhereItWas(t *testing.T) {
// A general detour is configured, which is what makes the test sharp: an
// unmigrated blank would inherit THIS and move the feed into a tunnel.
g := Globals{FetchDetour: "group:sim-bypass"}
pre, err := ParseUCIExport(v2Config)
if err != nil {
t.Fatalf("parse v2: %v", err)
}
// The v2 rule, verbatim: proxy => its own detour, everything else => direct.
oldPath := map[string]string{}
for _, s := range pre.Subscriptions {
if strings.EqualFold(strings.TrimSpace(s.FetchVia), "proxy") {
oldPath[s.Name] = s.FetchDetour
} else {
oldPath[s.Name] = "direct"
}
}
f := newFakeUCI(v2Config)
migrateNotef = func(string, ...any) {}
if err := migrateWith(f); err != nil {
t.Fatalf("migrate: %v", err)
}
text, _ := f.Export("shater")
post, err := ParseUCIExport(text)
if err != nil {
t.Fatalf("parse v3: %v", err)
}
for _, s := range post.Subscriptions {
got, ok := SubscriptionFetchDetour(s, g, nil)
if !ok {
t.Fatalf("subscription %q does not resolve after the migration", s.Name)
}
if got.Value != oldPath[s.Name] {
t.Fatalf("subscription %q now fetches through %q, it used to fetch through %q — the "+
"migration was supposed to preserve every existing path",
s.Name, got.Value, oldPath[s.Name])
}
}
}
// TestOverrideAccessors: a consumer that needs the profile name back out must not
// have to know how From is spelled. Slicing on a hard-coded ":" in three files is
// how the two sides of a contract drift apart.
func TestOverrideAccessors(t *testing.T) {
g := Globals{ResolverDefault: "quad9", FetchDetour: "direct"}
prof := &Profile{Name: "mobile-uplink", ResolverDefault: "yandex"}
got := EffectiveResolverDefault(g, prof)
name, ok := got.ProfileName()
if !ok || name != "mobile-uplink" {
t.Fatalf("ProfileName() = (%q, %v), want (mobile-uplink, true)", name, ok)
}
if got.IsFromGlobals() {
t.Fatal("a profile-supplied value must not report IsFromGlobals")
}
fromG := EffectiveResolverDefault(g, nil)
if _, ok := fromG.ProfileName(); ok {
t.Fatal("a globals value must not report a profile name")
}
if !fromG.IsFromGlobals() {
t.Fatalf("IsFromGlobals() = false for From=%q", fromG.From)
}
sub, _ := SubscriptionFetchDetour(Subscription{Name: "qomar", FetchVia: "direct"}, g, prof)
sname, sok := sub.SubscriptionName()
if !sok || sname != "qomar" {
t.Fatalf("SubscriptionName() = (%q, %v), want (qomar, true)", sname, sok)
}
if _, ok := sub.ProfileName(); ok {
t.Fatal("a subscription-supplied value must not report a profile name")
}
}
+108 -1
View File
@@ -19,7 +19,7 @@ import (
// CurrentSchemaVersion is the schema this build understands. Bump it when adding
// a migration step below.
const CurrentSchemaVersion = 2
const CurrentSchemaVersion = 3
// uciRunner abstracts uci get/set/delete/commit/import so migrations AND the
// config-write path (WriteUCI) are unit-testable. Import feeds `uci export`-format
@@ -110,6 +110,7 @@ type migration struct {
var migrations = []migration{
{from: 0, to: 1, apply: migrate0to1},
{from: 1, to: 2, apply: migrate1to2},
{from: 2, to: 3, apply: migrate2to3},
}
func readSchemaVersion(u uciRunner) int {
@@ -651,6 +652,112 @@ func migrateDomainEntry(e string) string {
return "full:" + v
}
// --- v2 -> v3: a subscription's `fetch_via` says which of THREE things it means
//
// WHAT CHANGED. `option fetch_detour` arrives on `config globals` (and as a
// per-profile override) as the GENERAL fetch detour, and `fetch_via` on a
// subscription becomes an OVERRIDE of it rather than the only statement about
// where that feed is fetched. That gives the option a third state — absent,
// meaning "no opinion, use the general setting" — and until now absent was not a
// state at all: ReadUCI parsed a missing `fetch_via` as the literal string
// "direct" (`s.optOr("fetch_via", "direct")`), so "the operator chose clear-text"
// and "the operator never touched this row" were the same value.
//
// WHAT THIS STEP DOES. Every `config subscription` whose `fetch_via` is absent,
// blank, or anything OTHER than `proxy`/`direct` gets an explicit
// `fetch_via='direct'`. After it runs, every existing subscription fetches
// exactly where it fetched before, and the empty value means only what it now
// means: inherit.
//
// WHY A MIGRATION AND NOT A PARSER DEFAULT. Keeping the old default in ReadUCI
// would have been one line and would have preserved behaviour just as well — but
// it preserves it by making the new state unreachable, which is the whole feature.
// Removing the default WITHOUT this step is worse: the reinterpretation would
// then happen continuously and invisibly, at every read, and an operator who
// later set `globals.fetch_detour='group:sim-bypass'` would find every
// subscription that had never mentioned `fetch_via` silently moved into the
// tunnel. The change of meaning is real; it happens ONCE, it is written into
// /etc/config/shater where `uci show` can see it, and the pre-migration file is
// kept beside it (migrateWith takes the copy before any step runs).
//
// AN UNRECOGNISED VALUE IS NORMALISED, NOT PRESERVED, and that is deliberate.
// `fetch_via='yes'` meant `direct` in v2 — every reader in the tree asks
// `EqualFold(v, "proxy")` and treats everything else as direct — so writing
// `direct` records what the config ALREADY did rather than changing it. Leaving
// the junk in place would have handed the same decision to the runtime, where
// ClassifyFetchVia now refuses to make it (FetchViaUnknown), and a router whose
// subscriptions stopped fetching after an upgrade is a worse outcome than one
// whose config says out loud what it was doing. Each rewrite is announced through
// migrateNotef; none is silent.
//
// SCHEMA VERSION IS BUMPED, unlike the two additive changes that deliberately did
// not bump it (RetiredEgressTypes and Device.Blocklists — see their comments in
// model.go). Both of those left every stored field meaning exactly what it meant.
// This one does not: the ABSENCE of `fetch_via` changes meaning, and the bump is
// what guarantees the rewrite happens before a reader can act on the new meaning.
// Without it a v2 config would simply be read by a v3 parser, and there is no
// second chance to tell the two cases apart afterwards.
//
// IDEMPOTENCE. The step only writes where the value is not already `proxy` or
// `direct`, so a second run finds nothing to do. It needs no read-back of its own
// writes: the export is taken once, up front, and every decision comes from it.
func migrate2to3(u uciRunner) error {
text, ok := u.Export("shater")
if !ok || strings.TrimSpace(text) == "" {
return nil // no config yet (fresh install): nothing to migrate
}
secs, err := parseSections(text)
if err != nil {
return fmt.Errorf("read the current config: %w", err)
}
// Anonymous sections are addressed positionally and the index is PER TYPE, so
// it counts subscriptions only. This step adds no sections, so nothing shifts.
subIdx := -1
for _, s := range secs {
if s.Type != "subscription" {
continue
}
subIdx++
raw := s.opt("fetch_via")
switch ClassifyFetchVia(raw) {
case FetchViaProxy, FetchViaDirect:
// Already explicit and already in the closed set: leave it alone.
continue
case FetchViaInherit:
// Absent or blank — the case this whole step exists for.
case FetchViaUnknown:
migrateNotef("subscription %q: fetch_via %q is not a value this option has; "+
"it has always been read as %q and is now written down as %q, so nothing about "+
"where this feed is fetched changes",
firstNonEmpty(s.opt("name"), s.Name), raw, FetchViaValueDirect, FetchViaValueDirect)
}
key := fmt.Sprintf("shater.@subscription[%d].fetch_via", subIdx)
if err := u.Set(key, FetchViaValueDirect); err != nil {
return staged(u, fmt.Errorf("record the existing direct fetch of subscription %d: %w", subIdx, err))
}
}
if err := u.Commit("shater"); err != nil {
return staged(u, err)
}
return nil
}
// migrateNotef announces a migration rewrite that is not a plain copy — today,
// an unrecognised `fetch_via` being normalised to the value it already behaved
// as. It exists so no migration silently edits a value the operator wrote.
//
// Same seam and same reasoning as reportBackupProblem and subCacheLogf: model has
// no logger dependency and could not have one (logsink imports model), and
// os.Stderr is the daemon's log channel under procd and the terminal when shaterd
// is run by hand. Replaced in tests.
var migrateNotef = func(format string, args ...any) {
fmt.Fprintf(os.Stderr, "shater: migrate: "+format+"\n", args...)
}
// containsString reports whether list holds want (trimmed comparison).
func containsString(list []string, want string) bool {
for _, v := range list {
+78 -6
View File
@@ -179,11 +179,28 @@ type Globals struct {
// "" = not set. Consumed by the generator (generate/*, another agent); this is
// only the model contract.
EndpointResolver string
ProbeURL string // health-probe URL (default gstatic/generate_204)
ProbeInterval string // probe interval (e.g. 60s)
SchemaVersion int // UCI schema revision (0 = pre-versioned legacy)
ActiveProfile string // last profile switched to (display bookkeeping)
PanelPort int // admin-panel HTTP port; 0 = use the built-in default (8088)
// FetchDetour is the GENERAL fetch detour: the engine outbound every fetch
// this product makes dials through — blocklist, allowlist, ruleset, geoip and
// geosite downloads, and (unless the subscription overrides it, see
// Subscription.FetchVia) subscription feeds. "" = not set.
//
// Accepted forms are exactly SplitTarget's, the same vocabulary a rule target
// uses: direct | block | node:X | group:X | egress:X | chain:X, plus a bare
// name that resolves only if a NODE or GROUP carries that tag. Whether the
// named thing EXISTS is the generator's verdict, not this package's — model is
// a stdlib-only leaf and knows nothing about emitted tags. What model does own
// is the SHAPE: FetchDetourFormWarning names a prefix that is not one of the
// four, because such a value is silently demoted to "bare name" downstream and
// then misses.
//
// A per-profile override lives on Profile.FetchDetour; EffectiveFetchDetour
// resolves the two.
FetchDetour string
ProbeURL string // health-probe URL (default gstatic/generate_204)
ProbeInterval string // probe interval (e.g. 60s)
SchemaVersion int // UCI schema revision (0 = pre-versioned legacy)
ActiveProfile string // last profile switched to (display bookkeeping)
PanelPort int // admin-panel HTTP port; 0 = use the built-in default (8088)
// Geo-data source for `source=geosite`/`source=geoip` rule-sets. Resolved by
// generate.SetGeoProvider / GeoRuleSetURL; this package only carries the values
@@ -524,7 +541,36 @@ type Subscription struct {
Enabled bool
URL string
UpdateInterval string
FetchVia string // direct|proxy
// FetchVia is this subscription's OVERRIDE of the general fetch detour
// (Globals.FetchDetour, or the active profile's). Three states, and the third
// one is the point:
//
// "proxy" this feed goes through THIS subscription's own FetchDetour below
// "direct" this feed goes out in the clear, whatever the general setting says
// "" NOT SET: the general setting applies (ClassifyFetchVia -> FetchViaInherit)
//
// "" AND "direct" USED TO BE THE SAME THING and they are not any more. Until
// schema v3 ReadUCI parsed an absent `option fetch_via` as the literal string
// "direct" (`s.optOr("fetch_via", "direct")`), so the model could not tell "the
// operator chose a clear-text fetch" from "the operator never touched this
// row" — and with a general fetch detour to inherit, those two must land in
// different places. The distinction is carried by the EMPTY STRING rather than
// a *string or a companion bool because UCI already draws it exactly there
// (`option` present vs absent), strOpt already omits an empty value, and the
// accepted vocabulary is a closed positive set that "" is not in. A pointer
// would change the JSON the panel exchanges over /api/config and add a nil
// class to a stdlib-only leaf; a companion bool would allow the contradictory
// state (not-set + "proxy") that nothing could reject.
//
// Migrate v2->v3 (migrate2to3) writes `fetch_via='direct'` into every
// subscription that had no value, so no existing feed changes where it goes:
// the reinterpretation is performed ONCE, in the open, by a migration that
// leaves its result in /etc/config/shater, instead of continuously and
// invisibly by the parser.
//
// Anything OTHER than those three is FetchViaUnknown — a named branch, not a
// silent demotion to direct. See ClassifyFetchVia.
FetchVia string
// FetchDetour names the engine outbound this subscription's fetch dials
// through when FetchVia=="proxy". It is read by apply.(*Applier).HTTPClient,
@@ -943,6 +989,32 @@ type Profile struct {
// active WAN: SIM->yandex, WiFi->DoH). "" = no override. Consumed by the
// generator (another agent); model contract only.
EndpointResolver string
// ResolverDefault / ResolverFallback are per-profile overrides of the two
// Globals scalars of the same name — the resolver consulted FIRST and the one
// consulted LAST. Built to the SAME contract as EndpointResolver above: "" = no
// override, inherit from globals.
//
// INHERITANCE IS PER FIELD, not per section. A profile that sets only
// resolver_fallback keeps globals' resolver_default. The alternative — "a
// profile that names any resolver owns all of them" — would silently drag a
// field the operator never wrote into the override, which is the same class of
// defect as an open `default:`.
//
// Why they exist: measured on the production router, an operator's SIM uplink
// refuses TCP/443 to 9.9.9.9 and 1.1.1.1 while carrying everything else, so the
// globals resolver pair that works on ethernet resolves NOTHING on mobile — and
// the node server names, the hop in front of an AWG endpoint and therefore the
// whole chain died with it. The profile that already knows which uplink is live
// is the right place to say which resolvers that uplink can reach.
ResolverDefault string
ResolverFallback string
// FetchDetour is a per-profile override of Globals.FetchDetour — same accepted
// forms, same "" = no override rule. Same motivation: on the SIM uplink the
// list/geo/subscription fetches have to travel through a group of nodes that
// operator does not block, and on ethernet they may go direct.
FetchDetour string
}
// Ruleset is a `config ruleset` (reusable domain/ip list).
+11
View File
@@ -74,6 +74,10 @@ func RenderUCIExport(m *Model) string {
w.strOpt("resolver_default", g.ResolverDefault)
w.strOpt("resolver_fallback", g.ResolverFallback)
w.strOpt("endpoint_resolver", g.EndpointResolver)
// strOpt (omit-empty), NOT an always-emit variant: an empty FetchDetour means
// "not set", and `option fetch_detour ''` would read back as "set to nothing",
// which is a different state the moment a profile inherits from it.
w.strOpt("fetch_detour", g.FetchDetour)
w.strOpt("probe_url", g.ProbeURL)
w.strOpt("probe_interval", g.ProbeInterval)
w.intOpt("schema_version", g.SchemaVersion)
@@ -251,7 +255,14 @@ func RenderUCIExport(m *Model) string {
w.listOpt("match_iface", p.MatchIface)
w.listOpt("enable_rule", p.EnableRules)
w.listOpt("disable_rule", p.DisableRules)
// The four per-profile overrides. Every one of them is strOpt: an empty
// override MUST leave no option behind, or the next read turns "inherit"
// into "override with nothing" and the profile silently blanks the globals
// value it was meant to leave alone.
w.strOpt("endpoint_resolver", p.EndpointResolver)
w.strOpt("resolver_default", p.ResolverDefault)
w.strOpt("resolver_fallback", p.ResolverFallback)
w.strOpt("fetch_detour", p.FetchDetour)
}
for _, res := range m.Resolvers {
+223 -43
View File
@@ -17,13 +17,18 @@ package model
// OBJECT (not a bare array — an object carries the format version and the
// authoritative subscription name, which a bare array cannot):
//
// {"version": 1, "sub": "<subscription name>", "nodes": [Node, ...]}
// {"version": 1, "sub": "<name>", "seq": <generation>, "nodes": [Node, ...]}
//
// Node uses its natural Go-field JSON encoding — the same shape the panel
// already exchanges over GET/PUT /api/config. Unknown fields are ignored on
// read (forward compatibility); an unknown version is skipped with a warning.
// Files are written indented so `cat` on the router stays a usable debugger.
//
// `seq` is the generation counter that decides which of two copies is in force.
// It is carried INSIDE the file because the filesystem cannot answer the
// question: see the subCacheFile doc comment for the measurement that killed the
// modification-time version of this rule.
//
// # Where
//
// The persistent home is /etc/shater/subs (survives reboot). Like
@@ -32,8 +37,10 @@ package model
// warning — a reboot then costs one re-fetch, never a broken config. The
// SHATER_SUBS_DIR env var overrides the directory outright (tests, dev hosts).
// Writes are atomic: temp file in the target dir, then rename, so a reader
// (daemon vs CLI) never observes a torn file. A refresh that produces the
// byte-identical file is skipped entirely (no overlay churn).
// (daemon vs CLI) never observes a torn file. A refresh whose node set is
// unchanged is skipped entirely (no overlay churn) — unless the copy in force is
// the tmpfs one, in which case the persistent home has to be rewritten to take
// the authority back.
//
// # The single merge point
//
@@ -46,7 +53,6 @@ package model
// /etc/config/shater on the next write.
import (
"bytes"
"encoding/json"
"fmt"
"os"
@@ -56,12 +62,6 @@ import (
)
const (
// subCacheDirPersistent is the preferred (survives reboot) home for the
// per-subscription node caches; subCacheDirFallback is the tmpfs degradation
// used when the overlay cannot take the write (same policy as generate/cache.go).
subCacheDirPersistent = "/etc/shater/subs"
subCacheDirFallback = "/tmp/shater-subs"
// subCacheDirEnv overrides the cache directory entirely (no fallback chain):
// tests point it at a temp dir; a dev host keeps /etc clean.
subCacheDirEnv = "SHATER_SUBS_DIR"
@@ -71,10 +71,55 @@ const (
subCacheVersion = 1
)
// subCacheDirPersistent is the preferred (survives reboot) home for the
// per-subscription node caches; subCacheDirFallback is the tmpfs degradation used
// when the overlay cannot take the write (same policy as generate/cache.go).
//
// Variables, not constants, for the same reason migrate.go's liveConfigPath is:
// the DEGRADED state — a persistent dir that holds a readable file and refuses
// new ones — is a two-directory state, and SHATER_SUBS_DIR collapses the chain to
// one directory, so it cannot produce that state at all. A test that cannot
// produce the positive case proves nothing about the negative one.
var (
subCacheDirPersistent = "/etc/shater/subs"
subCacheDirFallback = "/tmp/shater-subs"
)
// subCacheFile is the on-disk JSON shape (see the package comment).
//
// Seq is the GENERATION COUNTER, and it is what decides which of two copies of
// the same subscription is in force. Every write takes the highest Seq found in
// any candidate directory and adds one, so "newer" is a fact carried inside the
// file rather than a property of the filesystem.
//
// IT REPLACED MODIFICATION TIME, WHICH DOES NOT WORK HERE. Two consecutive
// writes land in the same kernel timer tick and receive BIT-IDENTICAL
// timestamps — measured in the Linux CI container, the persistent and the tmpfs
// copy came out at the same UnixNano to the digit, so a strict "is newer than"
// test was false and the STALE copy won every time. That is not a rare tie: two
// small writes in one operation always fall in one tick, so on Linux it was the
// normal outcome, and it is why this only ever looked correct on a Windows host.
//
// Worse, and independent of resolution: this router has no RTC. OpenWrt boots
// with the clock at the epoch until NTP syncs, and the uplink this whole feature
// exists for is a SIM that drops. So a cache written before a sync carries a
// 1970 stamp and would lose to any older file written after a previous one —
// mtime ordering can invert across a reboot. This codebase already refuses to
// act on an unsynced clock elsewhere (alert/expiry.go); the cache must not
// depend on one either.
//
// Seq is absent (0) in every file written before this change. That is handled
// as a legitimate value, not a special case: a legacy pair ties at 0 and the
// tie-break keeps the persistent copy, which is exactly the behaviour those
// files had before. The FORMAT VERSION IS NOT BUMPED — a new field is
// transparent in both directions (decodeSubCache uses a plain json.Unmarshal, so
// an older daemon ignores `seq` and a newer one reads 0), and bumping it would
// make every cache file already on the router unreadable, which is precisely the
// cold-start wipe this file exists to prevent.
type subCacheFile struct {
Version int `json:"version"`
Sub string `json:"sub"`
Seq int `json:"seq"`
Nodes []Node `json:"nodes"`
}
@@ -113,29 +158,106 @@ func subCacheFileName(sub string) string {
return b.String() + ".json"
}
// marshalSubCache renders the canonical bytes for a subscription's cache file.
// Deterministic (fixed field order, fixed indent) so "did it change" is a plain
// byte comparison.
func marshalSubCache(sub string, nodes []Node) ([]byte, error) {
// marshalSubCache renders the canonical bytes for one generation of a
// subscription's cache file. Deterministic (fixed field order, fixed indent) so
// two renderings of the same generation are byte-comparable.
func marshalSubCache(sub string, seq int, nodes []Node) ([]byte, error) {
if nodes == nil {
nodes = []Node{} // encode as [], never null
}
data, err := json.MarshalIndent(subCacheFile{Version: subCacheVersion, Sub: sub, Nodes: nodes}, "", "\t")
data, err := json.MarshalIndent(
subCacheFile{Version: subCacheVersion, Sub: sub, Seq: seq, Nodes: nodes}, "", "\t")
if err != nil {
return nil, err
}
return append(data, '\n'), nil
}
// subCacheCandidate is one readable copy of a subscription's cache, and the
// directory it was found in. dirIndex is the position in subCacheDirs(), so 0 is
// the persistent home.
type subCacheCandidate struct {
dirIndex int
path string
file *subCacheFile
}
// subCacheCandidatesFor reads every readable, decodable copy of one
// subscription's cache, in directory order.
func subCacheCandidatesFor(sub string) []subCacheCandidate {
name := subCacheFileName(sub)
var out []subCacheCandidate
for i, dir := range subCacheDirs() {
path := filepath.Join(dir, name)
data, rerr := os.ReadFile(path)
if rerr != nil {
continue
}
f, derr := decodeSubCache(data)
if derr != nil {
subCacheLogf("%s: %v — ignoring the file (the next refresh rewrites it)", path, derr)
continue
}
out = append(out, subCacheCandidate{dirIndex: i, path: path, file: f})
}
return out
}
// authoritativeSubCache picks the copy in force: the HIGHEST generation counter
// wins, and a tie keeps the earliest directory — i.e. the persistent home.
//
// The tie-break is not arbitrary. A tie means neither copy claims to supersede
// the other, which happens in exactly two situations: both were written before
// the counter existed (both 0), or there is only one copy. In both, the copy
// that survives a reboot is the one to keep. Nothing else may resolve a tie,
// and in particular not the clock — see the subCacheFile doc comment.
func authoritativeSubCache(cands []subCacheCandidate) *subCacheCandidate {
var best *subCacheCandidate
for i := range cands {
c := &cands[i]
if best == nil || c.file.Seq > best.file.Seq {
best = c
}
}
return best
}
// nextSubCacheSeq is the generation number a new write must carry: one more than
// the highest already on disk anywhere. Monotonic without a clock, and immune to
// a failed cleanup — a superseded copy simply keeps a lower number for ever.
func nextSubCacheSeq(cands []subCacheCandidate) int {
high := 0
for i := range cands {
if s := cands[i].file.Seq; s > high {
high = s
}
}
return high + 1
}
// SaveSubCache atomically persists the node set of one subscription: temp file
// + rename in the persistent dir, degrading to tmpfs with a warning when the
// overlay refuses the write. Writing the byte-identical content is a no-op, so
// a refresh that changed nothing costs no overlay churn.
// overlay refuses the write. The new copy carries the next generation number, so
// it outranks whatever it could not replace.
//
// A refresh that changes nothing costs no write at all — but only when the copy
// in force is ALREADY the persistent one. If a tmpfs copy currently outranks it
// (the overlay was full last time and has since recovered), the identical node
// set is written to the persistent home anyway, with a higher generation, so the
// authoritative copy moves back to the home that survives a reboot. That is the
// one case where "nothing changed" must still cost a write, and skipping it is
// how the cache would stay one reboot away from serving a stale set.
func SaveSubCache(sub string, nodes []Node) error {
if strings.TrimSpace(sub) == "" {
return fmt.Errorf("subs cache: empty subscription name")
}
data, err := marshalSubCache(sub, nodes)
cands := subCacheCandidatesFor(sub)
if best := authoritativeSubCache(cands); best != nil &&
best.dirIndex == 0 && sameNodeSet(best.file.Nodes, nodes) {
return nil // unchanged, and already in force from the persistent home
}
seq := nextSubCacheSeq(cands)
data, err := marshalSubCache(sub, seq, nodes)
if err != nil {
return fmt.Errorf("subs cache %q: marshal: %w", sub, err)
}
@@ -144,9 +266,6 @@ func SaveSubCache(sub string, nodes []Node) error {
var firstErr error
for i, dir := range dirs {
path := filepath.Join(dir, name)
if existing, rerr := os.ReadFile(path); rerr == nil && bytes.Equal(existing, data) {
return nil // unchanged — skip the write entirely
}
if werr := writeFileAtomic(dir, path, data); werr != nil {
if firstErr == nil {
firstErr = werr
@@ -155,19 +274,64 @@ func SaveSubCache(sub string, nodes []Node) error {
}
if i > 0 {
// Same degradation policy as generate/cache.go: tmpfs keeps things
// working, but the cache is gone after a reboot (one re-fetch).
// working, but the cache is gone after a reboot (one re-fetch). The
// generation counter is what makes the daemon actually READ this copy
// while it is the newest one.
subCacheLogf("%s is not writable (%v); cached %q to tmpfs %s — the cache will not survive a reboot", dirs[0], firstErr, sub, path)
} else {
// The persistent home took the write, so any lower-generation copy left
// in tmpfs is dead weight. Best-effort ONLY: the generation counter has
// already decided the outcome, so a removal that fails costs nothing but
// a few KB — which is why this is not allowed to fail the save.
dropSupersededSubCaches(cands, path)
}
return nil
}
return fmt.Errorf("subs cache %q: %w", sub, firstErr)
}
// writeFileAtomic writes data to path via a temp file + rename in the same
// dropSupersededSubCaches removes the copies a successful write to the primary
// home has just outranked. keep is the path just written.
func dropSupersededSubCaches(cands []subCacheCandidate, keep string) {
for i := range cands {
if cands[i].path == keep {
continue
}
_ = os.Remove(cands[i].path)
}
}
// sameNodeSet reports whether two node sets are equal for caching purposes,
// treating nil and empty as the same thing (marshalSubCache encodes both as []).
func sameNodeSet(a, b []Node) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
// writeFileAtomic is the one write in this file, behind a seam.
//
// The failure that decides everything about the cache — "the filesystem took the
// bytes for one home and refused them for the other" — is precisely the one a
// temp directory cannot be made to produce on demand, and it is the only way to
// prove the degraded path both works AND is read back. Same reasoning, and the
// same shape, as migrate.go's statBackup/writeBackupFile. Nothing else may
// replace it.
var writeFileAtomic = writeFileAtomicOS
// writeFileAtomicOS writes data to path via a temp file + rename in the same
// directory, creating the directory first. Rename is atomic on the same
// filesystem, so a concurrent reader sees either the old file or the new one,
// never a torn write.
func writeFileAtomic(dir, path string, data []byte) error {
// never a torn write — and a write that fails at any step leaves the PREVIOUS
// cache exactly as it was, which is what makes a failed refresh cost a stale node
// set rather than an empty one.
func writeFileAtomicOS(dir, path string, data []byte) error {
if err := os.MkdirAll(dir, 0o755); err != nil {
return err
}
@@ -197,28 +361,39 @@ func writeFileAtomic(dir, path string, data []byte) error {
// the daemon must come up on whatever it finds, and the next refresh rewrites
// the file anyway.
func LoadSubCache(sub string) (nodes []Node, ok bool, err error) {
name := subCacheFileName(sub)
for _, dir := range subCacheDirs() {
path := filepath.Join(dir, name)
data, rerr := os.ReadFile(path)
if rerr != nil {
continue
}
f, derr := decodeSubCache(data)
if derr != nil {
subCacheLogf("%s: %v — ignoring the file (the next refresh rewrites it)", path, derr)
continue
}
return f.Nodes, true, nil
best := authoritativeSubCache(subCacheCandidatesFor(sub))
if best == nil {
return nil, false, nil
}
return nil, false, nil
return best.file.Nodes, true, nil
}
// LoadAllSubCaches reads every cache file from the candidate dirs into a
// sub-name -> nodes map. The persistent dir shadows tmpfs for the same
// subscription. Undecodable files are reported and skipped.
// sub-name -> nodes map. Undecodable files are reported and skipped.
//
// WHEN THE SAME SUBSCRIPTION IS IN BOTH DIRS, THE HIGHEST GENERATION WINS — not
// the persistent one, and not the newer modification time. The rule used to be
// "the earlier (higher-priority) dir wins", and it had a hole with real
// consequences: SaveSubCache degrades to tmpfs exactly when the persistent home
// REFUSED the write (a full or read-only overlay, which is a routine OpenWrt
// state), so at that moment the persistent copy is by definition the stale one.
// Under the old rule a successful `sub update` reported "N nodes cached", wrote
// them to tmpfs, and the daemon went on serving the previous node set from /etc —
// with nothing anywhere saying the refresh had not taken effect. That is the
// failure this codebase keeps paying for: an instrument that reads the same in
// the working and the broken case.
//
// The first attempt at a fix compared modification times, and it was wrong on
// this platform for two independent reasons — identical timestamps within one
// kernel tick, and a router with no RTC. See the subCacheFile doc comment; the
// generation counter is the replacement.
//
// A reboot needs no special case: tmpfs is empty then, so the persistent copy is
// the only candidate and wins by default — the cold-start-on-cache guarantee is
// unchanged. This is the call ReadUCI makes, so it is the one the daemon boots on.
func LoadAllSubCaches() map[string][]Node {
out := map[string][]Node{}
seq := map[string]int{}
for _, dir := range subCacheDirs() {
entries, err := os.ReadDir(dir)
if err != nil {
@@ -243,10 +418,15 @@ func LoadAllSubCaches() map[string][]Node {
subCacheLogf("%s: missing \"sub\" field — skipped", path)
continue
}
if _, dup := out[f.Sub]; dup {
continue // earlier (higher-priority) dir wins
// Same rule as authoritativeSubCache, and it has to STAY the same: a
// strict > keeps the earlier directory on a tie, so a legacy pair
// (both generation 0) resolves to the persistent copy exactly as it
// did before the counter existed.
if prev, dup := seq[f.Sub]; dup && f.Seq <= prev {
continue
}
out[f.Sub] = f.Nodes
seq[f.Sub] = f.Seq
}
}
return out
+268
View File
@@ -0,0 +1,268 @@
package model
// Which of two copies of the same subscription cache is IN FORCE.
//
// This is a separate file because the rule has already been got wrong once, in a
// way that a Windows test run reported as green: the first fix compared file
// modification times, and it failed on Linux 5 runs out of 5. The guard against
// repeating that is TestAuthorityIgnoresTheClock, and everything here exists to
// keep the answer a property of the DATA rather than of the filesystem.
import (
"os"
"path/filepath"
"reflect"
"testing"
"time"
)
// TestAuthorityIgnoresTheClock is the regression guard for the fix that did NOT
// work, and it is deliberately hostile: it hands the STALE copy every advantage a
// clock could give it and requires the fresh one to win anyway.
//
// The measurement that condemned modification time, taken in the Linux CI
// container, was that two writes in one operation get BIT-IDENTICAL timestamps —
// both files came out at the same UnixNano to the digit, so a strict "is newer
// than" was false and the first-visited (persistent, stale) copy won. That is not
// a rare tie; two small writes always land in one kernel timer tick, so on Linux
// it was the normal outcome.
//
// The second reason is independent of resolution and cannot be tuned away: this
// router has no RTC. It boots with the clock at the epoch until NTP syncs, over
// the very SIM uplink that drops. A cache written before a sync carries a 1970
// stamp and would lose to an older file written after a previous sync, so the
// ordering can invert across a reboot.
//
// So this test sets the stale file's mtime to the FUTURE and the fresh file's to
// the epoch. Any implementation that consults the clock — coarse or fine, strict
// or not — reads the stale set here and fails, on every platform.
func TestAuthorityIgnoresTheClock(t *testing.T) {
persistent, fallback := twoDirCache(t)
quietCacheLog(t)
stale := []Node{{Name: "old-01", Enabled: true, URI: "vless://old", FromSub: "qomar"}}
if err := SaveSubCache("qomar", stale); err != nil {
t.Fatal(err)
}
failWritesTo(t, persistent, errRefused{})
fresh := []Node{{Name: "new-01", Enabled: true, URI: "vless://new", FromSub: "qomar"}}
if err := SaveSubCache("qomar", fresh); err != nil {
t.Fatal(err)
}
stalePath := filepath.Join(persistent, subCacheFileName("qomar"))
freshPath := filepath.Join(fallback, subCacheFileName("qomar"))
future := time.Now().Add(72 * time.Hour)
past := time.Unix(0, 0)
if err := os.Chtimes(stalePath, future, future); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(freshPath, past, past); err != nil {
t.Fatal(err)
}
// CONTROL: the timestamps really are inverted, so an implementation that read
// them would demonstrably get the wrong answer here.
si, err := os.Stat(stalePath)
if err != nil {
t.Fatal(err)
}
fi, err := os.Stat(freshPath)
if err != nil {
t.Fatal(err)
}
if !si.ModTime().After(fi.ModTime()) {
t.Fatalf("CONTROL FAILED: the stale copy is not newer by the clock (%v vs %v), so this test "+
"does not actually punish a clock-based rule", si.ModTime(), fi.ModTime())
}
got, ok, _ := LoadSubCache("qomar")
if !ok || !reflect.DeepEqual(got, fresh) {
t.Fatalf("LoadSubCache returned %+v, want the FRESH set %+v — the copy in force is being "+
"chosen by the filesystem clock, which on this platform is neither monotonic nor "+
"fine-grained enough to order two writes", got, fresh)
}
if all := LoadAllSubCaches(); !reflect.DeepEqual(all["qomar"], fresh) {
t.Fatalf("LoadAllSubCaches returned %+v, want the FRESH set %+v — this is the call ReadUCI "+
"makes, so it is what the daemon boots on", all["qomar"], fresh)
}
}
// TestGenerationIsMonotonicAcrossHomes: every write claims a number higher than
// anything on disk anywhere, so a copy that could not be replaced is outranked
// for ever — and a cleanup that fails can never resurrect it.
func TestGenerationIsMonotonicAcrossHomes(t *testing.T) {
persistent, fallback := twoDirCache(t)
quietCacheLog(t)
seqAt := func(dir string) int {
t.Helper()
data, err := os.ReadFile(filepath.Join(dir, subCacheFileName("qomar")))
if err != nil {
return -1
}
f, derr := decodeSubCache(data)
if derr != nil {
t.Fatalf("decode %s: %v", dir, derr)
}
return f.Seq
}
if err := SaveSubCache("qomar", []Node{{Name: "a", URI: "ss://a", FromSub: "qomar"}}); err != nil {
t.Fatal(err)
}
first := seqAt(persistent)
if first < 1 {
t.Fatalf("the first write must carry a generation of at least 1, got %d", first)
}
// A degraded write must outrank the persistent copy it could not replace.
failWritesTo(t, persistent, errRefused{})
if err := SaveSubCache("qomar", []Node{{Name: "b", URI: "ss://b", FromSub: "qomar"}}); err != nil {
t.Fatal(err)
}
if degraded := seqAt(fallback); degraded <= first {
t.Fatalf("the tmpfs copy carries generation %d and the persistent one %d — a copy that "+
"supersedes another must say so with a HIGHER number, or the reader cannot tell",
degraded, first)
}
}
// TestAuthorityReturnsToThePersistentHome: once the overlay recovers, the next
// refresh must put the authoritative copy back where a reboot can find it — even
// when the node set has not changed, which is exactly the case an unconditional
// "skip if identical" would swallow. Without this the router stays permanently
// one reboot away from serving a stale set.
func TestAuthorityReturnsToThePersistentHome(t *testing.T) {
persistent, fallback := twoDirCache(t)
quietCacheLog(t)
if err := SaveSubCache("qomar", []Node{{Name: "old-01", URI: "vless://old", FromSub: "qomar"}}); err != nil {
t.Fatal(err)
}
// The overlay is full: the refresh lands in tmpfs and takes authority.
failWritesTo(t, persistent, errRefused{})
fresh := []Node{{Name: "new-01", URI: "vless://new", FromSub: "qomar"}}
if err := SaveSubCache("qomar", fresh); err != nil {
t.Fatal(err)
}
// The overlay recovers. The node set is UNCHANGED — the provider returned the
// same feed — so a plain "nothing changed, skip" would write nothing at all.
writeFileAtomic = writeFileAtomicOS
if err := SaveSubCache("qomar", fresh); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(filepath.Join(persistent, subCacheFileName("qomar")))
if err != nil {
t.Fatalf("the persistent home was never rewritten, so a reboot still serves the stale "+
"set: %v", err)
}
f, derr := decodeSubCache(data)
if derr != nil {
t.Fatal(derr)
}
if !reflect.DeepEqual(f.Nodes, fresh) {
t.Fatalf("the persistent copy holds %+v, want the fresh set %+v", f.Nodes, fresh)
}
// The reboot: tmpfs is gone and the fresh set still comes up.
if err := os.RemoveAll(fallback); err != nil {
t.Fatal(err)
}
got, ok, _ := LoadSubCache("qomar")
if !ok || !reflect.DeepEqual(got, fresh) {
t.Fatalf("after the overlay recovered and the router rebooted, the cache serves %+v, "+
"want %+v", got, fresh)
}
}
// TestLegacyPairTiesToThePersistentCopy: two cache files written before the
// generation counter existed both read as generation 0. That tie must resolve
// exactly the way it did before the counter — to the persistent home — so an
// upgrade changes nothing for a router that never degraded to tmpfs, and the
// counter's absence is a legitimate value rather than a special case.
func TestLegacyPairTiesToThePersistentCopy(t *testing.T) {
persistent, fallback := twoDirCache(t)
quietCacheLog(t)
// Both writes are spelled out against the temp directories rather than routed
// through a helper that takes the directory as a PARAMETER. That is not style:
// shater/testguard reads this source and requires every mutating call's path to
// be PROVABLY rooted in t.TempDir(), and it cannot follow a value handed to a
// closure. A path it cannot prove is a failure, not a default — the check exists
// because a test once deleted the router's real stats database and passed.
if err := os.MkdirAll(persistent, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(persistent, subCacheFileName("qomar")),
[]byte(legacyCacheBody("from-etc")), 0o600); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(fallback, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(fallback, subCacheFileName("qomar")),
[]byte(legacyCacheBody("from-tmp")), 0o600); err != nil {
t.Fatal(err)
}
got, ok, _ := LoadSubCache("qomar")
if !ok || len(got) != 1 || got[0].Name != "from-etc" {
t.Fatalf("a legacy pair resolved to %+v, want the persistent copy (from-etc): a file with "+
"no generation must behave exactly as it did before the counter existed", got)
}
if all := LoadAllSubCaches(); len(all["qomar"]) != 1 || all["qomar"][0].Name != "from-etc" {
t.Fatalf("LoadAllSubCaches resolved the legacy pair to %+v, want the persistent copy",
all["qomar"])
}
}
// TestLegacyFileIsSupersededByANewWrite: the upgrade path. A router whose cache
// predates the counter has generation-0 files; the first refresh after the
// upgrade must produce a copy that outranks them, in either home.
func TestLegacyFileIsSupersededByANewWrite(t *testing.T) {
persistent, _ := twoDirCache(t)
quietCacheLog(t)
if err := os.MkdirAll(persistent, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(persistent, subCacheFileName("qomar")),
[]byte(legacyCacheBody("legacy")), 0o600); err != nil {
t.Fatal(err)
}
// A legacy file must still READ (no version bump, or the router boots empty).
if got, ok, _ := LoadSubCache("qomar"); !ok || len(got) != 1 || got[0].Name != "legacy" {
t.Fatalf("a pre-counter cache file no longer reads: ok=%v %+v — an upgrade must not empty "+
"the node inventory", ok, got)
}
fresh := []Node{{Name: "new-01", URI: "vless://new", FromSub: "qomar"}}
if err := SaveSubCache("qomar", fresh); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(filepath.Join(persistent, subCacheFileName("qomar")))
if err != nil {
t.Fatal(err)
}
f, derr := decodeSubCache(data)
if derr != nil {
t.Fatal(derr)
}
if f.Seq < 1 {
t.Fatalf("the write after a legacy file carries generation %d; it must claim a number the "+
"legacy file cannot match", f.Seq)
}
if got, _, _ := LoadSubCache("qomar"); !reflect.DeepEqual(got, fresh) {
t.Fatalf("after the first post-upgrade refresh the cache still serves %+v, want %+v", got, fresh)
}
}
// legacyCacheBody renders a cache file as it looked BEFORE the generation counter
// existed: no "seq" key at all, so decodeSubCache reads generation 0. It returns
// only the BYTES — never a path — so the callers can spell their own temp
// directory out at the call site, which is what shater/testguard requires.
func legacyCacheBody(nodeName string) string {
return `{"version":1,"sub":"qomar","nodes":[{"Name":"` + nodeName + `","URI":"ss://x"}]}`
}
+376
View File
@@ -0,0 +1,376 @@
package model
// The subscription cache as the DEFAULT SOURCE, not as a consolation prize.
//
// The product boots off this cache: /etc/init.d/shater starts shaterd, which runs
// Migrate + ReadUCI (= ParseUCIExport + MergeSubCaches) and applies the result.
// No fetch happens on that path at all — the refresh is a separate cron job. So
// "cache first, refresh later" is not a fallback the daemon reaches for when the
// network fails; it is the only way the node inventory ever enters the process,
// and a boot with no network is the ORDINARY case, not the exceptional one.
//
// What these tests hold down is the two halves of that promise:
//
// 1. a cold start with nothing but the disk produces the full node inventory;
// 2. a write that FAILS leaves the previous inventory exactly as it was, and a
// write that had to degrade to tmpfs is actually the one that gets read back.
//
// Half 2 needs the degraded two-directory state, which SHATER_SUBS_DIR cannot
// produce (it collapses the chain to one dir) and which a temp directory cannot
// produce on demand — hence the writeFileAtomic seam. Every test below that uses
// it carries its own POSITIVE CONTROL: the same instrument, unbroken, is shown to
// give the other answer, because a "no damage" result from an instrument that
// could not have seen damage is not a result.
import (
"os"
"path/filepath"
"reflect"
"strings"
"testing"
)
// twoDirCache points the persistent/tmpfs chain at two temp directories and
// clears SHATER_SUBS_DIR, so subCacheDirs() returns the real two-element chain.
func twoDirCache(t *testing.T) (persistent, fallback string) {
t.Helper()
t.Setenv("SHATER_SUBS_DIR", "")
persistent, fallback = t.TempDir(), t.TempDir()
oldP, oldF := subCacheDirPersistent, subCacheDirFallback
subCacheDirPersistent, subCacheDirFallback = persistent, fallback
t.Cleanup(func() { subCacheDirPersistent, subCacheDirFallback = oldP, oldF })
return persistent, fallback
}
// failWritesTo makes writeFileAtomic refuse every write into dir and pass the
// rest through. Returns a pointer to the call count so a test can prove the
// instrument was actually exercised.
func failWritesTo(t *testing.T, dir string, err error) *int {
t.Helper()
calls := 0
old := writeFileAtomic
writeFileAtomic = func(d, path string, data []byte) error {
calls++
if d == dir {
return err
}
return old(d, path, data)
}
t.Cleanup(func() { writeFileAtomic = old })
return &calls
}
// quietCacheLog silences the degradation notices so a deliberate failure does not
// print pages of noise, and returns the messages for inspection.
func quietCacheLog(t *testing.T) *[]string {
t.Helper()
var got []string
old := subCacheLogf
subCacheLogf = func(f string, a ...any) { got = append(got, f) }
t.Cleanup(func() { subCacheLogf = old })
return &got
}
// --- half 1: the cold start --------------------------------------------------
// TestColdStartServesTheWholeInventoryFromDisk: with no network and nothing but
// /etc/config/shater plus the cache files, a read produces every subscription
// node — in configuration order, with FromSub restored — alongside the manual
// nodes. This is what a reboot on a dead uplink actually does.
func TestColdStartServesTheWholeInventoryFromDisk(t *testing.T) {
setSubsDir(t)
if err := SaveSubCache("qomar", []Node{
{Name: "nl-01", Enabled: true, URI: "vless://nl", FromSub: "qomar"},
{Name: "de-02", Enabled: false, URI: "vless://de", FromSub: "qomar"},
}); err != nil {
t.Fatal(err)
}
if err := SaveSubCache("second", []Node{
{Name: "jp-01", Enabled: true, URI: "vless://jp", FromSub: "second"},
}); err != nil {
t.Fatal(err)
}
// The config as it is on disk: subscriptions and a manual node, no `config
// node` for anything the feed supplied (RenderUCIExport does not emit those).
m, err := ParseUCIExport(`package shater
config subscription
option name 'qomar'
option url 'https://p.example/q'
config subscription
option name 'second'
option url 'https://p.example/s'
config node
option name 'manual-wg'
option uri 'wireguard://home'
`)
if err != nil {
t.Fatalf("parse: %v", err)
}
// ReadUCI is ParseUCIExport + this call; the merge is the whole cold start.
MergeSubCaches(m)
var names []string
for _, n := range m.Nodes {
names = append(names, n.Name)
}
want := []string{"manual-wg", "nl-01", "de-02", "jp-01"}
if !reflect.DeepEqual(names, want) {
t.Fatalf("cold start produced %v, want %v — a reboot with no network must come up with the "+
"whole inventory, not with the manual nodes only", names, want)
}
// The per-node state the operator set must survive too: a node parked by hand
// that comes back enabled after a reboot puts traffic on it again.
for _, n := range m.Nodes {
if n.Name == "de-02" && n.Enabled {
t.Fatal("a node switched OFF by hand came back enabled from the cache")
}
if n.Name == "nl-01" && n.FromSub != "qomar" {
t.Fatalf("FromSub not restored from the file's own key: %+v", n)
}
}
}
// --- half 2: a failed write may not cost the inventory -----------------------
// TestFailedWriteLeavesThePreviousCacheIntact: when every home refuses the bytes,
// SaveSubCache reports the failure and the node set already on disk is untouched.
// A refresh that cannot be persisted must cost a stale inventory, never an empty
// one — the daemon boots off this file.
//
// POSITIVE CONTROL: the same instrument with the failure removed IS shown to
// replace the file, so "unchanged" here means the write was stopped and not that
// the test could never have observed a change.
func TestFailedWriteLeavesThePreviousCacheIntact(t *testing.T) {
persistent, fallback := twoDirCache(t)
quietCacheLog(t)
good := []Node{{Name: "nl-01", Enabled: true, URI: "vless://nl", FromSub: "qomar"}}
if err := SaveSubCache("qomar", good); err != nil {
t.Fatalf("seed: %v", err)
}
// CONTROL: with the instrument unbroken, a second save DOES change the file.
replacement := []Node{{Name: "jp-09", Enabled: true, URI: "vless://jp", FromSub: "qomar"}}
if err := SaveSubCache("qomar", replacement); err != nil {
t.Fatalf("control save: %v", err)
}
if got, _, _ := LoadSubCache("qomar"); len(got) != 1 || got[0].Name != "jp-09" {
t.Fatalf("CONTROL FAILED: a good save did not replace the cache (%+v) — this test could not "+
"have detected a clobber either", got)
}
// Put the known-good set back and make every home refuse.
if err := SaveSubCache("qomar", good); err != nil {
t.Fatalf("re-seed: %v", err)
}
callsP := failWritesTo(t, persistent, errRefused{})
_ = callsP
failWritesTo(t, fallback, errRefused{}) // stacked: both homes now refuse
if err := SaveSubCache("qomar", replacement); err == nil {
t.Fatal("SaveSubCache reported success while every home refused the write")
}
got, ok, _ := LoadSubCache("qomar")
if !ok {
t.Fatal("the previous cache is GONE after a failed write — the daemon now boots with no nodes")
}
if !reflect.DeepEqual(got, good) {
t.Fatalf("the previous cache was damaged by a failed write: %+v, want %+v", got, good)
}
}
// errRefused stands in for the filesystem declining the bytes (ENOSPC/EROFS on a
// full or read-only overlay — the routine OpenWrt state this path exists for).
type errRefused struct{}
func (errRefused) Error() string { return "no space left on device" }
// TestDegradedWriteIsTheOneReadBack is the defect this file was written for.
//
// SaveSubCache degrades to tmpfs precisely when the persistent home REFUSED the
// write, so at that instant the persistent copy is by definition the stale one.
// The load order used to be "the earlier (higher-priority) dir wins", which meant
// a successful `sub update` wrote the fresh nodes to /tmp, reported "N nodes
// cached", and every reader went on serving the OLD set out of /etc — an
// instrument reading identically in the working and the broken case.
//
// The winner is decided by the generation counter in the file. The first attempt
// compared modification times and this test FAILED on Linux 5 runs out of 5 while
// passing on Windows — see TestAuthorityIgnoresTheClock, which is the guard that
// makes that mistake unrepeatable.
//
// POSITIVE CONTROL: the stale persistent file is shown to be present and
// readable, so "the fresh set was returned" is a choice between two real
// candidates and not the absence of one.
func TestDegradedWriteIsTheOneReadBack(t *testing.T) {
persistent, fallback := twoDirCache(t)
quietCacheLog(t)
stale := []Node{{Name: "old-01", Enabled: true, URI: "vless://old", FromSub: "qomar"}}
if err := SaveSubCache("qomar", stale); err != nil {
t.Fatalf("seed the persistent home: %v", err)
}
if _, err := statFile(filepath.Join(persistent, subCacheFileName("qomar"))); err != nil {
t.Fatalf("seed did not land in the persistent home: %v", err)
}
// The overlay fills up; the refresh succeeds over the network and degrades.
failWritesTo(t, persistent, errRefused{})
fresh := []Node{
{Name: "new-01", Enabled: true, URI: "vless://new1", FromSub: "qomar"},
{Name: "new-02", Enabled: true, URI: "vless://new2", FromSub: "qomar"},
}
if err := SaveSubCache("qomar", fresh); err != nil {
t.Fatalf("degraded save should still succeed via tmpfs: %v", err)
}
// CONTROL: both candidates exist. The choice below is a real choice.
if _, err := statFile(filepath.Join(persistent, subCacheFileName("qomar"))); err != nil {
t.Fatalf("CONTROL FAILED: the stale persistent copy is gone, so preferring the fresh one "+
"proves nothing: %v", err)
}
if _, err := statFile(filepath.Join(fallback, subCacheFileName("qomar"))); err != nil {
t.Fatalf("the degraded write did not reach tmpfs: %v", err)
}
got, ok, _ := LoadSubCache("qomar")
if !ok || !reflect.DeepEqual(got, fresh) {
t.Fatalf("LoadSubCache returned %+v, want the FRESH set %+v — a refresh that reported "+
"success is not in force", got, fresh)
}
all := LoadAllSubCaches()
if !reflect.DeepEqual(all["qomar"], fresh) {
t.Fatalf("LoadAllSubCaches returned %+v, want the FRESH set %+v (this is the call ReadUCI "+
"makes, so it is the one the daemon actually uses)", all["qomar"], fresh)
}
}
// TestPersistentCopySurvivesTheRebootAfterADegradedWrite is the other side of the
// same rule, and the reason it is "newest wins" rather than "tmpfs wins": once
// tmpfs is gone (a reboot), the persistent copy is the only candidate and must
// still come up. Stale nodes beat no nodes.
func TestPersistentCopySurvivesTheRebootAfterADegradedWrite(t *testing.T) {
_, fallback := twoDirCache(t)
quietCacheLog(t)
stale := []Node{{Name: "old-01", Enabled: true, URI: "vless://old", FromSub: "qomar"}}
if err := SaveSubCache("qomar", stale); err != nil {
t.Fatal(err)
}
// The reboot: tmpfs is empty again.
if err := removeAllFiles(fallback); err != nil {
t.Fatal(err)
}
got, ok, _ := LoadSubCache("qomar")
if !ok || !reflect.DeepEqual(got, stale) {
t.Fatalf("after a reboot the persistent cache must still come up: ok=%v %+v", ok, got)
}
}
// TestSyncSubCachesRewritesToTheModelItIsGiven pins the panel-PUT contract in the
// direction that can LOSE nodes, so any future change to it is deliberate.
//
// SyncSubCaches makes the cache files equal to the FromSub nodes carried in the
// model — including emptying one whose nodes are all gone, which is what makes
// "delete the last node" stick. The consequence is that a caller which builds a
// model WITHOUT MergeSubCaches (ParseUCIExport alone: no FromSub nodes at all)
// and hands it here empties every cache the router has. The panel never does this
// — it PUTs back the merged model it GETs — but nothing in this package enforces
// that, and the failure would be silent and total.
func TestSyncSubCachesRewritesToTheModelItIsGiven(t *testing.T) {
setSubsDir(t)
if err := SaveSubCache("qomar", []Node{{Name: "nl-01", URI: "vless://nl", FromSub: "qomar"}}); err != nil {
t.Fatal(err)
}
unmerged := &Model{Subscriptions: []Subscription{{Name: "qomar", Enabled: true}}}
if err := SyncSubCaches(unmerged); err != nil {
t.Fatalf("sync: %v", err)
}
nodes, ok, _ := LoadSubCache("qomar")
if !ok || len(nodes) != 0 {
t.Fatalf("SyncSubCaches(model without FromSub nodes) left %+v (ok=%v). If this now KEEPS the "+
"nodes the behaviour changed deliberately — update this test and the comment above it, "+
"because \"delete the last node\" depends on the emptying half", nodes, ok)
}
}
// statFile / removeAllFiles are the two filesystem pokes these tests need; they
// live here rather than in subcache.go because nothing in the product does them.
func statFile(path string) (os.FileInfo, error) { return os.Stat(path) }
func removeAllFiles(dir string) error {
entries, err := os.ReadDir(dir)
if err != nil {
return err
}
for _, e := range entries {
if err := os.Remove(filepath.Join(dir, e.Name())); err != nil {
return err
}
}
return nil
}
// TestWriteIsTempThenRename pins the STRUCTURE of the real writer: it builds the
// new file beside the destination and moves it into place, so the destination is
// never opened for writing and a failure leaves it as it was — and it cleans up
// after itself when the move cannot happen.
//
// What this test can and cannot see is worth stating. There is no portable way to
// make a filesystem answer ENOSPC on demand, so the behaviour of this function
// under a full overlay is NOT exercised here; that is the whole reason
// writeFileAtomic is a seam and why the guarantee under test in
// TestFailedWriteLeavesThePreviousCacheIntact is the caller-level one. What IS
// exercised is the part a black-box test can reach: a move that cannot succeed
// costs neither the destination nor a stray temp file in the cache directory,
// where LoadAllSubCaches would later have to skip it by name.
func TestWriteIsTempThenRename(t *testing.T) {
dir := t.TempDir()
if err := writeFileAtomicOS(dir, filepath.Join(dir, "ok.json"), []byte("one")); err != nil {
t.Fatalf("plain write: %v", err)
}
if n := tempLitter(t, dir); n != 0 {
t.Fatalf("a SUCCESSFUL write left %d temp file(s) behind in the cache directory", n)
}
// A destination the move cannot possibly take: a non-empty directory. Rename
// onto one fails on every platform this ships to.
blocked := filepath.Join(dir, "blocked.json")
if err := os.MkdirAll(filepath.Join(blocked, "inside"), 0o755); err != nil {
t.Fatal(err)
}
if err := writeFileAtomicOS(dir, blocked, []byte("two")); err == nil {
t.Fatal("writeFileAtomicOS reported success onto a destination it cannot replace")
}
fi, err := os.Stat(blocked)
if err != nil || !fi.IsDir() {
t.Fatalf("the destination was disturbed by a write that failed: %v %v", fi, err)
}
if _, err := os.Stat(filepath.Join(blocked, "inside")); err != nil {
t.Fatalf("the destination's contents were destroyed by a write that failed: %v", err)
}
if n := tempLitter(t, dir); n != 0 {
t.Fatalf("a FAILED write left %d temp file(s) in the cache directory; they accumulate on "+
"every retry of a subscription whose refresh cannot be persisted", n)
}
}
// tempLitter counts leftover .subcache-*.tmp files in dir.
func tempLitter(t *testing.T, dir string) int {
t.Helper()
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatal(err)
}
n := 0
for _, e := range entries {
if strings.HasPrefix(e.Name(), ".subcache-") {
n++
}
}
return n
}
+25 -5
View File
@@ -71,11 +71,16 @@ func ParseUCIExport(text string) (*Model, error) {
})
case "subscription":
m.Subscriptions = append(m.Subscriptions, Subscription{
Name: firstNonEmpty(s.opt("name"), s.Name),
Enabled: s.optBool("enabled", true),
URL: s.opt("url"),
UpdateInterval: s.opt("update_interval"),
FetchVia: s.optOr("fetch_via", "direct"),
Name: firstNonEmpty(s.opt("name"), s.Name),
Enabled: s.optBool("enabled", true),
URL: s.opt("url"),
UpdateInterval: s.opt("update_interval"),
// s.opt, NOT s.optOr(..., "direct"): an ABSENT `option fetch_via`
// must stay absent in the model, because "" now means "inherit the
// general fetch detour" and `direct` means "explicitly in the clear".
// Defaulting here is what made the two indistinguishable; the v2->v3
// migration is what converts the existing absences, once, on disk.
FetchVia: s.opt("fetch_via"),
FetchDetour: s.opt("fetch_detour"),
UA: s.opt("ua"),
HWID: s.opt("hwid"),
@@ -232,6 +237,15 @@ func ParseUCIExport(text string) (*Model, error) {
EnableRules: nonEmpty(s.list("enable_rule")),
DisableRules: nonEmpty(s.list("disable_rule")),
EndpointResolver: s.opt("endpoint_resolver"),
// Per-profile DNS/fetch overrides. All three are SCALARS and are read
// with s.opt (never s.list): one profile names one resolver and one
// detour. "" (absent) = inherit from globals, per field — so a profile
// that sets only resolver_fallback keeps globals' resolver_default.
// There is deliberately no default here: optOr would turn "not set"
// into a value and destroy the inheritance the fields exist for.
ResolverDefault: s.opt("resolver_default"),
ResolverFallback: s.opt("resolver_fallback"),
FetchDetour: s.opt("fetch_detour"),
})
case "resolver":
m.Resolvers = append(m.Resolvers, Resolver{
@@ -343,6 +357,12 @@ func applyGlobals(g *Globals, s uciSection) {
g.ResolverDefault = s.opt("resolver_default")
g.ResolverFallback = s.opt("resolver_fallback")
g.EndpointResolver = s.opt("endpoint_resolver")
// The GENERAL fetch detour. No default: "" is a state of its own ("nothing is
// configured, fetches go direct as they always did"), and seeding it with
// `direct` would make a config that never mentioned the option indistinguishable
// from one that chose clear-text on purpose — the same mistake `fetch_via`
// carried until v3.
g.FetchDetour = s.opt("fetch_detour")
g.ProbeURL = s.opt("probe_url")
g.ProbeInterval = s.opt("probe_interval")
// No sweep_interval: the background sweep is gone (the observatory probes only
+84
View File
@@ -287,6 +287,12 @@ func ValidateGlobals(g Globals) []Warning {
var out []Warning
add := func(msg string) { out = append(out, Warning{Section: "globals", Message: msg}) }
// The general fetch detour, judged for SHAPE only — whether the named
// node/group/egress/chain exists is the generator's verdict.
if msg := FetchDetourFormWarning("fetch_detour", g.FetchDetour); msg != "" {
add(msg)
}
if lvl := strings.ToLower(strings.TrimSpace(g.LogLevel)); lvl != "" &&
!inSet(lvl, EngineLogLevels) && !inSet(lvl, SilentLogLevels) {
add(fmt.Sprintf("log level %q is not recognised and is applied as \"warn\"; "+
@@ -417,6 +423,84 @@ func ValidateSubscriptions(subs []Subscription) []Warning {
"auto-detected instead; valid values are %s.", s.Format, strings.Join(SubFormatNames, "/")),
})
}
// `fetch_via` has exactly three states and one of them has no spelling
// (absent = inherit the general fetch detour). Anything else is
// FetchViaUnknown, and nothing downstream may pick a side for it: this is
// the branch that names it instead.
//
// Before schema v3 this could not be said at all, because every reader
// asked `EqualFold(v, "proxy")` and a typo fell into the else — so
// `fetch_via='prxoy'` put the feed URL and this router's real address on
// the plain WAN, successfully, with nothing to see anywhere.
if ClassifyFetchVia(s.FetchVia) == FetchViaUnknown {
out = append(out, Warning{
Section: "subscription",
Name: s.Name,
Message: fmt.Sprintf("fetch_via %q is not a value this option has; the accepted "+
"values are %s, and leaving it out means \"use the general fetch_detour\". "+
"Nothing routes this feed until it says one of them: the value is not demoted "+
"to `direct` any more, because that silently put the feed URL and this "+
"router's real address on the plain WAN.",
s.FetchVia, strings.Join(FetchViaNames, "/")),
})
}
// A subscription's own detour, judged for SHAPE only (see
// FetchDetourFormWarning). Only meaningful when the subscription actually
// uses it — with fetch_via=direct or absent the value is inert, and
// apply.subscriptionFetchWarnings already owns the "proxy with no detour"
// finding, which is about content, not shape.
if ClassifyFetchVia(s.FetchVia) == FetchViaProxy {
if msg := FetchDetourFormWarning("fetch_detour", s.FetchDetour); msg != "" {
out = append(out, Warning{Section: "subscription", Name: s.Name, Message: msg})
}
}
}
return out
}
// ValidateProfiles reports per-profile overrides that name something this
// configuration does not have, or that are shaped so they cannot resolve.
//
// A profile override is the one setting whose failure is INVISIBLE by
// construction: it only applies while that profile is active, so a typo in the
// SIM profile's `resolver_default` is not observable at all until the ethernet
// cable comes out — at which point DNS stops working and the config that broke it
// looks exactly like the config that worked. That is why an unresolvable name is
// reported here rather than left to the moment it is used.
//
// resolvers is the configured `config resolver` set. What is checked is
// EXISTENCE BY NAME, which is all model can check; whether the resolver is
// reachable is nobody's compile-time verdict.
func ValidateProfiles(profiles []Profile, resolvers []Resolver) []Warning {
known := make(map[string]bool, len(resolvers))
for _, r := range resolvers {
known[strings.TrimSpace(r.Name)] = true
}
var out []Warning
for _, p := range profiles {
// Positive, closed table: field name -> the value the profile set. Adding a
// fifth override means adding a row here, not remembering to.
for _, f := range []struct{ field, value string }{
{"resolver_default", p.ResolverDefault},
{"resolver_fallback", p.ResolverFallback},
{"endpoint_resolver", p.EndpointResolver},
} {
v := strings.TrimSpace(f.value)
if v == "" || known[v] {
continue // not overridden, or overridden with something that exists
}
out = append(out, Warning{
Section: "profile",
Name: p.Name,
Message: fmt.Sprintf("%s %q names no `config resolver` in this configuration. "+
"The override cannot be honoured, so the globals value stays in force while "+
"this profile is active — and because a profile only applies on its own "+
"uplink, this is invisible until that uplink is the live one.", f.field, v),
})
}
if msg := FetchDetourFormWarning("fetch_detour", p.FetchDetour); msg != "" {
out = append(out, Warning{Section: "profile", Name: p.Name, Message: msg})
}
}
return out
}
+6
View File
@@ -23,7 +23,13 @@ var apiRoutes = []struct {
}{
{"/api/status", []string{http.MethodGet}},
{"/api/config", []string{http.MethodGet, http.MethodPut}},
{"/api/config/effective", []string{http.MethodGet}},
{"/api/apply", []string{http.MethodPost}},
// Added with /api/config/effective: both were live routes missing from this
// checklist, so the auth matrix below had never walked them. The unknown-path
// probe cannot catch that — they are known paths; only this list can.
{"/api/log", []string{http.MethodGet}},
{"/api/rules/reachability", []string{http.MethodGet}},
{"/api/confirm", []string{http.MethodPost}},
{"/api/rollback", []string{http.MethodPost}},
{"/api/stats", []string{http.MethodGet}},
+203
View File
@@ -0,0 +1,203 @@
package panel
// GET /api/config/effective — the four profile-overridable scalars as the ENGINE
// will read them, computed on the daemon.
//
// # Why this exists
//
// The DNS page drew `Globals.EndpointResolver` and called it the answer. On the
// production router that printed `local` while the engine, under the active
// `mobile-uplink` profile, was really using `yandex`. The owner spotted it.
//
// The first fix recomputed the winner IN THE PANEL (panel/src/effective.ts): a
// second copy of "which profile is active" and "whose value wins", written in
// another language and shipped on its own schedule. Two implementations of one
// rule drift, and the drift IS the defect — the panel confidently naming the
// wrong effective value. So the rule stays in exactly one place, model, and the
// daemon answers the question.
//
// # Why a separate endpoint, and not a field on /api/config
//
// /api/config is the DESIRED state and the panel PUTs that same body back
// verbatim. A derived field on it would be decoded by handleConfigPut, whose
// decoder has DisallowUnknownFields() — the round trip would answer 400 — and
// the only way out is for the panel to learn which fields are derived and strip
// them, which is more client-side knowledge of the server's rules, not less.
// The identical argument is already written down for /api/rules/reachability
// (see handleRulesReachability): "a derived verdict has no business travelling
// round-trip through /api/config".
//
// # Why not /api/status either
//
// /api/status is polled every five seconds by every open browser tab and its
// handler is documented to open no socket and start no goroutine, because work
// there is paid once per poll per tab, forever (see statusResponse). This
// answer needs a full config read — a `uci show` shell-out — and is only looked
// at while somebody has the DNS or Profiles page open. It is fetched with the
// config, not with the heartbeat.
//
// # What it does NOT promise
//
// Whether the named resolver or the named detour target EXISTS. That is the
// generator's verdict; this endpoint reports which value is in force, not
// whether it resolves. A profile pointing at a resolver that was deleted still
// shows up here as the value in force, and the operator learns it is broken
// from the generator's critical warning, which is the component that can tell.
import (
"net/http"
"strings"
"github.com/sagernet/sing-box/shater/model"
)
// effectiveOverride is one resolved scalar: what is in force, what is stored,
// and whether a profile made those two differ.
//
// Profile and Overridden are redundant BY CONSTRUCTION — `Profile != ""` is
// exactly `Overridden`, and effectiveConfigTest pins that. The redundancy is
// deliberate: a caller that renders a line whenever Profile is set gets the same
// behaviour as one that checks Overridden, so neither reading can produce the
// noise line this shape exists to prevent.
type effectiveOverride struct {
// Value is what the engine takes. "" is a legitimate answer (nothing set
// anywhere) and not an error.
Value string
// Stored is the value in `config globals` — the one the panel's input edits.
// It is here so the Overridden verdict is auditable against the globals it was
// computed from: a client holding a config it fetched at another moment (or
// carrying unsaved edits) can see WHICH stored value this verdict is about
// instead of assuming it is the one on screen.
Stored string
// Profile names the active profile responsible for the DIFFERENCE, or "".
//
// "" when no profile is active, when the profile set nothing for this field,
// AND when the profile set the very same value the globals carry. The last
// case is the one worth stating: a profile that pins `endpoint_resolver
// 'local'` while globals already say `local` has changed nothing, and naming
// it would give the page a permanent "in effect now: local (profile X)" line
// that reports no news forever.
Profile string
// Overridden is true only when Value differs from Stored. It is the whole
// condition for saying anything at all.
Overridden bool
}
// effectiveConfigResponse is the GET /api/config/effective body.
//
// Field names match the model's (Globals.ResolverDefault etc.) so a reader can
// line each one up with the stored field it is about without a mapping table.
type effectiveConfigResponse struct {
// ActiveProfile is the profile whose overrides apply, or "" when none does.
// Reported independently of whether it overrode anything here: "a profile is
// active and changed none of these" and "no profile is active" are different
// facts, and a page that says "no profile" for the first one is wrong.
ActiveProfile string
ResolverDefault effectiveOverride
ResolverFallback effectiveOverride
EndpointResolver effectiveOverride
FetchDetour effectiveOverride
}
// effectiveConfigRead is handleEffectiveConfig's test seam, the same pattern as
// configGetRead / reachConfigRead: production binds the real UCI read, tests
// substitute a canned model so the handler runs without a `uci` binary.
var effectiveConfigRead = model.ReadUCI
// handleEffectiveConfig → GET /api/config/effective.
func (s *Server) handleEffectiveConfig(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "method not allowed")
return
}
m, err := effectiveConfigRead()
if err != nil {
writeError(w, http.StatusInternalServerError, "read config: "+err.Error())
return
}
writeJSON(w, http.StatusOK, effectiveConfig(m))
}
// effectiveConfig resolves the four scalars for one model.
//
// Profile selection is model.ResolveActiveProfile and nothing else — the SAME
// call generate and netplane make, so all three cannot disagree about which
// profile is running. Its warnings are dropped here for the reason
// handleRulesReachability drops them: this endpoint answers one question, and a
// pin that names a missing profile is already reported to the operator by
// `shaterd status` on every apply, from the component that acts on it.
func effectiveConfig(m *model.Model) effectiveConfigResponse {
prof, _ := model.ResolveActiveProfile(m)
g := m.Globals
out := effectiveConfigResponse{
ResolverDefault: resolved(g.ResolverDefault, model.EffectiveResolverDefault(g, prof), prof, canonResolver),
ResolverFallback: resolved(g.ResolverFallback, model.EffectiveResolverFallback(g, prof), prof, canonResolver),
EndpointResolver: resolved(g.EndpointResolver, model.EffectiveEndpointResolver(g, prof), prof, canonResolver),
FetchDetour: resolved(g.FetchDetour, model.EffectiveFetchDetour(g, prof), prof, canonFetchDetour),
}
if prof != nil {
out.ActiveProfile = prof.Name
}
return out
}
// resolved turns one model.Override into the wire shape, deciding the one thing
// model deliberately does not: whether the winner actually DIFFERS from what is
// stored.
//
// model.Override.From says `profile:<name>` whenever the profile carried a
// non-blank value, which is the right answer to "who supplied this" and the
// wrong answer to "is there anything to tell the operator". Both are needed, and
// the second is computed here, once, so no client has to.
//
// `From` is consulted POSITIVELY: the globals case returns early by name rather
// than the profile case being an `else`. A future third source added to
// model.Override would then be reported as "not overridden by the active
// profile" — which is true — instead of being blamed on whatever profile happens
// to be running.
func resolved(stored string, ov model.Override, prof *model.Profile, canon func(string) string) effectiveOverride {
e := effectiveOverride{
Value: strings.TrimSpace(ov.Value),
Stored: strings.TrimSpace(stored),
}
if ov.From == model.FromGlobals || prof == nil {
return e
}
if canon(e.Value) == canon(e.Stored) {
return e
}
e.Overridden = true
e.Profile = prof.Name
return e
}
// canonResolver normalises a resolver name for the differs-or-not comparison.
// Resolver names are `config resolver` section names and are matched
// case-sensitively by the generator, so this trims and nothing more — folding
// case here would call `Quad9` and `quad9` the same setting when the engine
// will not.
func canonResolver(v string) string { return strings.TrimSpace(v) }
// canonFetchDetour normalises a fetch detour for the same comparison. It knows
// exactly one equivalence: an UNSET detour and an explicit `direct` are the same
// route, so a profile that spells out `fetch_detour 'direct'` over empty globals
// has changed nothing and must not produce an "in effect now" line.
//
// It stops there on purpose. It does NOT resolve a bare name against the live
// node/group catalog, even though the panel's canonDetour does: model is a
// stdlib-only leaf that cannot see emitted tags, and the two bare-name lookup
// orders already disagree (generate.resolveTarget tries NODE then GROUP; the
// panel tries egress, group, chain, node). Encoding either order here would add
// a third answer to a question this endpoint was built to stop having three
// answers to. A bare name compares verbatim, which can only ever over-report a
// difference — the recoverable direction: a visible line about a route that did
// not really change, never silence about one that did.
func canonFetchDetour(v string) string {
v = strings.TrimSpace(v)
if v == "" || strings.EqualFold(v, model.TargetDirect) {
return model.TargetDirect
}
return v
}
+467
View File
@@ -0,0 +1,467 @@
package panel
// Tests for GET /api/config/effective and for the round trip that the four new
// config fields have to survive.
//
// Every assertion here is about a lie the panel could tell. The endpoint exists
// because the DNS page printed `local` from globals while the engine, under the
// active profile, used `yandex`; each test below pins one way that could come
// back — the wrong profile chosen, the stored value reported as effective, or a
// permanent "in effect now" line that reports no news.
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/sagernet/sing-box/shater/model"
)
// getEffective GETs /api/config/effective (authenticated) and returns the status
// code plus the decoded reply.
func getEffective(t *testing.T, srv *httptest.Server, cookie *http.Cookie) (int, effectiveConfigResponse) {
t.Helper()
req, _ := http.NewRequest(http.MethodGet, srv.URL+"/api/config/effective", nil)
req.AddCookie(cookie)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("GET /api/config/effective: %v", err)
}
defer resp.Body.Close()
var out effectiveConfigResponse
_ = json.NewDecoder(resp.Body).Decode(&out)
return resp.StatusCode, out
}
// setEffectiveRead points the handler's read seam at a canned model for the
// duration of the test.
func setEffectiveRead(t *testing.T, m *model.Model) {
t.Helper()
orig := effectiveConfigRead
t.Cleanup(func() { effectiveConfigRead = orig })
effectiveConfigRead = func() (*model.Model, error) { return m, nil }
}
// dnsProfileModel is the production shape that produced the reported defect,
// plus the trap that catches a hand-rolled profile picker.
//
// TWO enabled profiles exist. `mobile-uplink` carries the overrides and is
// selected ONLY because Globals.ActiveProfile pins it — the WAN watcher writes
// that pin on every uplink change. Auto-select would pick `ethernet-uplink`
// instead: it has the higher Priority and (unlike mobile-uplink) no MatchIface
// condition, so it is the one an "obvious" implementation lands on. Any picker
// that is not model.ResolveActiveProfile therefore reports ethernet's values and
// fails every assertion below.
func dnsProfileModel() *model.Model {
return &model.Model{
Globals: model.Globals{
ActiveProfile: "mobile-uplink",
ResolverDefault: "quad9",
ResolverFallback: "cloudflare",
EndpointResolver: "local",
FetchDetour: "direct",
},
Profiles: []model.Profile{
{
Name: "ethernet-uplink", Enabled: true, Priority: 50,
ResolverDefault: "quad9",
ResolverFallback: "cloudflare",
EndpointResolver: "local",
FetchDetour: "direct",
},
{
Name: "mobile-uplink", Enabled: true, Priority: 10,
MatchIface: []string{"wwan0"},
ResolverDefault: "yandex",
ResolverFallback: "yandex-sec",
EndpointResolver: "yandex",
FetchDetour: "group:sim-bypass",
},
},
}
}
// TestEffectiveConfigReportsProfileValues: the reported defect, end to end. The
// active profile overrides all four scalars and the endpoint must name the
// profile's value, the stored one it displaced, and the profile responsible.
func TestEffectiveConfigReportsProfileValues(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
setEffectiveRead(t, dnsProfileModel())
code, out := getEffective(t, srv, cookie)
if code != http.StatusOK {
t.Fatalf("got %d, want 200", code)
}
if out.ActiveProfile != "mobile-uplink" {
t.Fatalf("ActiveProfile = %q, want mobile-uplink (the pin wins over auto-select)", out.ActiveProfile)
}
for _, tc := range []struct {
field string
got effectiveOverride
value, stored string
}{
{"ResolverDefault", out.ResolverDefault, "yandex", "quad9"},
{"ResolverFallback", out.ResolverFallback, "yandex-sec", "cloudflare"},
{"EndpointResolver", out.EndpointResolver, "yandex", "local"},
{"FetchDetour", out.FetchDetour, "group:sim-bypass", "direct"},
} {
if tc.got.Value != tc.value {
t.Errorf("%s.Value = %q, want %q — this is the value the ENGINE uses", tc.field, tc.got.Value, tc.value)
}
if tc.got.Stored != tc.stored {
t.Errorf("%s.Stored = %q, want %q", tc.field, tc.got.Stored, tc.stored)
}
if !tc.got.Overridden {
t.Errorf("%s.Overridden = false, but %q displaced %q", tc.field, tc.value, tc.stored)
}
if tc.got.Profile != "mobile-uplink" {
t.Errorf("%s.Profile = %q, want mobile-uplink", tc.field, tc.got.Profile)
}
}
}
// TestEffectiveConfigSameValueIsNotOverridden is the noise guard, and the reason
// Overridden is computed on the daemon at all.
//
// model.Override.From says `profile:<name>` whenever the profile carried a
// non-blank value — even when that value is identical to globals'. A page keyed
// on From alone would print "in effect now: local (profile ethernet-uplink)"
// underneath an input already reading `local`, forever, on a router where
// nothing is being overridden at all.
func TestEffectiveConfigSameValueIsNotOverridden(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
m := dnsProfileModel()
m.Globals.ActiveProfile = "ethernet-uplink" // the profile that repeats globals verbatim
setEffectiveRead(t, m)
_, out := getEffective(t, srv, cookie)
if out.ActiveProfile != "ethernet-uplink" {
t.Fatalf("ActiveProfile = %q, want ethernet-uplink", out.ActiveProfile)
}
for _, tc := range []struct {
field string
got effectiveOverride
value string
}{
{"ResolverDefault", out.ResolverDefault, "quad9"},
{"ResolverFallback", out.ResolverFallback, "cloudflare"},
{"EndpointResolver", out.EndpointResolver, "local"},
{"FetchDetour", out.FetchDetour, "direct"},
} {
if tc.got.Value != tc.value {
t.Errorf("%s.Value = %q, want %q", tc.field, tc.got.Value, tc.value)
}
if tc.got.Overridden {
t.Errorf("%s: the profile set the SAME value %q as globals — that is not an override "+
"and must not put a permanent line under the input", tc.field, tc.value)
}
if tc.got.Profile != "" {
t.Errorf("%s.Profile = %q, want \"\": a profile that changed nothing is not "+
"responsible for a difference", tc.field, tc.got.Profile)
}
}
// A profile IS active — that fact is reported independently of whether it
// overrode anything. "No profile is active" would be a different, false claim.
if out.ActiveProfile == "" {
t.Error("a profile is active and changed nothing; ActiveProfile must still name it")
}
}
// TestEffectiveConfigUnsetFetchDetourEqualsDirect: an absent fetch detour and an
// explicit `direct` are the same route, so a profile spelling out `direct` over
// empty globals has changed nothing. The reverse — globals unset, profile naming
// a group — is a real change and must be reported, which is what keeps this test
// from passing on a canon that collapses everything.
func TestEffectiveConfigUnsetFetchDetourEqualsDirect(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
base := func(profDetour string) *model.Model {
return &model.Model{
Globals: model.Globals{ActiveProfile: "p", FetchDetour: ""},
Profiles: []model.Profile{{
Name: "p", Enabled: true, FetchDetour: profDetour,
}},
}
}
setEffectiveRead(t, base("direct"))
_, out := getEffective(t, srv, cookie)
if out.FetchDetour.Overridden || out.FetchDetour.Profile != "" {
t.Errorf("unset globals vs explicit `direct` is the same route, not an override: %+v", out.FetchDetour)
}
if out.FetchDetour.Value != "direct" {
t.Errorf("FetchDetour.Value = %q, want direct", out.FetchDetour.Value)
}
// Positive control for the same instrument: with the SAME empty globals, a
// profile that names a group really does change the route.
setEffectiveRead(t, base("group:sim-bypass"))
_, out = getEffective(t, srv, cookie)
if !out.FetchDetour.Overridden {
t.Errorf("empty globals vs group:sim-bypass IS an override: %+v", out.FetchDetour)
}
if out.FetchDetour.Value != "group:sim-bypass" || out.FetchDetour.Profile != "p" {
t.Errorf("FetchDetour = %+v, want value group:sim-bypass from profile p", out.FetchDetour)
}
}
// TestEffectiveConfigInheritanceIsPerField: a profile that overrides only the
// fallback keeps globals' default. Per-field inheritance is the contract
// model.overrideOf implements; a page told otherwise would show the wrong
// resolver as first-consulted on the uplink where DNS is already fragile.
func TestEffectiveConfigInheritanceIsPerField(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
setEffectiveRead(t, &model.Model{
Globals: model.Globals{
ActiveProfile: "p",
ResolverDefault: "quad9",
ResolverFallback: "cloudflare",
},
Profiles: []model.Profile{{Name: "p", Enabled: true, ResolverFallback: "yandex-sec"}},
})
_, out := getEffective(t, srv, cookie)
if out.ResolverDefault.Value != "quad9" || out.ResolverDefault.Overridden {
t.Errorf("the profile said nothing about resolver_default; globals stands: %+v", out.ResolverDefault)
}
if out.ResolverFallback.Value != "yandex-sec" || !out.ResolverFallback.Overridden {
t.Errorf("the profile DID override resolver_fallback: %+v", out.ResolverFallback)
}
}
// TestEffectiveConfigNoActiveProfile: with nothing active, every stored value
// stands and no row may name a profile.
func TestEffectiveConfigNoActiveProfile(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
m := dnsProfileModel()
m.Globals.ActiveProfile = ""
for i := range m.Profiles {
m.Profiles[i].Enabled = false
}
setEffectiveRead(t, m)
_, out := getEffective(t, srv, cookie)
if out.ActiveProfile != "" {
t.Fatalf("ActiveProfile = %q, want \"\" (no enabled profile)", out.ActiveProfile)
}
for _, tc := range []struct {
field string
got effectiveOverride
want string
}{
{"ResolverDefault", out.ResolverDefault, "quad9"},
{"ResolverFallback", out.ResolverFallback, "cloudflare"},
{"EndpointResolver", out.EndpointResolver, "local"},
{"FetchDetour", out.FetchDetour, "direct"},
} {
if tc.got.Value != tc.want || tc.got.Stored != tc.want {
t.Errorf("%s = %+v, want the stored %q on both halves", tc.field, tc.got, tc.want)
}
if tc.got.Overridden || tc.got.Profile != "" {
t.Errorf("%s: no profile is active, so nothing may be blamed: %+v", tc.field, tc.got)
}
}
}
// TestEffectiveConfigProfileImpliesOverridden pins the redundancy the wire shape
// rests on: `Profile != ""` and `Overridden` are the same fact. A client that
// renders on either one behaves identically, so neither reading can resurrect
// the permanent no-news line.
func TestEffectiveConfigProfileImpliesOverridden(t *testing.T) {
for _, m := range []*model.Model{
dnsProfileModel(),
func() *model.Model { c := dnsProfileModel(); c.Globals.ActiveProfile = "ethernet-uplink"; return c }(),
func() *model.Model { c := dnsProfileModel(); c.Globals = model.Globals{}; return c }(),
{},
} {
out := effectiveConfig(m)
for field, e := range map[string]effectiveOverride{
"ResolverDefault": out.ResolverDefault,
"ResolverFallback": out.ResolverFallback,
"EndpointResolver": out.EndpointResolver,
"FetchDetour": out.FetchDetour,
} {
if (e.Profile != "") != e.Overridden {
t.Errorf("%s: Profile=%q but Overridden=%v — the two must always agree", field, e.Profile, e.Overridden)
}
if e.Overridden && e.Value == e.Stored {
t.Errorf("%s: Overridden with Value == Stored (%q) — nothing differs", field, e.Value)
}
}
}
}
// TestEffectiveConfigMethodNotAllowed: it is a read endpoint. (Authentication is
// covered for every route by the apiRoutes matrix in audit_test.go.)
func TestEffectiveConfigMethodNotAllowed(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
for _, m := range []string{http.MethodPut, http.MethodPost, http.MethodDelete} {
resp := do(t, srv, m, "/api/config/effective", cookie, "{}")
resp.Body.Close()
if resp.StatusCode != http.StatusMethodNotAllowed {
t.Errorf("%s /api/config/effective: got %d, want 405 — a derived view is read-only "+
"and must not look writable", m, resp.StatusCode)
}
}
}
// TestConfigRoundTripsProfileAndFetchFields is the round trip the four new
// config fields had never made.
//
// It takes the EXACT BYTES of GET /api/config and PUTs them straight back.
// handleConfigPut decodes with DisallowUnknownFields(), so any key the GET emits
// that the PUT cannot name answers 400 — which is how the panel would have
// learned about a field-name mismatch, or about a derived field parked on
// /api/config: on the live router, from a red toast, after a release. The four
// values are then read off the write seam, so a PUT that returns 200 while
// dropping them cannot pass either.
func TestConfigRoundTripsProfileAndFetchFields(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
src := dnsProfileModel()
origGet := configGetRead
configGetRead = func() (*model.Model, error) { return src, nil }
var wrote *model.Model
origWrite, origSync, origCheck := writeConfig, syncSubCaches, checkWritable
writeConfig = func(m *model.Model) error { wrote = m; return nil }
syncSubCaches = func(*model.Model) error { return nil }
checkWritable = func(*model.Model) error { return nil }
t.Cleanup(func() {
configGetRead = origGet
writeConfig, syncSubCaches, checkWritable = origWrite, origSync, origCheck
})
req, _ := http.NewRequest(http.MethodGet, srv.URL+"/api/config", nil)
req.AddCookie(cookie)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("GET /api/config: %v", err)
}
body, err := io.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
t.Fatalf("read GET body: %v", err)
}
// The GET must actually carry the new fields, or the round trip below proves
// nothing about them.
for _, want := range []string{`"FetchDetour": "direct"`, `"ResolverDefault": "yandex"`,
`"ResolverFallback": "yandex-sec"`, `"EndpointResolver": "yandex"`} {
if !bytes.Contains(body, []byte(want)) {
t.Fatalf("GET /api/config does not carry %s:\n%s", want, body)
}
}
// And it must NOT carry the derived view: that is the whole reason
// /api/config/effective is a separate endpoint. A key like this on the desired
// state is exactly what DisallowUnknownFields rejects on the way back.
for _, forbidden := range []string{"Effective", "Overridden"} {
if bytes.Contains(body, []byte(forbidden)) {
t.Fatalf("GET /api/config carries the derived key %q; it belongs on "+
"/api/config/effective, not on the body the panel PUTs back", forbidden)
}
}
putReq, _ := http.NewRequest(http.MethodPut, srv.URL+"/api/config", bytes.NewReader(body))
putReq.Header.Set("Content-Type", "application/json")
putReq.AddCookie(cookie)
putResp, err := http.DefaultClient.Do(putReq)
if err != nil {
t.Fatalf("PUT /api/config: %v", err)
}
defer putResp.Body.Close()
if putResp.StatusCode != http.StatusOK {
b, _ := readAll(putResp)
t.Fatalf("PUT of the body GET just returned: got %d, want 200 (body %s)", putResp.StatusCode, b)
}
if wrote == nil {
t.Fatal("the PUT returned 200 but never reached the write seam")
}
if wrote.Globals.FetchDetour != "direct" {
t.Errorf("Globals.FetchDetour survived the round trip as %q, want direct", wrote.Globals.FetchDetour)
}
var mobile *model.Profile
for i := range wrote.Profiles {
if wrote.Profiles[i].Name == "mobile-uplink" {
mobile = &wrote.Profiles[i]
}
}
if mobile == nil {
t.Fatalf("the written model lost the mobile-uplink profile: %+v", wrote.Profiles)
}
for _, tc := range []struct{ field, got, want string }{
{"ResolverDefault", mobile.ResolverDefault, "yandex"},
{"ResolverFallback", mobile.ResolverFallback, "yandex-sec"},
{"EndpointResolver", mobile.EndpointResolver, "yandex"},
{"FetchDetour", mobile.FetchDetour, "group:sim-bypass"},
} {
if tc.got != tc.want {
t.Errorf("Profile.%s survived the round trip as %q, want %q", tc.field, tc.got, tc.want)
}
}
}
// TestConfigPutRejectsDerivedField is the other half of the same contract, from
// the client's side: if the panel ever PUTs the effective view back with the
// config, the daemon must refuse it loudly rather than accept and ignore it.
// This is what makes "do not park derived data on /api/config" enforceable
// instead of advisory.
func TestConfigPutRejectsDerivedField(t *testing.T) {
s := newTestServer(t)
srv := httptest.NewServer(s.Handler())
defer srv.Close()
cookie := login(t, srv, s)
wrote := false
origWrite, origSync, origCheck := writeConfig, syncSubCaches, checkWritable
writeConfig = func(*model.Model) error { wrote = true; return nil }
syncSubCaches = func(*model.Model) error { return nil }
checkWritable = func(*model.Model) error { return nil }
t.Cleanup(func() { writeConfig, syncSubCaches, checkWritable = origWrite, origSync, origCheck })
body := `{"Globals": {"FetchDetour": "direct"},
"EffectiveDNS": {"FetchDetour": {"Value": "group:sim-bypass"}}}`
resp := doPut(t, srv, cookie, body)
defer resp.Body.Close()
if resp.StatusCode != http.StatusBadRequest {
t.Fatalf("PUT carrying a derived field: got %d, want 400", resp.StatusCode)
}
if wrote {
t.Fatal("a rejected PUT must not reach the writer")
}
var out map[string]string
_ = json.NewDecoder(resp.Body).Decode(&out)
if !strings.Contains(out["error"], "EffectiveDNS") {
t.Errorf("the refusal must name the offending field; got %q", out["error"])
}
}
+1
View File
@@ -269,6 +269,7 @@ func (s *Server) buildRouter() http.Handler {
// Read + control API — all require a valid session cookie.
mux.Handle("/api/status", s.requireSession(http.HandlerFunc(s.handleStatus)))
mux.Handle("/api/config", s.requireSession(http.HandlerFunc(s.handleConfig)))
mux.Handle("/api/config/effective", s.requireSession(http.HandlerFunc(s.handleEffectiveConfig)))
mux.Handle("/api/apply", s.requireSession(http.HandlerFunc(s.handleApply)))
mux.Handle("/api/confirm", s.requireSession(http.HandlerFunc(s.handleConfirm)))
mux.Handle("/api/rollback", s.requireSession(http.HandlerFunc(s.handleRollback)))