feat(l3): ping travels the tunnel by default, and every LAN zone can reach it

l3_tunnel was opt-in, and "off" had no honest win left in it. Off, a LAN ping
is decided by `untunnelable` alone and every rung is a drop (block) or a
disclosure (icmp/direct send the echo out of the WAN with the client's real
address). "Ping works" was never the state where ping was tunnelled — it was
the state where ping was leaking. On, an L3-capable outbound carries the echo
and one that is not drops it honestly: adapter.JudgeFlow returns ActionDrop for
an ICMP flow whose outbound is not a tun.Port, so no reply is forged. The price
is a standing TUN + gVisor netstack, ~2 MB RSS, and it is stated where the
option is.

The switch stays. It is a real answer on a 32/64 MB device and when bisecting
whether the L3 ingress is what broke a box — but it is now a WARNED answer:
ValidateGlobals says what the off state does to ping and names the policy that
takes over. Two combinations also changed meaning and are now reported:
untunnelable=icmp is no longer "block plus working ping" (the prerouting L3
mark claims every ICMP packet before the forward chain the echo accept lives
in, and a LAN host's ICMP errors are marked in with them and dropped in the
TUN), and the existing =direct report gains a sibling rather than standing
alone.

The fw4 seeding was the second half of the same problem. The divert set spans
every LAN inbound and every iface:/zone: rule source, but 30_shater-core seeded
a forwarding into shater_l3 for `lan` only — so on a multi-zone router ICMP
from the other zones is marked, routed, accepted by `inet shater`, and dropped
by fw4's zone policy with nothing in any log. Every zone gets a forwarding now,
guarded by a scan of the actual src/dest pairs so a re-run adds nothing. Every
zone including an uplink, because guessing which zones hold clients is wrong
somewhere and a superfluous entry authorises nothing: accept_to_shater_l3 is
`oifname "shater-l3*" accept`, and the only thing that routes a packet into
that device is our own fwmark rule.

scripts/testbed-lao.sh builds the second LAN zone this needs to be visible at
all. It is not installed by the package — that is the whole opt-in mechanism.

Verified on local_openwrt (ImmortalWrt 25.12.1 r37978): three runs of the
seeder leave exactly one forwarding per zone (lan/wan/lao) and no existing
section altered; deleting the lao forwarding removes `jump accept_to_shater_l3`
from chain forward_lao and re-seeding restores it; with the idempotency guard
disabled two runs produce nine forwardings instead of three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BHw89tdWddzhjUc4bAH4tS
This commit is contained in:
2026-07-27 00:26:00 +03:00
co-authored by Claude Opus 5
parent de6fa8ebf4
commit 164b703a7d
9 changed files with 590 additions and 32 deletions
@@ -49,6 +49,31 @@ config globals 'globals'
# Queries aimed at an EXTERNAL resolver are dropped with the rest of the LAN's
# forwarded traffic.
option dns_intercept '1'
# Carry LAN ping through the tunnel. ON by default, and the alternative is
# why: without it a ping is decided by `untunnelable` below, whose rungs are
# "drop it" (block, the default) or "let it out of the WAN interface with the
# client's real IP on it" (icmp/direct). There was no setting in which ping
# both worked and stayed inside the tunnel. With this on, the engine opens a
# TUN, LAN ICMP is routed into it, and an outbound that speaks layer 3
# (WireGuard/AmneziaWG, or a direct route) carries the echo for real. An
# outbound that does not (vless/trojan/shadowsocks) makes the ping DROP —
# honestly: no reply is forged, ping reports loss. So a ping that used to
# "work" through such a node was a ping that was leaking.
#
# It costs a permanent TUN device plus the gVisor netstack behind it, about
# 2 MB of RSS for as long as the daemon runs.
#
# Set to '0' to opt out — worth it on a 32/64 MB router, or to bisect whether
# the L3 ingress is what broke something. `shaterd apply` will tell you what
# the off state costs. Your explicit value is never overwritten: this file is
# a conffile and the daemon always writes the option back as '1'/'0'.
#
# NOTE the interaction: with this ON, `untunnelable` no longer governs ping at
# all (the L3 route decision happens before the firewall chain its verdicts
# live in). It still governs ESP/AH/GRE/IGMP/SCTP, which no tunnel of ours can
# carry. `untunnelable 'icmp'` in particular stops meaning "block plus working
# ping" and is reported as such.
option l3_tunnel '1'
option ipv6 '1'
# Reserved fwmark base and routing-table base (do not overlap fw4/other apps).
option fwmark_base '0x2000'
@@ -76,9 +76,11 @@ fi
#
# Idempotency: `config zone`/`config forwarding` are normally ANONYMOUS
# sections, and a naive `uci add firewall zone` would append a duplicate on
# every re-run (uci-defaults re-run on package upgrade/reinstall). All sections
# here are NAMED instead, guarded by an existence check — a re-run re-finds the
# section and touches nothing.
# every re-run (uci-defaults re-run on package upgrade/reinstall). The zone is
# NAMED instead, guarded by an existence check — a re-run re-finds the section
# and touches nothing. The forwardings are named too where the name is free, but
# their guard is a scan of the actual src/dest pairs, which is stronger; see
# seed_l3_forwarding below.
seed_l3_zone() {
# No fw4 on this image (bare nftables build) => nothing drops the forward
# on fw4's behalf and there is nothing to punch through.
@@ -122,13 +124,103 @@ seed_l3_zone() {
# every slot and no firewall reload is needed when the slot changes.
uci add_list firewall.shater_l3.device='shater-l3*'
fi
if ! uci -q get firewall.shater_l3_fwd >/dev/null; then
uci set firewall.shater_l3_fwd=forwarding
uci set firewall.shater_l3_fwd.src='lan'
uci set firewall.shater_l3_fwd.dest='shater_l3'
fi
seed_l3_forwardings
uci -q commit firewall
}
# EVERY zone gets a forwarding into shater_l3, not just `lan`.
#
# The bug this closes is silent by construction. The daemon's divert set is built
# from every enabled `config inbound`'s network PLUS every device a rule names
# through an `iface:`/`zone:` source (shater/netplane/nft.go, nftDivertRefs) — so
# on a router with several LAN zones, ICMP from ALL of them is marked and routed
# into the TUN by our table. Our table then accepts it and fw4 drops it anyway,
# because the forward is judged in `forward_<source zone>` and only `lan` had a
# jump to `accept_to_shater_l3`. Result: ping through the tunnel works from one
# subnet and not from the next, with nothing in any log to say why — fw4's drop
# is the zone's policy verdict, not a rule with a name. The owner's production
# router has a single LAN zone, which is exactly why this went unnoticed; his
# second router has three.
#
# Every zone, including an uplink zone, and that is deliberate rather than lazy:
#
# - The alternative is guessing which zones hold clients, and every available
# signal is wrong somewhere. `masq='1'` marks the WAN on a stock config and
# also marks a double-NAT LAN. The name `wan*` is a convention, not a rule.
# A guess that is wrong reintroduces exactly the silent breakage above, while
# a superfluous entry costs a line of ruleset.
# - A forwarding into shater_l3 permits nothing on its own. It authorises the
# forward of packets ROUTED INTO the TUN, and the only thing that routes a
# packet there is our own fwmark rule, which matches solely on the divert
# device set. A packet arriving on the WAN is not marked and never reaches
# this decision; if an operator ever puts a WAN device in the divert set,
# they meant to and this is the entry that makes it work.
# - The reverse direction is NOT opened: no `src shater_l3` forwarding exists,
# so nothing comes out of the TUN into a zone by way of these sections. The
# engine's own replies return on the conntrack `established,related accept`
# at the top of fw4's forward chain.
#
# LIMIT, stated because it is not obvious: this is a SNAPSHOT. uci-defaults run
# at first boot and on package install/upgrade, so a zone created AFTER the last
# shater-core install has no forwarding until the next one. Re-running this
# script (or reinstalling the package) re-seeds. The durable fix belongs in the
# daemon, which recomputes the divert set on every apply and already knows which
# zones are in it; it is deliberately not attempted from here.
seed_l3_forwardings() {
uci -q show firewall 2>/dev/null |
sed -n "s/^firewall\.\([^.=]*\)=zone\$/\1/p" |
while read -r sid; do
zone=$(uci -q get "firewall.$sid.name")
# Unnamed zone: fw4 cannot reference it from a forwarding either.
[ -n "$zone" ] || continue
# Our own zone: a forwarding from shater_l3 to itself is meaningless.
[ "$zone" = "shater_l3" ] && continue
seed_l3_forwarding "$zone"
done
}
# One `config forwarding` <zone> -> shater_l3, created only if no such forwarding
# exists yet.
#
# The guard scans the ACTUAL src/dest pairs rather than trusting a section id,
# which covers all three ways one can already be there: the legacy named section
# `shater_l3_fwd` seeded by earlier releases (src=lan), the per-zone names this
# function writes, and an anonymous one an operator added by hand. Without that,
# a re-run — uci-defaults re-run on every package upgrade — would append a
# duplicate for `lan` on every upgrade.
seed_l3_forwarding() {
local zone="$1" sid found
found=$(uci -q show firewall 2>/dev/null |
sed -n "s/^firewall\.\([^.=]*\)=forwarding\$/\1/p" |
while read -r f; do
[ "$(uci -q get "firewall.$f.dest")" = "shater_l3" ] || continue
[ "$(uci -q get "firewall.$f.src")" = "$zone" ] || continue
echo yes
break
done)
[ -n "$found" ] && return 0
# Section ids are [a-zA-Z0-9_] only, while a zone name may legally carry a
# hyphen — sanitise, and keep the legacy id for `lan` so an existing install
# is recognised as already seeded rather than gaining a second section.
if [ "$zone" = "lan" ]; then
sid="shater_l3_fwd"
else
sid="shater_l3_fwd_$(printf '%s' "$zone" | sed 's/[^a-zA-Z0-9_]/_/g')"
fi
# The id may still be taken — by a section for a DIFFERENT zone whose name
# sanitises to the same thing, or by something else entirely. Fall back to an
# anonymous section rather than overwrite: the src/dest scan above is what
# makes this idempotent, the name is only there to be readable.
if uci -q get "firewall.$sid" >/dev/null; then
sid=$(uci add firewall forwarding) || return 0
else
uci set "firewall.$sid=forwarding"
fi
uci set "firewall.$sid.src=$zone"
uci set "firewall.$sid.dest=shater_l3"
}
seed_l3_zone
# Upgrade path for routers seeded by a pre-slot build.
+123
View File
@@ -0,0 +1,123 @@
#!/bin/sh
# scripts/testbed-lao.sh — add a SECOND LAN network ("lao", 10.67.1.0/24) in its
# own fw4 zone, on a testbed router.
#
# WHY IT EXISTS
#
# Almost every zone-related defect in this project is invisible on a router with
# one LAN zone, because "the zone" and "the LAN" are the same thing there. The
# divert set the daemon builds spans every LAN inbound and every `iface:`/`zone:`
# rule source, so on a multi-zone router traffic from the other zones is marked,
# routed, accepted by `inet shater` — and then dropped by fw4's zone policy,
# silently. Reproducing that needs a second zone and nothing else: no second
# physical port, no client, no traffic. This script makes one.
#
# WHY IT IS NOT SHIPPED
#
# It lives in scripts/ and is NOT installed by openwrt/shater-core/Makefile,
# which lists every file it installs by name. That is the whole opt-in mechanism,
# and it was chosen over the alternatives on purpose:
#
# - an extra /etc/uci-defaults/ file would run on EVERY install, handing a
# second network and a second firewall zone to every ordinary user — the one
# thing this must not do;
# - an environment variable read inside 30_shater-core is unreachable in
# practice: that script is deleted after its first successful run, so there
# is no later moment at which an operator could set the variable and re-run it;
# - a separate package would need a feed entry, a build, a release and a
# version, for a file that exists to be scp'd onto one VM.
#
# USAGE
#
# scp scripts/testbed-lao.sh root@testbed:/tmp/ && ssh root@testbed sh /tmp/testbed-lao.sh
# ssh root@testbed sh /tmp/testbed-lao.sh --remove
#
# It is idempotent (every section is NAMED and guarded), purely additive, and
# touches no existing section. Running it three times in a row leaves exactly one
# of everything.
set -e
REMOVE=0
[ "$1" = "--remove" ] && REMOVE=1
if [ "$REMOVE" = 1 ]; then
uci -q delete firewall.lao_fwd
uci -q delete firewall.lao
uci -q delete dhcp.lao
uci -q delete network.lao
uci -q delete network.br_lao
uci -q commit firewall
uci -q commit dhcp
uci -q commit network
/etc/init.d/network reload
/etc/init.d/firewall reload
echo "lao removed"
exit 0
fi
# --- L2: an empty bridge -----------------------------------------------------
# No ports on purpose: the point is a second ROUTED network with its own firewall
# zone, and giving it a switch port would mean re-cabling a testbed for nothing.
#
# bridge_empty is what makes a portless bridge usable. Without it netifd leaves a
# member-less bridge down (no carrier), the `lao` interface never comes up, fw4
# resolves `list network 'lao'` to an EMPTY device set, and the zone silently
# matches nothing — which would make this script a worse instrument than no
# instrument, since it would look set up and prove nothing.
if ! uci -q get network.br_lao >/dev/null; then
uci set network.br_lao=device
uci set network.br_lao.name='br-lao'
uci set network.br_lao.type='bridge'
uci set network.br_lao.bridge_empty='1'
fi
# --- L3: the interface -------------------------------------------------------
# 10.67.1.0/24 is deliberately far from anything a home LAN or a proxy node uses.
# ipaddr+netmask rather than CIDR: CIDR in `ipaddr` is a 25.12 convenience and
# this script should also run on an older testbed image.
if ! uci -q get network.lao >/dev/null; then
uci set network.lao=interface
uci set network.lao.proto='static'
uci set network.lao.device='br-lao'
uci set network.lao.ipaddr='10.67.1.1'
uci set network.lao.netmask='255.255.255.0'
fi
# --- DHCP: same shape as lan -------------------------------------------------
if ! uci -q get dhcp.lao >/dev/null; then
uci set dhcp.lao=dhcp
uci set dhcp.lao.interface='lao'
uci set dhcp.lao.start='100'
uci set dhcp.lao.limit='150'
uci set dhcp.lao.leasetime='12h'
fi
# --- Firewall: its OWN zone, which is the entire point -----------------------
# Same policies as the stock lan zone and its own forwarding to wan, so the
# network behaves like a second LAN. What it does NOT get here is a forwarding
# into shater_l3: seeding that for every zone is the job under test, done by
# /etc/uci-defaults/30_shater-core. If this script seeded it, the test would be
# testing itself.
if ! uci -q get firewall.lao >/dev/null; then
uci set firewall.lao=zone
uci set firewall.lao.name='lao'
uci set firewall.lao.input='ACCEPT'
uci set firewall.lao.output='ACCEPT'
uci set firewall.lao.forward='ACCEPT'
uci add_list firewall.lao.network='lao'
fi
if ! uci -q get firewall.lao_fwd >/dev/null; then
uci set firewall.lao_fwd=forwarding
uci set firewall.lao_fwd.src='lao'
uci set firewall.lao_fwd.dest='wan'
fi
uci commit network
uci commit dhcp
uci commit firewall
/etc/init.d/network reload
/etc/init.d/firewall reload
echo "lao seeded: br-lao 10.67.1.1/24, fw4 zone lao -> wan"
+187
View File
@@ -0,0 +1,187 @@
package model
// Tests for the `l3_tunnel` default flip: carrying LAN ping through the tunnel
// is now the DEFAULT, not an opt-in.
//
// Why it flipped, so a future reader does not "restore" the old default as a
// safety measure: with l3_tunnel off, a LAN ping is decided by `untunnelable`
// alone, and every rung of that ladder is either a drop (block) or a disclosure
// (icmp/direct let the echo out of the WAN interface with the client's real
// address). There was no configuration in which ping both worked and stayed
// inside the tunnel. With l3_tunnel on, an L3-capable outbound carries the echo
// and one that is not drops it honestly — adapter.JudgeFlow returns
// tun.ActionDrop for an ICMP flow whose outbound is not a tun.Port
// (adapter/judgeflow_icmp_lx_test.go), so no reply is ever forged. Turning it on
// therefore breaks nothing that was working truthfully.
//
// Four properties are pinned, and the middle two pull against each other:
//
// 1. the seed and an option-less config both come back ON (every install
// written before the option existed gains the ingress on upgrade);
// 2. an EXPLICIT `option l3_tunnel '0'` stays OFF across a WriteUCI->ReadUCI
// round-trip — a bool omitted at false would be re-read as the seed and
// silently flip the operator's decision back on;
// 3. the SHIPPED /etc/config/shater agrees with the seed (it is a conffile: a
// fresh install's posture is that file's literal text, not DefaultGlobals);
// 4. neither the off state nor the combinations the flip made misleading are
// reached in silence — ValidateGlobals reports all three.
import (
"os"
"path/filepath"
"strings"
"testing"
)
// TestL3TunnelDefaultOn: the seed and an option-less config both say ON.
func TestL3TunnelDefaultOn(t *testing.T) {
if !DefaultGlobals().L3Tunnel {
t.Fatal("DefaultGlobals().L3Tunnel = false, want true — with the L3 ingress off a LAN ping is either dropped or sent out of the WAN with the client's real address, and neither is what a tunnel is for")
}
m, err := ParseUCIExport("package shater\n\nconfig globals 'globals'\n\toption enabled '1'\n")
if err != nil {
t.Fatal(err)
}
if !m.Globals.L3Tunnel {
t.Fatal("a config with no l3_tunnel option parsed as OFF; an absent option must fall back to the ON seed, or every install predating the option keeps leaking its pings")
}
}
// TestL3TunnelExplicitOffPreserved: the operator's opt-out survives both the
// parse (over an ON seed) and the render->parse round-trip. Off is a supported
// answer — a 32/64 MB router where the standing TUN + gVisor netstack costs real
// money, or a bisection — so it has to be a decision the config keeps.
func TestL3TunnelExplicitOffPreserved(t *testing.T) {
m, err := ParseUCIExport("package shater\n\nconfig globals 'globals'\n" +
"\toption enabled '1'\n\toption l3_tunnel '0'\n")
if err != nil {
t.Fatal(err)
}
if m.Globals.L3Tunnel {
t.Fatal("explicit l3_tunnel '0' was overwritten by the ON seed")
}
rendered := RenderUCIExport(m)
if !strings.Contains(rendered, "option l3_tunnel '0'") {
t.Fatalf("render must emit the explicit '0' (an omitted bool would be re-read as ON):\n%s", rendered)
}
back, err := ParseUCIExport(rendered)
if err != nil {
t.Fatal(err)
}
if back.Globals.L3Tunnel {
t.Fatal("l3_tunnel '0' did not survive the WriteUCI->ReadUCI round-trip — the opt-out would be silently re-enabled on the next save")
}
}
// TestShippedConfigEnablesL3Tunnel reads the file the package actually installs.
// /etc/config/shater is a CONFFILE: written once on first install and never
// replaced on upgrade, so a fresh install's ping posture is decided by this
// file's literal text and not by DefaultGlobals. The two must agree, and only a
// test that reads the shipped bytes can say that they do.
func TestShippedConfigEnablesL3Tunnel(t *testing.T) {
path := filepath.Join("..", "..", "openwrt", "shater-core", "files", "etc", "config", "shater")
raw, err := os.ReadFile(path)
if err != nil {
t.Fatalf("shipped config %s is unreadable (%v) — this test must not be skipped: it is the only check that the installed file and DefaultGlobals agree", path, err)
}
text := string(raw)
// Line-wise and comment-aware on purpose. That file is more than half
// commented-out examples, so a plain strings.Contains would be satisfied by
// `#option l3_tunnel '1'` — and the parse below cannot tell the difference
// either, because a commented option falls back to the seed, which is now
// also true. Nothing else in this test can catch that, so this check has to.
live := false
for _, line := range strings.Split(text, "\n") {
if strings.TrimSpace(line) == "option l3_tunnel '1'" {
live = true
break
}
}
if !live {
t.Error("the shipped /etc/config/shater must carry an UNCOMMENTED `option l3_tunnel '1'`: a fresh install reads this file, and an operator who later opts out must be able to see the option they are flipping")
}
m, err := ParseUCIExport(text)
if err != nil {
t.Fatalf("shipped config does not parse: %v", err)
}
if !m.Globals.L3Tunnel {
t.Error("shipped config parses with L3Tunnel off")
}
if m.Globals.Enabled {
t.Error("shipped config must stay inert (enabled '0'): a fresh install may not touch connectivity")
}
}
// TestL3TunnelOffIsWarned: nobody reaches the off state by accident any more —
// absent means ON — so a false is an explicit opt-out and must be told what it
// costs, rather than be discovered later as "ping stopped working" or, worse and
// silently, as "ping works, from the WAN address".
func TestL3TunnelOffIsWarned(t *testing.T) {
g := DefaultGlobals()
g.L3Tunnel = false
ws := ValidateGlobals(g)
if !warnsMention(ws, "l3_tunnel is off") {
t.Fatalf("turning the L3 ingress off produced no warning: %v", warningTexts(ws))
}
// The default policy has to be NAMED, not implied: "block" and "direct" have
// opposite consequences for the packet and the operator cannot tell which
// they are in from the option's absence.
if !warnsMention(ws, "untunnelable (block)") {
t.Fatalf("the off warning does not name the policy that takes over instead: %v", warningTexts(ws))
}
// The control: with the ingress ON the same prober must stay quiet, or a
// green "no warning" above would prove nothing about the check firing.
if ws := ValidateGlobals(DefaultGlobals()); warnsMention(ws, "l3_tunnel is off") {
t.Fatalf("the default (L3 ingress ON) was reported as off: %v", warningTexts(ws))
}
}
// TestL3TunnelWithUntunnelableICMPIsReported is the combination the flip made
// misleading. `icmp` used to mean "block, plus working ping". With l3_tunnel on
// the prerouting L3 mark claims every ICMP packet from a diverted device before
// the forward chain where that echo accept lives, so the ping half is dead and
// only the half the operator may not have wanted survives: ESP/AH/GRE/IGMP/SCTP
// out with the real address toward directly-routed destinations. Under the old
// opt-in default the pair was exotic; under the new one it is ordinary, and
// silence about it would be the panel lying about state.
func TestL3TunnelWithUntunnelableICMPIsReported(t *testing.T) {
g := DefaultGlobals()
g.Untunnelable = "icmp"
ws := ValidateGlobals(g)
if !warnsMention(ws, "ping no longer reaches this policy") {
t.Fatalf("l3_tunnel=1 + untunnelable=icmp was accepted in silence: %v", warningTexts(ws))
}
// Control 1: the same policy WITHOUT the L3 ingress is the state the rung was
// designed for and must not be reported.
g.L3Tunnel = false
if ws := ValidateGlobals(g); warnsMention(ws, "ping no longer reaches this policy") {
t.Fatalf("untunnelable=icmp under l3_tunnel=0 was reported as dead: %v", warningTexts(ws))
}
// Control 2: the L3 ingress with the DEFAULT policy is the ordinary posture
// and must not be reported either.
if ws := ValidateGlobals(DefaultGlobals()); warnsMention(ws, "ping no longer reaches this policy") {
t.Fatalf("the default posture (l3_tunnel=1 + untunnelable=block) was reported: %v", warningTexts(ws))
}
}
// warnsMention reports whether any warning text contains sub.
func warnsMention(ws []Warning, sub string) bool {
for _, w := range ws {
if strings.Contains(w.Message, sub) {
return true
}
}
return false
}
// warningTexts flattens warnings for a failure message.
func warningTexts(ws []Warning) []string {
out := make([]string, 0, len(ws))
for _, w := range ws {
out = append(out, w.Message)
}
return out
}
+7 -2
View File
@@ -87,14 +87,19 @@ func TestLogKnobsAliases(t *testing.T) {
// TestValidateGlobalsLogMaxKB: out-of-range positive values warn (they are
// applied clamped by the sink); 0 and in-range values do not.
//
// Every fixture carries L3Tunnel: true — the DEFAULT since the L3 ingress became
// opt-out. Left at the Go zero value it would read as an explicit opt-out and
// draw a warning of its own (TestL3TunnelOffIsWarned), which has nothing to do
// with the log budget this test is about.
func TestValidateGlobalsLogMaxKB(t *testing.T) {
for _, ok := range []int{0, LogMaxKBMin, 2048, LogMaxKBMax} {
if ws := ValidateGlobals(Globals{LogMaxKB: ok}); len(ws) != 0 {
if ws := ValidateGlobals(Globals{LogMaxKB: ok, L3Tunnel: true}); len(ws) != 0 {
t.Errorf("log_max_kb=%d should not warn: %v", ok, ws)
}
}
for _, bad := range []int{1, LogMaxKBMin - 1, LogMaxKBMax + 1, 1 << 20} {
ws := ValidateGlobals(Globals{LogMaxKB: bad})
ws := ValidateGlobals(Globals{LogMaxKB: bad, L3Tunnel: true})
if len(ws) != 1 || !strings.Contains(ws[0].Message, "log_max_kb") {
t.Errorf("log_max_kb=%d should warn once about the clamp, got %v", bad, ws)
}
+44 -3
View File
@@ -260,7 +260,7 @@ type Globals struct {
// current behaviour exactly.
Untunnelable string
// L3Tunnel opts the router into the L3 ingress: the engine opens a TUN device
// L3Tunnel puts the router on the L3 ingress: the engine opens a TUN device
// (netplane.L3Device) and the nft prerouting chain policy-routes LAN traffic
// the tunnel can only carry at layer 3 into it. There the engine's ordinary
// route rules decide the outbound, and an L3-capable one (wireguard/AWG —
@@ -279,8 +279,37 @@ type Globals struct {
// have. Those stay governed by Untunnelable and UntunnelableEgress below. Ping
// and Windows `tracert` through the tunnel are the whole of what this buys.
//
// Default false: without it nothing changes, and Untunnelable alone decides
// what happens to non-TCP/UDP traffic.
// DEFAULT TRUE (opt-out). The reason it flipped is that "off" had no honest
// win left in it. Off, a LAN ping is decided by Untunnelable alone, and each
// of the three rungs is either a drop or a leak: `block` drops it, `icmp` and
// `direct` let the echo out of the WAN interface with the client's real
// address on it. "Ping works" was never a state where ping was tunnelled — it
// was the state where ping was disclosing. On, an outbound that carries L3
// (WireGuard/AmneziaWG, or `direct`) carries the echo for real, and an
// outbound that does not (vless/trojan/ss) drops it: the packet enters the
// TUN, sing-tun's dispatcher finds no FlowOutbound on the selected route and
// nothing goes out. That drop is HONEST — verified on hardware, no reply is
// forged, ping simply reports loss — so turning this on breaks nothing that
// was working truthfully and closes what was leaking.
//
// The price is not zero and is not hidden: a permanent TUN device plus the
// gVisor netstack behind it, ~2 MB RSS on the routers this runs on, for as
// long as the daemon is up.
//
// Off is therefore still a supported answer, not a vestige — on a 32/64 MB
// device where 2 MB is real money, and when bisecting whether the L3 ingress
// is what broke a box. It is a WARNED answer: ValidateGlobals reports what
// turning it off costs, so nobody reaches the off state without being told
// what ping does there (validate.go, and see UntunnelableEgress below for the
// protocols this cannot help either way).
//
// Interaction with Untunnelable, which the validator also reports: with this
// ON the L3 mark is stamped in prerouting and the routing decision carries
// ICMP into the TUN before the forward chain — where Untunnelable's verdicts
// live — is ever consulted. So Untunnelable no longer governs ping at all; it
// keeps governing exactly the protocols the dispatcher will not take (ESP,
// AH, GRE, IGMP, SCTP) and, under `icmp`, the ICMP ERRORS a LAN host raises,
// which are marked into the TUN with the echoes and are not dispatched there.
L3Tunnel bool
// UntunnelableEgress names an egress of type interface/tunnel that carries the
@@ -382,6 +411,17 @@ const (
// the clients that were trying to evade the router (see the field comment). An
// EXPLICIT `option dns_intercept '0'` parses over this seed and is preserved:
// render.go always emits the bool, so opting out is a decision the config keeps.
//
// L3Tunnel is seeded ON (opt-out) for the same reason and by the same mechanism.
// A config that never mentions `l3_tunnel` — every install predating the option —
// gains the L3 ingress on upgrade, and that is the intent: without it a LAN ping
// is either dropped (Untunnelable "block", the default) or sent out of the WAN
// with the client's real address ("icmp"/"direct"), so there was no configuration
// in which ping both worked and stayed inside the tunnel. With it, ping is
// carried by an L3-capable outbound and honestly dropped by one that is not — no
// reply is ever forged. The cost is a standing TUN + gVisor netstack, ~2 MB RSS.
// An EXPLICIT `option l3_tunnel '0'` parses over this seed and is preserved
// (render.go always emits the bool) and ValidateGlobals says what it costs.
func DefaultGlobals() Globals {
return Globals{
Enabled: true,
@@ -391,6 +431,7 @@ func DefaultGlobals() Globals {
LogMaxKB: LogMaxKBDefault,
KillSwitch: "closed",
Untunnelable: "block",
L3Tunnel: true,
IPv6: true,
FwmarkBase: 0x2000,
TableBase: 0x2000,
+9 -7
View File
@@ -21,7 +21,7 @@ func richModel() *Model {
LogLevel: "debug",
KillSwitch: "open",
Untunnelable: "direct",
L3Tunnel: true, // opt-in; exercise the non-default via round-trip
L3Tunnel: false, // opt-out; exercise the non-default via round-trip
UntunnelableEgress: "frag", // exercise the non-default via round-trip
IPv6: false,
FwmarkBase: 0x2000,
@@ -314,9 +314,11 @@ func TestGroupHealthRoundTrip(t *testing.T) {
}
}
// TestL3TunnelRoundTrip pins the l3_tunnel opt-in: an EXPLICIT true (L3 ingress
// on) survives WriteUCI->ReadUCI, and an ABSENT option stays false (opt-in,
// never accidentally on).
// TestL3TunnelRoundTrip pins the l3_tunnel round-trip: BOTH spellings survive
// WriteUCI->ReadUCI, and an ABSENT option comes back ON — the option is opt-OUT
// now, so the interesting direction is that an explicit '0' is not silently
// re-enabled. The default itself, the shipped config and the warnings that go
// with the off state are pinned in l3tunnel_default_test.go.
func TestL3TunnelRoundTrip(t *testing.T) {
for _, v := range []bool{true, false} {
g := DefaultGlobals()
@@ -329,13 +331,13 @@ func TestL3TunnelRoundTrip(t *testing.T) {
t.Fatalf("v=%v round-trip: l3_tunnel=%v, want %v", v, got.Globals.L3Tunnel, v)
}
}
// Absent option => off (opt-in default, the Go zero value).
// Absent option => ON (the DefaultGlobals seed, opt-out).
got, err := ParseUCIExport("package shater\n")
if err != nil {
t.Fatal(err)
}
if got.Globals.L3Tunnel {
t.Fatalf("absent l3_tunnel = %v, want false (default, opt-in)", got.Globals.L3Tunnel)
if !got.Globals.L3Tunnel {
t.Fatalf("absent l3_tunnel = %v, want true (default, opt-out)", got.Globals.L3Tunnel)
}
}
+52
View File
@@ -342,6 +342,44 @@ func ValidateGlobals(g Globals) []Warning {
"instead of failing. With \"block\" the same failure is honest packet loss.")
}
// The same collision, one rung down, and it went unreported until l3_tunnel
// became the default and made the pair ORDINARY rather than exotic. `icmp`
// buys two things: echo everywhere, and the other untunnelable protocols
// toward destinations the routing rules send direct. With l3_tunnel on the
// FIRST of those is dead — the prerouting L3 mark claims every ICMP/ICMPv6
// packet from a diverted device before the forward chain where the echo
// accept lives — so the rung is not "block plus ping" any more, and an
// operator who picked it FOR the ping is now getting only the ESP/AH/GRE
// half they may not have wanted. Worse, the mark is not selective: a LAN
// host's ICMP ERRORS (fragmentation-needed and friends) are marked into the
// TUN too, where sing-tun's dispatcher classifies echo alone and drops the
// rest — so `icmp` no longer delivers those either, which `block` never
// promised but `icmp` did.
if g.L3Tunnel && strings.ToLower(strings.TrimSpace(g.Untunnelable)) == "icmp" {
add("l3_tunnel is on, so ping no longer reaches this policy at all — echo is routed " +
"into the tunnel in prerouting, before the forward chain where \"icmp\" would have " +
"let it out. What \"icmp\" still does is let the protocols the tunnel cannot carry " +
"(IPsec ESP/AH, PPTP/GRE, IGMP, SCTP) out with your real IP address toward " +
"directly-routed destinations, and it no longer delivers a LAN host's ICMP error " +
"packets (path-MTU discovery on a direct route), which are routed into the tunnel " +
"with the echoes and dropped there. If you chose \"icmp\" for working ping, " +
"\"block\" now gives you that and discloses less.")
}
// Nobody arrives at l3_tunnel=0 by accident any more: absent means ON
// (DefaultGlobals), so a false here is either an explicit opt-out or a
// config the daemon rewrote after one. Say what it costs, once, rather than
// let the operator discover it as "ping stopped working" or — worse, and
// silently — as "ping works, from the WAN address".
if !g.L3Tunnel {
add("l3_tunnel is off: LAN ping and Windows tracert do not travel through the tunnel. " +
"What happens to them is decided by untunnelable (" + untunnelableName(g.Untunnelable) +
") instead — \"block\" drops them, \"icmp\" and \"direct\" let them out of the WAN " +
"interface with the client's real IP address on them. Turning l3_tunnel back on " +
"costs a TUN device and roughly 2 MB of RAM and is the only setting in which ping " +
"both works and stays inside the tunnel.")
}
switch strings.ToLower(strings.TrimSpace(g.StatsBackend)) {
case "", "off", "memory", "sqlite":
default:
@@ -352,6 +390,20 @@ func ValidateGlobals(g Globals) []Warning {
return out
}
// untunnelableName spells Globals.Untunnelable the way the data plane reads it,
// for use inside a diagnostic: empty — and anything unrecognised, which is
// reported on its own line above — is "block", the same normalisation
// netplane.EffectiveUntunnelable performs (netplane imports model, so the
// function itself cannot be shared).
func untunnelableName(policy string) string {
switch p := strings.ToLower(strings.TrimSpace(policy)); p {
case "icmp", "direct":
return p
default:
return "block"
}
}
// ValidateSubscriptions reports subscription options that are accepted but inert.
func ValidateSubscriptions(subs []Subscription) []Warning {
var out []Warning
+43 -12
View File
@@ -112,7 +112,10 @@ func TestValidateRulesWithoutFirewallInfo(t *testing.T) {
// apply" — a router that will not start is worse than one with a warning, because
// the kill-switch is closed while the engine is down.
func TestValidateIsFailOpen(t *testing.T) {
m := &Model{Rules: []Rule{
// Globals at the DEFAULT posture (L3Tunnel on): the zero value would read as
// an explicit opt-out and add a globals warning to the per-rule count below,
// which is not what this test measures.
m := &Model{Globals: Globals{L3Tunnel: true}, Rules: []Rule{
{Name: "bad", Enabled: true, Src: []string{"iface:wan", "iface:lan"}},
{Name: "worse", Enabled: true, Src: []string{"zone:wan", "iface:wan6"}},
{Name: "good", Enabled: true, Src: []string{"iface:lan"}},
@@ -131,9 +134,19 @@ func TestValidateIsFailOpen(t *testing.T) {
if after := RenderUCIExport(m); after != before {
t.Error("Validate must not mutate the model")
}
// Nil model / no rules: no panic, no warnings.
if len((*Model)(nil).Validate(nil)) != 0 || len((&Model{}).Validate(nil)) != 0 {
t.Error("empty and nil models must validate cleanly")
// Nil model: no panic, no warnings, and no classifier needed.
if len((*Model)(nil).Validate(nil)) != 0 {
t.Error("a nil model must validate cleanly")
}
// An EMPTY model is not a clean one any more, and deliberately so: its zero
// Globals say l3_tunnel off, which is a real posture with a real cost and is
// reported. What must still hold is that nothing is invented about the
// sections it does not have — no rule, egress, group, alert or subscription
// warning can come out of a model with none of them.
for _, w := range (&Model{}).Validate(nil) {
if w.Section != "globals" {
t.Errorf("empty model produced a %s warning out of nothing: %v", w.Section, w)
}
}
}
@@ -183,7 +196,11 @@ func TestValidateGlobalsAcceptsEveryWorkingLevel(t *testing.T) {
levels = append(levels, SilentLogLevels...)
for _, lvl := range levels {
for _, spelling := range []string{lvl, strings.ToUpper(lvl), " " + lvl + " "} {
if ws := ValidateGlobals(Globals{LogLevel: spelling}); len(ws) != 0 {
// L3Tunnel is set to its DEFAULT here rather than left at the Go zero
// value: the default is now ON, so a false is an explicit opt-out and
// draws a warning of its own (TestL3TunnelOffIsWarned). This test is
// about log levels, so it starts from the default posture.
if ws := ValidateGlobals(Globals{LogLevel: spelling, L3Tunnel: true}); len(ws) != 0 {
t.Errorf("level %q must not warn, got %v", spelling, ws)
}
}
@@ -212,11 +229,18 @@ func TestValidateGlobalsUnknownEnums(t *testing.T) {
if !hasWarning(ws, "globals", "stats backend \"postgres\"") {
t.Errorf("bad stats backend must warn (it silently falls back to memory), got %v", ws)
}
// The three legal untunnelable values and three legal backends stay quiet.
// The three legal untunnelable values and three legal backends draw no
// VOCABULARY complaint. Deliberately not "no warnings at all" any more: since
// l3_tunnel defaults ON, "icmp" and "direct" are reported for a different
// reason — the L3 route claims ping before the forward chain those policies
// live in — and that report is the subject of
// TestValidateGlobalsL3TunnelDirectConflict / TestL3TunnelWithUntunnelableICMPIsReported,
// not of this test. Both enum checks phrase themselves as "not recognised",
// which is what is asserted absent here.
for _, u := range []string{"", "block", "icmp", "direct"} {
for _, b := range []string{"", "off", "memory", "sqlite"} {
if ws := ValidateGlobals(Globals{Untunnelable: u, StatsBackend: b}); len(ws) != 0 {
t.Errorf("untunnelable=%q backend=%q must not warn, got %v", u, b, ws)
if ws := ValidateGlobals(Globals{Untunnelable: u, StatsBackend: b, L3Tunnel: true}); hasWarning(ws, "globals", "not recognised") {
t.Errorf("untunnelable=%q backend=%q must not be reported as unrecognised, got %v", u, b, ws)
}
}
}
@@ -241,16 +265,23 @@ func TestValidateGlobalsL3TunnelDirectConflict(t *testing.T) {
if ws := ValidateGlobals(Globals{L3Tunnel: true, Untunnelable: " Direct "}); !hasWarning(ws, "globals", "l3_tunnel") {
t.Errorf("normalised \" Direct \" must still warn, got %v", ws)
}
// Only the combination warns: each option alone, and l3_tunnel with the
// policies that keep the failure mode honest, stay quiet.
// Only the combination draws THIS report: each option alone, and l3_tunnel
// with the other policies, must not be told that "direct" is redundant.
// Asserted by the message's own words rather than by "no warnings at all",
// because two of these fixtures legitimately draw a DIFFERENT report now that
// l3_tunnel defaults on — `{Untunnelable: "direct"}` has the ingress off, and
// `{L3Tunnel: true, Untunnelable: "icmp"}` is the sibling collision one rung
// down. Folding those into a bare len(ws)==0 would make this test fail for
// reasons that have nothing to do with `direct`.
const directReport = "\"direct\" no longer buys you anything for ping"
for _, g := range []Globals{
{Untunnelable: "direct"},
{L3Tunnel: true},
{L3Tunnel: true, Untunnelable: "block"},
{L3Tunnel: true, Untunnelable: "icmp"},
} {
if ws := ValidateGlobals(g); len(ws) != 0 {
t.Errorf("globals %+v must not warn, got %v", g, ws)
if ws := ValidateGlobals(g); hasWarning(ws, "globals", directReport) {
t.Errorf("globals %+v must not draw the direct-conflict report, got %v", g, ws)
}
}
}