feat(l3): ping travels the tunnel by default, and every LAN zone can reach it
l3_tunnel was opt-in, and "off" had no honest win left in it. Off, a LAN ping is decided by `untunnelable` alone and every rung is a drop (block) or a disclosure (icmp/direct send the echo out of the WAN with the client's real address). "Ping works" was never the state where ping was tunnelled — it was the state where ping was leaking. On, an L3-capable outbound carries the echo and one that is not drops it honestly: adapter.JudgeFlow returns ActionDrop for an ICMP flow whose outbound is not a tun.Port, so no reply is forged. The price is a standing TUN + gVisor netstack, ~2 MB RSS, and it is stated where the option is. The switch stays. It is a real answer on a 32/64 MB device and when bisecting whether the L3 ingress is what broke a box — but it is now a WARNED answer: ValidateGlobals says what the off state does to ping and names the policy that takes over. Two combinations also changed meaning and are now reported: untunnelable=icmp is no longer "block plus working ping" (the prerouting L3 mark claims every ICMP packet before the forward chain the echo accept lives in, and a LAN host's ICMP errors are marked in with them and dropped in the TUN), and the existing =direct report gains a sibling rather than standing alone. The fw4 seeding was the second half of the same problem. The divert set spans every LAN inbound and every iface:/zone: rule source, but 30_shater-core seeded a forwarding into shater_l3 for `lan` only — so on a multi-zone router ICMP from the other zones is marked, routed, accepted by `inet shater`, and dropped by fw4's zone policy with nothing in any log. Every zone gets a forwarding now, guarded by a scan of the actual src/dest pairs so a re-run adds nothing. Every zone including an uplink, because guessing which zones hold clients is wrong somewhere and a superfluous entry authorises nothing: accept_to_shater_l3 is `oifname "shater-l3*" accept`, and the only thing that routes a packet into that device is our own fwmark rule. scripts/testbed-lao.sh builds the second LAN zone this needs to be visible at all. It is not installed by the package — that is the whole opt-in mechanism. Verified on local_openwrt (ImmortalWrt 25.12.1 r37978): three runs of the seeder leave exactly one forwarding per zone (lan/wan/lao) and no existing section altered; deleting the lao forwarding removes `jump accept_to_shater_l3` from chain forward_lao and re-seeding restores it; with the idempotency guard disabled two runs produce nine forwardings instead of three. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BHw89tdWddzhjUc4bAH4tS
This commit is contained in:
@@ -49,6 +49,31 @@ config globals 'globals'
|
||||
# Queries aimed at an EXTERNAL resolver are dropped with the rest of the LAN's
|
||||
# forwarded traffic.
|
||||
option dns_intercept '1'
|
||||
# Carry LAN ping through the tunnel. ON by default, and the alternative is
|
||||
# why: without it a ping is decided by `untunnelable` below, whose rungs are
|
||||
# "drop it" (block, the default) or "let it out of the WAN interface with the
|
||||
# client's real IP on it" (icmp/direct). There was no setting in which ping
|
||||
# both worked and stayed inside the tunnel. With this on, the engine opens a
|
||||
# TUN, LAN ICMP is routed into it, and an outbound that speaks layer 3
|
||||
# (WireGuard/AmneziaWG, or a direct route) carries the echo for real. An
|
||||
# outbound that does not (vless/trojan/shadowsocks) makes the ping DROP —
|
||||
# honestly: no reply is forged, ping reports loss. So a ping that used to
|
||||
# "work" through such a node was a ping that was leaking.
|
||||
#
|
||||
# It costs a permanent TUN device plus the gVisor netstack behind it, about
|
||||
# 2 MB of RSS for as long as the daemon runs.
|
||||
#
|
||||
# Set to '0' to opt out — worth it on a 32/64 MB router, or to bisect whether
|
||||
# the L3 ingress is what broke something. `shaterd apply` will tell you what
|
||||
# the off state costs. Your explicit value is never overwritten: this file is
|
||||
# a conffile and the daemon always writes the option back as '1'/'0'.
|
||||
#
|
||||
# NOTE the interaction: with this ON, `untunnelable` no longer governs ping at
|
||||
# all (the L3 route decision happens before the firewall chain its verdicts
|
||||
# live in). It still governs ESP/AH/GRE/IGMP/SCTP, which no tunnel of ours can
|
||||
# carry. `untunnelable 'icmp'` in particular stops meaning "block plus working
|
||||
# ping" and is reported as such.
|
||||
option l3_tunnel '1'
|
||||
option ipv6 '1'
|
||||
# Reserved fwmark base and routing-table base (do not overlap fw4/other apps).
|
||||
option fwmark_base '0x2000'
|
||||
|
||||
@@ -76,9 +76,11 @@ fi
|
||||
#
|
||||
# Idempotency: `config zone`/`config forwarding` are normally ANONYMOUS
|
||||
# sections, and a naive `uci add firewall zone` would append a duplicate on
|
||||
# every re-run (uci-defaults re-run on package upgrade/reinstall). All sections
|
||||
# here are NAMED instead, guarded by an existence check — a re-run re-finds the
|
||||
# section and touches nothing.
|
||||
# every re-run (uci-defaults re-run on package upgrade/reinstall). The zone is
|
||||
# NAMED instead, guarded by an existence check — a re-run re-finds the section
|
||||
# and touches nothing. The forwardings are named too where the name is free, but
|
||||
# their guard is a scan of the actual src/dest pairs, which is stronger; see
|
||||
# seed_l3_forwarding below.
|
||||
seed_l3_zone() {
|
||||
# No fw4 on this image (bare nftables build) => nothing drops the forward
|
||||
# on fw4's behalf and there is nothing to punch through.
|
||||
@@ -122,13 +124,103 @@ seed_l3_zone() {
|
||||
# every slot and no firewall reload is needed when the slot changes.
|
||||
uci add_list firewall.shater_l3.device='shater-l3*'
|
||||
fi
|
||||
if ! uci -q get firewall.shater_l3_fwd >/dev/null; then
|
||||
uci set firewall.shater_l3_fwd=forwarding
|
||||
uci set firewall.shater_l3_fwd.src='lan'
|
||||
uci set firewall.shater_l3_fwd.dest='shater_l3'
|
||||
fi
|
||||
seed_l3_forwardings
|
||||
uci -q commit firewall
|
||||
}
|
||||
|
||||
# EVERY zone gets a forwarding into shater_l3, not just `lan`.
|
||||
#
|
||||
# The bug this closes is silent by construction. The daemon's divert set is built
|
||||
# from every enabled `config inbound`'s network PLUS every device a rule names
|
||||
# through an `iface:`/`zone:` source (shater/netplane/nft.go, nftDivertRefs) — so
|
||||
# on a router with several LAN zones, ICMP from ALL of them is marked and routed
|
||||
# into the TUN by our table. Our table then accepts it and fw4 drops it anyway,
|
||||
# because the forward is judged in `forward_<source zone>` and only `lan` had a
|
||||
# jump to `accept_to_shater_l3`. Result: ping through the tunnel works from one
|
||||
# subnet and not from the next, with nothing in any log to say why — fw4's drop
|
||||
# is the zone's policy verdict, not a rule with a name. The owner's production
|
||||
# router has a single LAN zone, which is exactly why this went unnoticed; his
|
||||
# second router has three.
|
||||
#
|
||||
# Every zone, including an uplink zone, and that is deliberate rather than lazy:
|
||||
#
|
||||
# - The alternative is guessing which zones hold clients, and every available
|
||||
# signal is wrong somewhere. `masq='1'` marks the WAN on a stock config and
|
||||
# also marks a double-NAT LAN. The name `wan*` is a convention, not a rule.
|
||||
# A guess that is wrong reintroduces exactly the silent breakage above, while
|
||||
# a superfluous entry costs a line of ruleset.
|
||||
# - A forwarding into shater_l3 permits nothing on its own. It authorises the
|
||||
# forward of packets ROUTED INTO the TUN, and the only thing that routes a
|
||||
# packet there is our own fwmark rule, which matches solely on the divert
|
||||
# device set. A packet arriving on the WAN is not marked and never reaches
|
||||
# this decision; if an operator ever puts a WAN device in the divert set,
|
||||
# they meant to and this is the entry that makes it work.
|
||||
# - The reverse direction is NOT opened: no `src shater_l3` forwarding exists,
|
||||
# so nothing comes out of the TUN into a zone by way of these sections. The
|
||||
# engine's own replies return on the conntrack `established,related accept`
|
||||
# at the top of fw4's forward chain.
|
||||
#
|
||||
# LIMIT, stated because it is not obvious: this is a SNAPSHOT. uci-defaults run
|
||||
# at first boot and on package install/upgrade, so a zone created AFTER the last
|
||||
# shater-core install has no forwarding until the next one. Re-running this
|
||||
# script (or reinstalling the package) re-seeds. The durable fix belongs in the
|
||||
# daemon, which recomputes the divert set on every apply and already knows which
|
||||
# zones are in it; it is deliberately not attempted from here.
|
||||
seed_l3_forwardings() {
|
||||
uci -q show firewall 2>/dev/null |
|
||||
sed -n "s/^firewall\.\([^.=]*\)=zone\$/\1/p" |
|
||||
while read -r sid; do
|
||||
zone=$(uci -q get "firewall.$sid.name")
|
||||
# Unnamed zone: fw4 cannot reference it from a forwarding either.
|
||||
[ -n "$zone" ] || continue
|
||||
# Our own zone: a forwarding from shater_l3 to itself is meaningless.
|
||||
[ "$zone" = "shater_l3" ] && continue
|
||||
seed_l3_forwarding "$zone"
|
||||
done
|
||||
}
|
||||
|
||||
# One `config forwarding` <zone> -> shater_l3, created only if no such forwarding
|
||||
# exists yet.
|
||||
#
|
||||
# The guard scans the ACTUAL src/dest pairs rather than trusting a section id,
|
||||
# which covers all three ways one can already be there: the legacy named section
|
||||
# `shater_l3_fwd` seeded by earlier releases (src=lan), the per-zone names this
|
||||
# function writes, and an anonymous one an operator added by hand. Without that,
|
||||
# a re-run — uci-defaults re-run on every package upgrade — would append a
|
||||
# duplicate for `lan` on every upgrade.
|
||||
seed_l3_forwarding() {
|
||||
local zone="$1" sid found
|
||||
|
||||
found=$(uci -q show firewall 2>/dev/null |
|
||||
sed -n "s/^firewall\.\([^.=]*\)=forwarding\$/\1/p" |
|
||||
while read -r f; do
|
||||
[ "$(uci -q get "firewall.$f.dest")" = "shater_l3" ] || continue
|
||||
[ "$(uci -q get "firewall.$f.src")" = "$zone" ] || continue
|
||||
echo yes
|
||||
break
|
||||
done)
|
||||
[ -n "$found" ] && return 0
|
||||
|
||||
# Section ids are [a-zA-Z0-9_] only, while a zone name may legally carry a
|
||||
# hyphen — sanitise, and keep the legacy id for `lan` so an existing install
|
||||
# is recognised as already seeded rather than gaining a second section.
|
||||
if [ "$zone" = "lan" ]; then
|
||||
sid="shater_l3_fwd"
|
||||
else
|
||||
sid="shater_l3_fwd_$(printf '%s' "$zone" | sed 's/[^a-zA-Z0-9_]/_/g')"
|
||||
fi
|
||||
# The id may still be taken — by a section for a DIFFERENT zone whose name
|
||||
# sanitises to the same thing, or by something else entirely. Fall back to an
|
||||
# anonymous section rather than overwrite: the src/dest scan above is what
|
||||
# makes this idempotent, the name is only there to be readable.
|
||||
if uci -q get "firewall.$sid" >/dev/null; then
|
||||
sid=$(uci add firewall forwarding) || return 0
|
||||
else
|
||||
uci set "firewall.$sid=forwarding"
|
||||
fi
|
||||
uci set "firewall.$sid.src=$zone"
|
||||
uci set "firewall.$sid.dest=shater_l3"
|
||||
}
|
||||
seed_l3_zone
|
||||
|
||||
# Upgrade path for routers seeded by a pre-slot build.
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
#!/bin/sh
|
||||
# scripts/testbed-lao.sh — add a SECOND LAN network ("lao", 10.67.1.0/24) in its
|
||||
# own fw4 zone, on a testbed router.
|
||||
#
|
||||
# WHY IT EXISTS
|
||||
#
|
||||
# Almost every zone-related defect in this project is invisible on a router with
|
||||
# one LAN zone, because "the zone" and "the LAN" are the same thing there. The
|
||||
# divert set the daemon builds spans every LAN inbound and every `iface:`/`zone:`
|
||||
# rule source, so on a multi-zone router traffic from the other zones is marked,
|
||||
# routed, accepted by `inet shater` — and then dropped by fw4's zone policy,
|
||||
# silently. Reproducing that needs a second zone and nothing else: no second
|
||||
# physical port, no client, no traffic. This script makes one.
|
||||
#
|
||||
# WHY IT IS NOT SHIPPED
|
||||
#
|
||||
# It lives in scripts/ and is NOT installed by openwrt/shater-core/Makefile,
|
||||
# which lists every file it installs by name. That is the whole opt-in mechanism,
|
||||
# and it was chosen over the alternatives on purpose:
|
||||
#
|
||||
# - an extra /etc/uci-defaults/ file would run on EVERY install, handing a
|
||||
# second network and a second firewall zone to every ordinary user — the one
|
||||
# thing this must not do;
|
||||
# - an environment variable read inside 30_shater-core is unreachable in
|
||||
# practice: that script is deleted after its first successful run, so there
|
||||
# is no later moment at which an operator could set the variable and re-run it;
|
||||
# - a separate package would need a feed entry, a build, a release and a
|
||||
# version, for a file that exists to be scp'd onto one VM.
|
||||
#
|
||||
# USAGE
|
||||
#
|
||||
# scp scripts/testbed-lao.sh root@testbed:/tmp/ && ssh root@testbed sh /tmp/testbed-lao.sh
|
||||
# ssh root@testbed sh /tmp/testbed-lao.sh --remove
|
||||
#
|
||||
# It is idempotent (every section is NAMED and guarded), purely additive, and
|
||||
# touches no existing section. Running it three times in a row leaves exactly one
|
||||
# of everything.
|
||||
|
||||
set -e
|
||||
|
||||
REMOVE=0
|
||||
[ "$1" = "--remove" ] && REMOVE=1
|
||||
|
||||
if [ "$REMOVE" = 1 ]; then
|
||||
uci -q delete firewall.lao_fwd
|
||||
uci -q delete firewall.lao
|
||||
uci -q delete dhcp.lao
|
||||
uci -q delete network.lao
|
||||
uci -q delete network.br_lao
|
||||
uci -q commit firewall
|
||||
uci -q commit dhcp
|
||||
uci -q commit network
|
||||
/etc/init.d/network reload
|
||||
/etc/init.d/firewall reload
|
||||
echo "lao removed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --- L2: an empty bridge -----------------------------------------------------
|
||||
# No ports on purpose: the point is a second ROUTED network with its own firewall
|
||||
# zone, and giving it a switch port would mean re-cabling a testbed for nothing.
|
||||
#
|
||||
# bridge_empty is what makes a portless bridge usable. Without it netifd leaves a
|
||||
# member-less bridge down (no carrier), the `lao` interface never comes up, fw4
|
||||
# resolves `list network 'lao'` to an EMPTY device set, and the zone silently
|
||||
# matches nothing — which would make this script a worse instrument than no
|
||||
# instrument, since it would look set up and prove nothing.
|
||||
if ! uci -q get network.br_lao >/dev/null; then
|
||||
uci set network.br_lao=device
|
||||
uci set network.br_lao.name='br-lao'
|
||||
uci set network.br_lao.type='bridge'
|
||||
uci set network.br_lao.bridge_empty='1'
|
||||
fi
|
||||
|
||||
# --- L3: the interface -------------------------------------------------------
|
||||
# 10.67.1.0/24 is deliberately far from anything a home LAN or a proxy node uses.
|
||||
# ipaddr+netmask rather than CIDR: CIDR in `ipaddr` is a 25.12 convenience and
|
||||
# this script should also run on an older testbed image.
|
||||
if ! uci -q get network.lao >/dev/null; then
|
||||
uci set network.lao=interface
|
||||
uci set network.lao.proto='static'
|
||||
uci set network.lao.device='br-lao'
|
||||
uci set network.lao.ipaddr='10.67.1.1'
|
||||
uci set network.lao.netmask='255.255.255.0'
|
||||
fi
|
||||
|
||||
# --- DHCP: same shape as lan -------------------------------------------------
|
||||
if ! uci -q get dhcp.lao >/dev/null; then
|
||||
uci set dhcp.lao=dhcp
|
||||
uci set dhcp.lao.interface='lao'
|
||||
uci set dhcp.lao.start='100'
|
||||
uci set dhcp.lao.limit='150'
|
||||
uci set dhcp.lao.leasetime='12h'
|
||||
fi
|
||||
|
||||
# --- Firewall: its OWN zone, which is the entire point -----------------------
|
||||
# Same policies as the stock lan zone and its own forwarding to wan, so the
|
||||
# network behaves like a second LAN. What it does NOT get here is a forwarding
|
||||
# into shater_l3: seeding that for every zone is the job under test, done by
|
||||
# /etc/uci-defaults/30_shater-core. If this script seeded it, the test would be
|
||||
# testing itself.
|
||||
if ! uci -q get firewall.lao >/dev/null; then
|
||||
uci set firewall.lao=zone
|
||||
uci set firewall.lao.name='lao'
|
||||
uci set firewall.lao.input='ACCEPT'
|
||||
uci set firewall.lao.output='ACCEPT'
|
||||
uci set firewall.lao.forward='ACCEPT'
|
||||
uci add_list firewall.lao.network='lao'
|
||||
fi
|
||||
if ! uci -q get firewall.lao_fwd >/dev/null; then
|
||||
uci set firewall.lao_fwd=forwarding
|
||||
uci set firewall.lao_fwd.src='lao'
|
||||
uci set firewall.lao_fwd.dest='wan'
|
||||
fi
|
||||
|
||||
uci commit network
|
||||
uci commit dhcp
|
||||
uci commit firewall
|
||||
|
||||
/etc/init.d/network reload
|
||||
/etc/init.d/firewall reload
|
||||
|
||||
echo "lao seeded: br-lao 10.67.1.1/24, fw4 zone lao -> wan"
|
||||
@@ -0,0 +1,187 @@
|
||||
package model
|
||||
|
||||
// Tests for the `l3_tunnel` default flip: carrying LAN ping through the tunnel
|
||||
// is now the DEFAULT, not an opt-in.
|
||||
//
|
||||
// Why it flipped, so a future reader does not "restore" the old default as a
|
||||
// safety measure: with l3_tunnel off, a LAN ping is decided by `untunnelable`
|
||||
// alone, and every rung of that ladder is either a drop (block) or a disclosure
|
||||
// (icmp/direct let the echo out of the WAN interface with the client's real
|
||||
// address). There was no configuration in which ping both worked and stayed
|
||||
// inside the tunnel. With l3_tunnel on, an L3-capable outbound carries the echo
|
||||
// and one that is not drops it honestly — adapter.JudgeFlow returns
|
||||
// tun.ActionDrop for an ICMP flow whose outbound is not a tun.Port
|
||||
// (adapter/judgeflow_icmp_lx_test.go), so no reply is ever forged. Turning it on
|
||||
// therefore breaks nothing that was working truthfully.
|
||||
//
|
||||
// Four properties are pinned, and the middle two pull against each other:
|
||||
//
|
||||
// 1. the seed and an option-less config both come back ON (every install
|
||||
// written before the option existed gains the ingress on upgrade);
|
||||
// 2. an EXPLICIT `option l3_tunnel '0'` stays OFF across a WriteUCI->ReadUCI
|
||||
// round-trip — a bool omitted at false would be re-read as the seed and
|
||||
// silently flip the operator's decision back on;
|
||||
// 3. the SHIPPED /etc/config/shater agrees with the seed (it is a conffile: a
|
||||
// fresh install's posture is that file's literal text, not DefaultGlobals);
|
||||
// 4. neither the off state nor the combinations the flip made misleading are
|
||||
// reached in silence — ValidateGlobals reports all three.
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestL3TunnelDefaultOn: the seed and an option-less config both say ON.
|
||||
func TestL3TunnelDefaultOn(t *testing.T) {
|
||||
if !DefaultGlobals().L3Tunnel {
|
||||
t.Fatal("DefaultGlobals().L3Tunnel = false, want true — with the L3 ingress off a LAN ping is either dropped or sent out of the WAN with the client's real address, and neither is what a tunnel is for")
|
||||
}
|
||||
m, err := ParseUCIExport("package shater\n\nconfig globals 'globals'\n\toption enabled '1'\n")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !m.Globals.L3Tunnel {
|
||||
t.Fatal("a config with no l3_tunnel option parsed as OFF; an absent option must fall back to the ON seed, or every install predating the option keeps leaking its pings")
|
||||
}
|
||||
}
|
||||
|
||||
// TestL3TunnelExplicitOffPreserved: the operator's opt-out survives both the
|
||||
// parse (over an ON seed) and the render->parse round-trip. Off is a supported
|
||||
// answer — a 32/64 MB router where the standing TUN + gVisor netstack costs real
|
||||
// money, or a bisection — so it has to be a decision the config keeps.
|
||||
func TestL3TunnelExplicitOffPreserved(t *testing.T) {
|
||||
m, err := ParseUCIExport("package shater\n\nconfig globals 'globals'\n" +
|
||||
"\toption enabled '1'\n\toption l3_tunnel '0'\n")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if m.Globals.L3Tunnel {
|
||||
t.Fatal("explicit l3_tunnel '0' was overwritten by the ON seed")
|
||||
}
|
||||
|
||||
rendered := RenderUCIExport(m)
|
||||
if !strings.Contains(rendered, "option l3_tunnel '0'") {
|
||||
t.Fatalf("render must emit the explicit '0' (an omitted bool would be re-read as ON):\n%s", rendered)
|
||||
}
|
||||
back, err := ParseUCIExport(rendered)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if back.Globals.L3Tunnel {
|
||||
t.Fatal("l3_tunnel '0' did not survive the WriteUCI->ReadUCI round-trip — the opt-out would be silently re-enabled on the next save")
|
||||
}
|
||||
}
|
||||
|
||||
// TestShippedConfigEnablesL3Tunnel reads the file the package actually installs.
|
||||
// /etc/config/shater is a CONFFILE: written once on first install and never
|
||||
// replaced on upgrade, so a fresh install's ping posture is decided by this
|
||||
// file's literal text and not by DefaultGlobals. The two must agree, and only a
|
||||
// test that reads the shipped bytes can say that they do.
|
||||
func TestShippedConfigEnablesL3Tunnel(t *testing.T) {
|
||||
path := filepath.Join("..", "..", "openwrt", "shater-core", "files", "etc", "config", "shater")
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("shipped config %s is unreadable (%v) — this test must not be skipped: it is the only check that the installed file and DefaultGlobals agree", path, err)
|
||||
}
|
||||
text := string(raw)
|
||||
// Line-wise and comment-aware on purpose. That file is more than half
|
||||
// commented-out examples, so a plain strings.Contains would be satisfied by
|
||||
// `#option l3_tunnel '1'` — and the parse below cannot tell the difference
|
||||
// either, because a commented option falls back to the seed, which is now
|
||||
// also true. Nothing else in this test can catch that, so this check has to.
|
||||
live := false
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
if strings.TrimSpace(line) == "option l3_tunnel '1'" {
|
||||
live = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !live {
|
||||
t.Error("the shipped /etc/config/shater must carry an UNCOMMENTED `option l3_tunnel '1'`: a fresh install reads this file, and an operator who later opts out must be able to see the option they are flipping")
|
||||
}
|
||||
m, err := ParseUCIExport(text)
|
||||
if err != nil {
|
||||
t.Fatalf("shipped config does not parse: %v", err)
|
||||
}
|
||||
if !m.Globals.L3Tunnel {
|
||||
t.Error("shipped config parses with L3Tunnel off")
|
||||
}
|
||||
if m.Globals.Enabled {
|
||||
t.Error("shipped config must stay inert (enabled '0'): a fresh install may not touch connectivity")
|
||||
}
|
||||
}
|
||||
|
||||
// TestL3TunnelOffIsWarned: nobody reaches the off state by accident any more —
|
||||
// absent means ON — so a false is an explicit opt-out and must be told what it
|
||||
// costs, rather than be discovered later as "ping stopped working" or, worse and
|
||||
// silently, as "ping works, from the WAN address".
|
||||
func TestL3TunnelOffIsWarned(t *testing.T) {
|
||||
g := DefaultGlobals()
|
||||
g.L3Tunnel = false
|
||||
ws := ValidateGlobals(g)
|
||||
if !warnsMention(ws, "l3_tunnel is off") {
|
||||
t.Fatalf("turning the L3 ingress off produced no warning: %v", warningTexts(ws))
|
||||
}
|
||||
// The default policy has to be NAMED, not implied: "block" and "direct" have
|
||||
// opposite consequences for the packet and the operator cannot tell which
|
||||
// they are in from the option's absence.
|
||||
if !warnsMention(ws, "untunnelable (block)") {
|
||||
t.Fatalf("the off warning does not name the policy that takes over instead: %v", warningTexts(ws))
|
||||
}
|
||||
|
||||
// The control: with the ingress ON the same prober must stay quiet, or a
|
||||
// green "no warning" above would prove nothing about the check firing.
|
||||
if ws := ValidateGlobals(DefaultGlobals()); warnsMention(ws, "l3_tunnel is off") {
|
||||
t.Fatalf("the default (L3 ingress ON) was reported as off: %v", warningTexts(ws))
|
||||
}
|
||||
}
|
||||
|
||||
// TestL3TunnelWithUntunnelableICMPIsReported is the combination the flip made
|
||||
// misleading. `icmp` used to mean "block, plus working ping". With l3_tunnel on
|
||||
// the prerouting L3 mark claims every ICMP packet from a diverted device before
|
||||
// the forward chain where that echo accept lives, so the ping half is dead and
|
||||
// only the half the operator may not have wanted survives: ESP/AH/GRE/IGMP/SCTP
|
||||
// out with the real address toward directly-routed destinations. Under the old
|
||||
// opt-in default the pair was exotic; under the new one it is ordinary, and
|
||||
// silence about it would be the panel lying about state.
|
||||
func TestL3TunnelWithUntunnelableICMPIsReported(t *testing.T) {
|
||||
g := DefaultGlobals()
|
||||
g.Untunnelable = "icmp"
|
||||
ws := ValidateGlobals(g)
|
||||
if !warnsMention(ws, "ping no longer reaches this policy") {
|
||||
t.Fatalf("l3_tunnel=1 + untunnelable=icmp was accepted in silence: %v", warningTexts(ws))
|
||||
}
|
||||
|
||||
// Control 1: the same policy WITHOUT the L3 ingress is the state the rung was
|
||||
// designed for and must not be reported.
|
||||
g.L3Tunnel = false
|
||||
if ws := ValidateGlobals(g); warnsMention(ws, "ping no longer reaches this policy") {
|
||||
t.Fatalf("untunnelable=icmp under l3_tunnel=0 was reported as dead: %v", warningTexts(ws))
|
||||
}
|
||||
// Control 2: the L3 ingress with the DEFAULT policy is the ordinary posture
|
||||
// and must not be reported either.
|
||||
if ws := ValidateGlobals(DefaultGlobals()); warnsMention(ws, "ping no longer reaches this policy") {
|
||||
t.Fatalf("the default posture (l3_tunnel=1 + untunnelable=block) was reported: %v", warningTexts(ws))
|
||||
}
|
||||
}
|
||||
|
||||
// warnsMention reports whether any warning text contains sub.
|
||||
func warnsMention(ws []Warning, sub string) bool {
|
||||
for _, w := range ws {
|
||||
if strings.Contains(w.Message, sub) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// warningTexts flattens warnings for a failure message.
|
||||
func warningTexts(ws []Warning) []string {
|
||||
out := make([]string, 0, len(ws))
|
||||
for _, w := range ws {
|
||||
out = append(out, w.Message)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -87,14 +87,19 @@ func TestLogKnobsAliases(t *testing.T) {
|
||||
|
||||
// TestValidateGlobalsLogMaxKB: out-of-range positive values warn (they are
|
||||
// applied clamped by the sink); 0 and in-range values do not.
|
||||
//
|
||||
// Every fixture carries L3Tunnel: true — the DEFAULT since the L3 ingress became
|
||||
// opt-out. Left at the Go zero value it would read as an explicit opt-out and
|
||||
// draw a warning of its own (TestL3TunnelOffIsWarned), which has nothing to do
|
||||
// with the log budget this test is about.
|
||||
func TestValidateGlobalsLogMaxKB(t *testing.T) {
|
||||
for _, ok := range []int{0, LogMaxKBMin, 2048, LogMaxKBMax} {
|
||||
if ws := ValidateGlobals(Globals{LogMaxKB: ok}); len(ws) != 0 {
|
||||
if ws := ValidateGlobals(Globals{LogMaxKB: ok, L3Tunnel: true}); len(ws) != 0 {
|
||||
t.Errorf("log_max_kb=%d should not warn: %v", ok, ws)
|
||||
}
|
||||
}
|
||||
for _, bad := range []int{1, LogMaxKBMin - 1, LogMaxKBMax + 1, 1 << 20} {
|
||||
ws := ValidateGlobals(Globals{LogMaxKB: bad})
|
||||
ws := ValidateGlobals(Globals{LogMaxKB: bad, L3Tunnel: true})
|
||||
if len(ws) != 1 || !strings.Contains(ws[0].Message, "log_max_kb") {
|
||||
t.Errorf("log_max_kb=%d should warn once about the clamp, got %v", bad, ws)
|
||||
}
|
||||
|
||||
+44
-3
@@ -260,7 +260,7 @@ type Globals struct {
|
||||
// current behaviour exactly.
|
||||
Untunnelable string
|
||||
|
||||
// L3Tunnel opts the router into the L3 ingress: the engine opens a TUN device
|
||||
// L3Tunnel puts the router on the L3 ingress: the engine opens a TUN device
|
||||
// (netplane.L3Device) and the nft prerouting chain policy-routes LAN traffic
|
||||
// the tunnel can only carry at layer 3 into it. There the engine's ordinary
|
||||
// route rules decide the outbound, and an L3-capable one (wireguard/AWG —
|
||||
@@ -279,8 +279,37 @@ type Globals struct {
|
||||
// have. Those stay governed by Untunnelable and UntunnelableEgress below. Ping
|
||||
// and Windows `tracert` through the tunnel are the whole of what this buys.
|
||||
//
|
||||
// Default false: without it nothing changes, and Untunnelable alone decides
|
||||
// what happens to non-TCP/UDP traffic.
|
||||
// DEFAULT TRUE (opt-out). The reason it flipped is that "off" had no honest
|
||||
// win left in it. Off, a LAN ping is decided by Untunnelable alone, and each
|
||||
// of the three rungs is either a drop or a leak: `block` drops it, `icmp` and
|
||||
// `direct` let the echo out of the WAN interface with the client's real
|
||||
// address on it. "Ping works" was never a state where ping was tunnelled — it
|
||||
// was the state where ping was disclosing. On, an outbound that carries L3
|
||||
// (WireGuard/AmneziaWG, or `direct`) carries the echo for real, and an
|
||||
// outbound that does not (vless/trojan/ss) drops it: the packet enters the
|
||||
// TUN, sing-tun's dispatcher finds no FlowOutbound on the selected route and
|
||||
// nothing goes out. That drop is HONEST — verified on hardware, no reply is
|
||||
// forged, ping simply reports loss — so turning this on breaks nothing that
|
||||
// was working truthfully and closes what was leaking.
|
||||
//
|
||||
// The price is not zero and is not hidden: a permanent TUN device plus the
|
||||
// gVisor netstack behind it, ~2 MB RSS on the routers this runs on, for as
|
||||
// long as the daemon is up.
|
||||
//
|
||||
// Off is therefore still a supported answer, not a vestige — on a 32/64 MB
|
||||
// device where 2 MB is real money, and when bisecting whether the L3 ingress
|
||||
// is what broke a box. It is a WARNED answer: ValidateGlobals reports what
|
||||
// turning it off costs, so nobody reaches the off state without being told
|
||||
// what ping does there (validate.go, and see UntunnelableEgress below for the
|
||||
// protocols this cannot help either way).
|
||||
//
|
||||
// Interaction with Untunnelable, which the validator also reports: with this
|
||||
// ON the L3 mark is stamped in prerouting and the routing decision carries
|
||||
// ICMP into the TUN before the forward chain — where Untunnelable's verdicts
|
||||
// live — is ever consulted. So Untunnelable no longer governs ping at all; it
|
||||
// keeps governing exactly the protocols the dispatcher will not take (ESP,
|
||||
// AH, GRE, IGMP, SCTP) and, under `icmp`, the ICMP ERRORS a LAN host raises,
|
||||
// which are marked into the TUN with the echoes and are not dispatched there.
|
||||
L3Tunnel bool
|
||||
|
||||
// UntunnelableEgress names an egress of type interface/tunnel that carries the
|
||||
@@ -382,6 +411,17 @@ const (
|
||||
// the clients that were trying to evade the router (see the field comment). An
|
||||
// EXPLICIT `option dns_intercept '0'` parses over this seed and is preserved:
|
||||
// render.go always emits the bool, so opting out is a decision the config keeps.
|
||||
//
|
||||
// L3Tunnel is seeded ON (opt-out) for the same reason and by the same mechanism.
|
||||
// A config that never mentions `l3_tunnel` — every install predating the option —
|
||||
// gains the L3 ingress on upgrade, and that is the intent: without it a LAN ping
|
||||
// is either dropped (Untunnelable "block", the default) or sent out of the WAN
|
||||
// with the client's real address ("icmp"/"direct"), so there was no configuration
|
||||
// in which ping both worked and stayed inside the tunnel. With it, ping is
|
||||
// carried by an L3-capable outbound and honestly dropped by one that is not — no
|
||||
// reply is ever forged. The cost is a standing TUN + gVisor netstack, ~2 MB RSS.
|
||||
// An EXPLICIT `option l3_tunnel '0'` parses over this seed and is preserved
|
||||
// (render.go always emits the bool) and ValidateGlobals says what it costs.
|
||||
func DefaultGlobals() Globals {
|
||||
return Globals{
|
||||
Enabled: true,
|
||||
@@ -391,6 +431,7 @@ func DefaultGlobals() Globals {
|
||||
LogMaxKB: LogMaxKBDefault,
|
||||
KillSwitch: "closed",
|
||||
Untunnelable: "block",
|
||||
L3Tunnel: true,
|
||||
IPv6: true,
|
||||
FwmarkBase: 0x2000,
|
||||
TableBase: 0x2000,
|
||||
|
||||
@@ -21,7 +21,7 @@ func richModel() *Model {
|
||||
LogLevel: "debug",
|
||||
KillSwitch: "open",
|
||||
Untunnelable: "direct",
|
||||
L3Tunnel: true, // opt-in; exercise the non-default via round-trip
|
||||
L3Tunnel: false, // opt-out; exercise the non-default via round-trip
|
||||
UntunnelableEgress: "frag", // exercise the non-default via round-trip
|
||||
IPv6: false,
|
||||
FwmarkBase: 0x2000,
|
||||
@@ -314,9 +314,11 @@ func TestGroupHealthRoundTrip(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestL3TunnelRoundTrip pins the l3_tunnel opt-in: an EXPLICIT true (L3 ingress
|
||||
// on) survives WriteUCI->ReadUCI, and an ABSENT option stays false (opt-in,
|
||||
// never accidentally on).
|
||||
// TestL3TunnelRoundTrip pins the l3_tunnel round-trip: BOTH spellings survive
|
||||
// WriteUCI->ReadUCI, and an ABSENT option comes back ON — the option is opt-OUT
|
||||
// now, so the interesting direction is that an explicit '0' is not silently
|
||||
// re-enabled. The default itself, the shipped config and the warnings that go
|
||||
// with the off state are pinned in l3tunnel_default_test.go.
|
||||
func TestL3TunnelRoundTrip(t *testing.T) {
|
||||
for _, v := range []bool{true, false} {
|
||||
g := DefaultGlobals()
|
||||
@@ -329,13 +331,13 @@ func TestL3TunnelRoundTrip(t *testing.T) {
|
||||
t.Fatalf("v=%v round-trip: l3_tunnel=%v, want %v", v, got.Globals.L3Tunnel, v)
|
||||
}
|
||||
}
|
||||
// Absent option => off (opt-in default, the Go zero value).
|
||||
// Absent option => ON (the DefaultGlobals seed, opt-out).
|
||||
got, err := ParseUCIExport("package shater\n")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Globals.L3Tunnel {
|
||||
t.Fatalf("absent l3_tunnel = %v, want false (default, opt-in)", got.Globals.L3Tunnel)
|
||||
if !got.Globals.L3Tunnel {
|
||||
t.Fatalf("absent l3_tunnel = %v, want true (default, opt-out)", got.Globals.L3Tunnel)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -342,6 +342,44 @@ func ValidateGlobals(g Globals) []Warning {
|
||||
"instead of failing. With \"block\" the same failure is honest packet loss.")
|
||||
}
|
||||
|
||||
// The same collision, one rung down, and it went unreported until l3_tunnel
|
||||
// became the default and made the pair ORDINARY rather than exotic. `icmp`
|
||||
// buys two things: echo everywhere, and the other untunnelable protocols
|
||||
// toward destinations the routing rules send direct. With l3_tunnel on the
|
||||
// FIRST of those is dead — the prerouting L3 mark claims every ICMP/ICMPv6
|
||||
// packet from a diverted device before the forward chain where the echo
|
||||
// accept lives — so the rung is not "block plus ping" any more, and an
|
||||
// operator who picked it FOR the ping is now getting only the ESP/AH/GRE
|
||||
// half they may not have wanted. Worse, the mark is not selective: a LAN
|
||||
// host's ICMP ERRORS (fragmentation-needed and friends) are marked into the
|
||||
// TUN too, where sing-tun's dispatcher classifies echo alone and drops the
|
||||
// rest — so `icmp` no longer delivers those either, which `block` never
|
||||
// promised but `icmp` did.
|
||||
if g.L3Tunnel && strings.ToLower(strings.TrimSpace(g.Untunnelable)) == "icmp" {
|
||||
add("l3_tunnel is on, so ping no longer reaches this policy at all — echo is routed " +
|
||||
"into the tunnel in prerouting, before the forward chain where \"icmp\" would have " +
|
||||
"let it out. What \"icmp\" still does is let the protocols the tunnel cannot carry " +
|
||||
"(IPsec ESP/AH, PPTP/GRE, IGMP, SCTP) out with your real IP address toward " +
|
||||
"directly-routed destinations, and it no longer delivers a LAN host's ICMP error " +
|
||||
"packets (path-MTU discovery on a direct route), which are routed into the tunnel " +
|
||||
"with the echoes and dropped there. If you chose \"icmp\" for working ping, " +
|
||||
"\"block\" now gives you that and discloses less.")
|
||||
}
|
||||
|
||||
// Nobody arrives at l3_tunnel=0 by accident any more: absent means ON
|
||||
// (DefaultGlobals), so a false here is either an explicit opt-out or a
|
||||
// config the daemon rewrote after one. Say what it costs, once, rather than
|
||||
// let the operator discover it as "ping stopped working" or — worse, and
|
||||
// silently — as "ping works, from the WAN address".
|
||||
if !g.L3Tunnel {
|
||||
add("l3_tunnel is off: LAN ping and Windows tracert do not travel through the tunnel. " +
|
||||
"What happens to them is decided by untunnelable (" + untunnelableName(g.Untunnelable) +
|
||||
") instead — \"block\" drops them, \"icmp\" and \"direct\" let them out of the WAN " +
|
||||
"interface with the client's real IP address on them. Turning l3_tunnel back on " +
|
||||
"costs a TUN device and roughly 2 MB of RAM and is the only setting in which ping " +
|
||||
"both works and stays inside the tunnel.")
|
||||
}
|
||||
|
||||
switch strings.ToLower(strings.TrimSpace(g.StatsBackend)) {
|
||||
case "", "off", "memory", "sqlite":
|
||||
default:
|
||||
@@ -352,6 +390,20 @@ func ValidateGlobals(g Globals) []Warning {
|
||||
return out
|
||||
}
|
||||
|
||||
// untunnelableName spells Globals.Untunnelable the way the data plane reads it,
|
||||
// for use inside a diagnostic: empty — and anything unrecognised, which is
|
||||
// reported on its own line above — is "block", the same normalisation
|
||||
// netplane.EffectiveUntunnelable performs (netplane imports model, so the
|
||||
// function itself cannot be shared).
|
||||
func untunnelableName(policy string) string {
|
||||
switch p := strings.ToLower(strings.TrimSpace(policy)); p {
|
||||
case "icmp", "direct":
|
||||
return p
|
||||
default:
|
||||
return "block"
|
||||
}
|
||||
}
|
||||
|
||||
// ValidateSubscriptions reports subscription options that are accepted but inert.
|
||||
func ValidateSubscriptions(subs []Subscription) []Warning {
|
||||
var out []Warning
|
||||
|
||||
@@ -112,7 +112,10 @@ func TestValidateRulesWithoutFirewallInfo(t *testing.T) {
|
||||
// apply" — a router that will not start is worse than one with a warning, because
|
||||
// the kill-switch is closed while the engine is down.
|
||||
func TestValidateIsFailOpen(t *testing.T) {
|
||||
m := &Model{Rules: []Rule{
|
||||
// Globals at the DEFAULT posture (L3Tunnel on): the zero value would read as
|
||||
// an explicit opt-out and add a globals warning to the per-rule count below,
|
||||
// which is not what this test measures.
|
||||
m := &Model{Globals: Globals{L3Tunnel: true}, Rules: []Rule{
|
||||
{Name: "bad", Enabled: true, Src: []string{"iface:wan", "iface:lan"}},
|
||||
{Name: "worse", Enabled: true, Src: []string{"zone:wan", "iface:wan6"}},
|
||||
{Name: "good", Enabled: true, Src: []string{"iface:lan"}},
|
||||
@@ -131,9 +134,19 @@ func TestValidateIsFailOpen(t *testing.T) {
|
||||
if after := RenderUCIExport(m); after != before {
|
||||
t.Error("Validate must not mutate the model")
|
||||
}
|
||||
// Nil model / no rules: no panic, no warnings.
|
||||
if len((*Model)(nil).Validate(nil)) != 0 || len((&Model{}).Validate(nil)) != 0 {
|
||||
t.Error("empty and nil models must validate cleanly")
|
||||
// Nil model: no panic, no warnings, and no classifier needed.
|
||||
if len((*Model)(nil).Validate(nil)) != 0 {
|
||||
t.Error("a nil model must validate cleanly")
|
||||
}
|
||||
// An EMPTY model is not a clean one any more, and deliberately so: its zero
|
||||
// Globals say l3_tunnel off, which is a real posture with a real cost and is
|
||||
// reported. What must still hold is that nothing is invented about the
|
||||
// sections it does not have — no rule, egress, group, alert or subscription
|
||||
// warning can come out of a model with none of them.
|
||||
for _, w := range (&Model{}).Validate(nil) {
|
||||
if w.Section != "globals" {
|
||||
t.Errorf("empty model produced a %s warning out of nothing: %v", w.Section, w)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -183,7 +196,11 @@ func TestValidateGlobalsAcceptsEveryWorkingLevel(t *testing.T) {
|
||||
levels = append(levels, SilentLogLevels...)
|
||||
for _, lvl := range levels {
|
||||
for _, spelling := range []string{lvl, strings.ToUpper(lvl), " " + lvl + " "} {
|
||||
if ws := ValidateGlobals(Globals{LogLevel: spelling}); len(ws) != 0 {
|
||||
// L3Tunnel is set to its DEFAULT here rather than left at the Go zero
|
||||
// value: the default is now ON, so a false is an explicit opt-out and
|
||||
// draws a warning of its own (TestL3TunnelOffIsWarned). This test is
|
||||
// about log levels, so it starts from the default posture.
|
||||
if ws := ValidateGlobals(Globals{LogLevel: spelling, L3Tunnel: true}); len(ws) != 0 {
|
||||
t.Errorf("level %q must not warn, got %v", spelling, ws)
|
||||
}
|
||||
}
|
||||
@@ -212,11 +229,18 @@ func TestValidateGlobalsUnknownEnums(t *testing.T) {
|
||||
if !hasWarning(ws, "globals", "stats backend \"postgres\"") {
|
||||
t.Errorf("bad stats backend must warn (it silently falls back to memory), got %v", ws)
|
||||
}
|
||||
// The three legal untunnelable values and three legal backends stay quiet.
|
||||
// The three legal untunnelable values and three legal backends draw no
|
||||
// VOCABULARY complaint. Deliberately not "no warnings at all" any more: since
|
||||
// l3_tunnel defaults ON, "icmp" and "direct" are reported for a different
|
||||
// reason — the L3 route claims ping before the forward chain those policies
|
||||
// live in — and that report is the subject of
|
||||
// TestValidateGlobalsL3TunnelDirectConflict / TestL3TunnelWithUntunnelableICMPIsReported,
|
||||
// not of this test. Both enum checks phrase themselves as "not recognised",
|
||||
// which is what is asserted absent here.
|
||||
for _, u := range []string{"", "block", "icmp", "direct"} {
|
||||
for _, b := range []string{"", "off", "memory", "sqlite"} {
|
||||
if ws := ValidateGlobals(Globals{Untunnelable: u, StatsBackend: b}); len(ws) != 0 {
|
||||
t.Errorf("untunnelable=%q backend=%q must not warn, got %v", u, b, ws)
|
||||
if ws := ValidateGlobals(Globals{Untunnelable: u, StatsBackend: b, L3Tunnel: true}); hasWarning(ws, "globals", "not recognised") {
|
||||
t.Errorf("untunnelable=%q backend=%q must not be reported as unrecognised, got %v", u, b, ws)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -241,16 +265,23 @@ func TestValidateGlobalsL3TunnelDirectConflict(t *testing.T) {
|
||||
if ws := ValidateGlobals(Globals{L3Tunnel: true, Untunnelable: " Direct "}); !hasWarning(ws, "globals", "l3_tunnel") {
|
||||
t.Errorf("normalised \" Direct \" must still warn, got %v", ws)
|
||||
}
|
||||
// Only the combination warns: each option alone, and l3_tunnel with the
|
||||
// policies that keep the failure mode honest, stay quiet.
|
||||
// Only the combination draws THIS report: each option alone, and l3_tunnel
|
||||
// with the other policies, must not be told that "direct" is redundant.
|
||||
// Asserted by the message's own words rather than by "no warnings at all",
|
||||
// because two of these fixtures legitimately draw a DIFFERENT report now that
|
||||
// l3_tunnel defaults on — `{Untunnelable: "direct"}` has the ingress off, and
|
||||
// `{L3Tunnel: true, Untunnelable: "icmp"}` is the sibling collision one rung
|
||||
// down. Folding those into a bare len(ws)==0 would make this test fail for
|
||||
// reasons that have nothing to do with `direct`.
|
||||
const directReport = "\"direct\" no longer buys you anything for ping"
|
||||
for _, g := range []Globals{
|
||||
{Untunnelable: "direct"},
|
||||
{L3Tunnel: true},
|
||||
{L3Tunnel: true, Untunnelable: "block"},
|
||||
{L3Tunnel: true, Untunnelable: "icmp"},
|
||||
} {
|
||||
if ws := ValidateGlobals(g); len(ws) != 0 {
|
||||
t.Errorf("globals %+v must not warn, got %v", g, ws)
|
||||
if ws := ValidateGlobals(g); hasWarning(ws, "globals", directReport) {
|
||||
t.Errorf("globals %+v must not draw the direct-conflict report, got %v", g, ws)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user