fix(wireguard): stop holding AmneziaWG down behind a WireGuard hop
The guard refused to start an AmneziaWG endpoint whose detour chain reached a WireGuard one, and refused silently: not an error, just started=false, after which every dial failed with "WireGuard is not ready yet". A selector hook went further and suspended an already-working node the moment its group switched to a WireGuard member. It existed because AmneziaWG inside WireGuard hung the kernel on Android. We do not ship Android, upstream dropped the guard once the cause was gone, and the cure landed here yesterday — the ClientBind reserved-gate plus the submodule pin that carries its twin. So the tree held both the cure and the prohibition on using it, and the configuration simply did not come up while looking like a node that "just does not work". Also takes the two fixes that belong with it. ClientBind.conn was read on a lock-free fast path and written under a mutex; upstream found that race with the same end-to-end test we wrote yesterday, so we had taken one half of a pair again. And the outer WireGuard UDP socket forced DF, unlike direct, hysteria and tuic — with encapsulation the datagram regularly exceeds the path MTU and the kernel drops it instead of fragmenting, a symptom indistinguishable from the bug we spent yesterday on. The race needed its own test: the existing e2e run did not flag it under -race even at -count=15. Eight goroutines over both connect branches reproduce it deterministically, naming the lock-free read and the guarded write. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -3,7 +3,33 @@
|
||||
| Поле | Значение |
|
||||
|------|----------|
|
||||
| Тип | B (bug) |
|
||||
| Статус | C (complete) |
|
||||
| Статус | C (complete) — guard **снят** (см. баннер ниже) |
|
||||
|
||||
> ## ⛔️ Guard снят (2026-07-26) — первопричина к shater не относится
|
||||
> **Оба guard'а (Start-guard в `protocol/wireguard/endpoint.go` и
|
||||
> selector-guard в `protocol/group/awg_selector_guard.go`) удалены**, вместе с
|
||||
> их adapter-хуками (`OutboundManager.ConsumersOf`, `AmneziaWGSuspendable`).
|
||||
> Апстрим снял их коммитом `5fa3a0a17`; сюда снятие приехало отдельно.
|
||||
>
|
||||
> **Почему.** Зависание было **Android-специфичным** (`Libbox.newService` не
|
||||
> возвращал управление). Android для shater не платформа и ей не станет —
|
||||
> мы собираем роутерный бинарь под OpenWrt/aarch64. При этом лекарство для
|
||||
> самой AWG-за-detour связки у нас уже есть: reserved-clear gate в
|
||||
> `ClientBind` (`d971eb85e` + пин сабмодуля `7d15f33`), без которого AWG не
|
||||
> поднимался вообще ни за каким detour'ом. Мы носили и лекарство, и запрет
|
||||
> на его применение.
|
||||
>
|
||||
> **Чем это было плохо на практике.** Guard отказывал **молча**: не ошибкой,
|
||||
> а `started=false`, после чего каждый дозвон падал с «WireGuard is not ready
|
||||
> yet». Конфигурация «AmneziaWG за WireGuard-хопом» выглядела не как
|
||||
> отклонённая, а как «нода почему-то не работает».
|
||||
>
|
||||
> **Регрессия:** `protocol/wireguard/awg_over_wireguard_start_lx_test.go`
|
||||
> (`with_gvisor && with_awg`) — AWG-эндпоинт с `detour` на outbound типа
|
||||
> `wireguard` доходит до PostStart и поднимает `started`. До снятия guard'а
|
||||
> тест краснел.
|
||||
>
|
||||
> **Осталось:** сквозной прогон на железе (AWG поверх реального WG-хопа).
|
||||
|
||||
Отклонять (по образцу ядрового запрета «empty direct detour») конфигурацию, где
|
||||
AmneziaWG-endpoint (источник с AWG-полями) имеет `detour` на **любой
|
||||
|
||||
@@ -45,30 +45,8 @@ type OutboundManager interface {
|
||||
Default() Outbound
|
||||
Remove(tag string) error
|
||||
Create(ctx context.Context, router Router, logger log.ContextLogger, tag string, outboundType string, options any) error
|
||||
// lx:begin awg
|
||||
// ConsumersOf returns the tags of outbounds that depend on (detour through)
|
||||
// the given tag — the reverse of Dependencies(). Used by the selector guard to
|
||||
// walk up to AmneziaWG consumers when a group switches to a WireGuard member.
|
||||
ConsumersOf(tag string) []string
|
||||
// lx:end awg
|
||||
}
|
||||
|
||||
// lx:begin awg
|
||||
// AmneziaWGSuspendable is implemented by an AmneziaWG endpoint so the selector
|
||||
// guard can suspend it (bring its device down) when a group it detours through
|
||||
// switches to a WireGuard member — AmneziaWG inside a WireGuard tunnel hangs the
|
||||
// kernel on Android. The marker lives in adapter so protocol/group can act on it
|
||||
// without importing protocol/wireguard.
|
||||
type AmneziaWGSuspendable interface {
|
||||
// IsAmneziaWG reports whether this endpoint runs AmneziaWG (has AWG params).
|
||||
IsAmneziaWG() bool
|
||||
// SuspendAmneziaWG brings the device down so no junk handshake is sent. It is
|
||||
// idempotent and safe to call on a not-yet-started or already-suspended endpoint.
|
||||
SuspendAmneziaWG()
|
||||
}
|
||||
|
||||
// lx:end awg
|
||||
|
||||
// lx:begin idle-suspend
|
||||
// IdleSuspendable is implemented by a WG/AWG endpoint so the router's idle tick
|
||||
// (SPEC 020) can suspend it when it is idle and unreachable, without importing
|
||||
|
||||
@@ -208,21 +208,6 @@ func (m *Manager) Outbound(tag string) (adapter.Outbound, bool) {
|
||||
return m.endpoint.Get(tag)
|
||||
}
|
||||
|
||||
// lx:begin awg
|
||||
// ConsumersOf returns a copy of the tags that detour through tag (reverse of
|
||||
// Dependencies()), built from the dependByTag ledger populated at Create time.
|
||||
func (m *Manager) ConsumersOf(tag string) []string {
|
||||
m.access.RLock()
|
||||
defer m.access.RUnlock()
|
||||
consumers := m.dependByTag[tag]
|
||||
if len(consumers) == 0 {
|
||||
return nil
|
||||
}
|
||||
return append([]string(nil), consumers...)
|
||||
}
|
||||
|
||||
// lx:end awg
|
||||
|
||||
func (m *Manager) Default() adapter.Outbound {
|
||||
m.access.RLock()
|
||||
defer m.access.RUnlock()
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
//go:build darwin
|
||||
|
||||
package dialer
|
||||
|
||||
import (
|
||||
"syscall"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// udpSocketDFSet reports whether the socket has "don't fragment" forced on
|
||||
// (control.DisableUDPFragment sets IP_DONTFRAG=1 on darwin).
|
||||
func udpSocketDFSet(t *testing.T, sysConn syscall.Conn) bool {
|
||||
t.Helper()
|
||||
rawConn, err := sysConn.SyscallConn()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var (
|
||||
value int
|
||||
sockErr error
|
||||
ctrlErr error
|
||||
)
|
||||
ctrlErr = rawConn.Control(func(fd uintptr) {
|
||||
value, sockErr = unix.GetsockoptInt(int(fd), unix.IPPROTO_IP, unix.IP_DONTFRAG)
|
||||
})
|
||||
if ctrlErr != nil {
|
||||
t.Fatal(ctrlErr)
|
||||
}
|
||||
if sockErr != nil {
|
||||
t.Fatal(sockErr)
|
||||
}
|
||||
return value != 0
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
//go:build linux
|
||||
|
||||
package dialer
|
||||
|
||||
import (
|
||||
"syscall"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// udpSocketDFSet reports whether the socket has "don't fragment" forced on
|
||||
// (control.DisableUDPFragment sets IP_MTU_DISCOVER=IP_PMTUDISC_DO on linux,
|
||||
// the same flag the user-visible failure was traced to on android).
|
||||
func udpSocketDFSet(t *testing.T, sysConn syscall.Conn) bool {
|
||||
t.Helper()
|
||||
rawConn, err := sysConn.SyscallConn()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var (
|
||||
value int
|
||||
sockErr error
|
||||
ctrlErr error
|
||||
)
|
||||
ctrlErr = rawConn.Control(func(fd uintptr) {
|
||||
value, sockErr = unix.GetsockoptInt(int(fd), unix.IPPROTO_IP, unix.IP_MTU_DISCOVER)
|
||||
})
|
||||
if ctrlErr != nil {
|
||||
t.Fatal(ctrlErr)
|
||||
}
|
||||
if sockErr != nil {
|
||||
t.Fatal(sockErr)
|
||||
}
|
||||
return value == unix.IP_PMTUDISC_DO
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
//go:build !darwin && !linux && !windows
|
||||
|
||||
package dialer
|
||||
|
||||
import (
|
||||
"syscall"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func udpSocketDFSet(t *testing.T, _ syscall.Conn) bool {
|
||||
t.Helper()
|
||||
t.Skip("DF socket-flag introspection implemented for darwin, linux and windows only")
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
//go:build windows
|
||||
|
||||
package dialer
|
||||
|
||||
import (
|
||||
"syscall"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/sys/windows"
|
||||
)
|
||||
|
||||
// IP_MTU_DISCOVER on windows (ws2ipdef.h); control.DisableUDPFragment sets it to
|
||||
// IP_PMTUDISC_DO, the same "don't fragment" state the linux helper checks.
|
||||
const (
|
||||
windowsIPMTUDiscover = 71
|
||||
windowsPMTUDiscDo = 1
|
||||
)
|
||||
|
||||
// udpSocketDFSet reports whether the socket has "don't fragment" forced on.
|
||||
// shater addition: upstream ships linux + darwin only, so the whole suite
|
||||
// skipped on the dev host — where it is the one platform we can actually run it
|
||||
// on before the router build.
|
||||
func udpSocketDFSet(t *testing.T, sysConn syscall.Conn) bool {
|
||||
t.Helper()
|
||||
rawConn, err := sysConn.SyscallConn()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var (
|
||||
value int
|
||||
sockErr error
|
||||
)
|
||||
ctrlErr := rawConn.Control(func(fd uintptr) {
|
||||
value, sockErr = windows.GetsockoptInt(windows.Handle(fd), windows.IPPROTO_IP, windowsIPMTUDiscover)
|
||||
})
|
||||
if ctrlErr != nil {
|
||||
t.Fatal(ctrlErr)
|
||||
}
|
||||
if sockErr != nil {
|
||||
t.Skip("IP_MTU_DISCOVER is not readable on this host: ", sockErr)
|
||||
}
|
||||
return value == windowsPMTUDiscDo
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
// lx: regression tests for the udp_fragment / UDPFragmentDefault
|
||||
// plumbing. The WireGuard endpoint (and MASQUE outbound) rely on
|
||||
// UDPFragmentDefault=true reaching the real UDP socket as "DF clear": with DF
|
||||
// set, an outer datagram larger than the path MTU is silently dropped instead
|
||||
// of fragmented, which blackholes nested tunnels (AWG-over-AWG, MASQUE-over-AWG)
|
||||
// and AWG s4 transport junk. These tests assert the socket flag itself, on both
|
||||
// paths a WireGuard bind can take: the dialer (ClientBind, detour case) and the
|
||||
// listener control (StdNetBind via WireGuardControl, no-detour case).
|
||||
package dialer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net"
|
||||
"syscall"
|
||||
"testing"
|
||||
|
||||
"github.com/sagernet/sing-box/option"
|
||||
M "github.com/sagernet/sing/common/metadata"
|
||||
N "github.com/sagernet/sing/common/network"
|
||||
)
|
||||
|
||||
func dialUDPForDF(t *testing.T, options option.DialerOptions) syscall.Conn {
|
||||
t.Helper()
|
||||
d, err := NewDefault(context.Background(), options)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
conn, err := d.DialContext(context.Background(), N.NetworkUDP, M.ParseSocksaddr("127.0.0.1:9"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = conn.Close() })
|
||||
sysConn, isSysConn := conn.(syscall.Conn)
|
||||
if !isSysConn {
|
||||
t.Fatalf("dialed UDP conn %T does not expose SyscallConn", conn)
|
||||
}
|
||||
return sysConn
|
||||
}
|
||||
|
||||
func listenUDPForDF(t *testing.T, options option.DialerOptions) syscall.Conn {
|
||||
t.Helper()
|
||||
d, err := NewDefault(context.Background(), options)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// WireGuardControl() is the listener control conn.StdNetBind installs on the
|
||||
// socket a no-detour WireGuard endpoint sends its outer datagrams from — the
|
||||
// exact socket the DF default decides the fate of.
|
||||
listenConfig := net.ListenConfig{Control: d.WireGuardControl()}
|
||||
packetConn, err := listenConfig.ListenPacket(context.Background(), "udp4", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = packetConn.Close() })
|
||||
sysConn, isSysConn := packetConn.(syscall.Conn)
|
||||
if !isSysConn {
|
||||
t.Fatalf("listened UDP conn %T does not expose SyscallConn", packetConn)
|
||||
}
|
||||
return sysConn
|
||||
}
|
||||
|
||||
// Upstream default: no UDPFragmentDefault, no udp_fragment → DF is set on both
|
||||
// the dial and listener paths. Pins the baseline the endpoint fix opts out of.
|
||||
func TestUDPFragmentDFByDefault_LX(t *testing.T) {
|
||||
if !udpSocketDFSet(t, dialUDPForDF(t, option.DialerOptions{})) {
|
||||
t.Fatal("default dialer must set DF on dialed UDP sockets")
|
||||
}
|
||||
if !udpSocketDFSet(t, listenUDPForDF(t, option.DialerOptions{})) {
|
||||
t.Fatal("default dialer must set DF on listener-control UDP sockets")
|
||||
}
|
||||
}
|
||||
|
||||
// UDPFragmentDefault=true (what the WireGuard endpoint and MASQUE outbound now
|
||||
// set) → DF clear on both paths, so oversize outer datagrams fragment instead
|
||||
// of vanishing.
|
||||
func TestUDPFragmentDefaultClearsDF_LX(t *testing.T) {
|
||||
options := option.DialerOptions{UDPFragmentDefault: true}
|
||||
if udpSocketDFSet(t, dialUDPForDF(t, options)) {
|
||||
t.Fatal("UDPFragmentDefault=true must leave DF clear on dialed UDP sockets")
|
||||
}
|
||||
if udpSocketDFSet(t, listenUDPForDF(t, options)) {
|
||||
t.Fatal("UDPFragmentDefault=true must leave DF clear on listener-control UDP sockets")
|
||||
}
|
||||
}
|
||||
|
||||
// Explicit user config always wins over the protocol default, in both
|
||||
// directions.
|
||||
func TestUDPFragmentExplicitOverride_LX(t *testing.T) {
|
||||
fragmentOff := false
|
||||
options := option.DialerOptions{UDPFragment: &fragmentOff, UDPFragmentDefault: true}
|
||||
if !udpSocketDFSet(t, dialUDPForDF(t, options)) {
|
||||
t.Fatal("udp_fragment=false must set DF even when the protocol default allows fragmentation")
|
||||
}
|
||||
fragmentOn := true
|
||||
options = option.DialerOptions{UDPFragment: &fragmentOn}
|
||||
if udpSocketDFSet(t, dialUDPForDF(t, options)) {
|
||||
t.Fatal("udp_fragment=true must leave DF clear even without a protocol default")
|
||||
}
|
||||
}
|
||||
@@ -1,93 +0,0 @@
|
||||
// lx:begin awg
|
||||
|
||||
package group
|
||||
|
||||
import (
|
||||
"github.com/sagernet/sing-box/adapter"
|
||||
C "github.com/sagernet/sing-box/constant"
|
||||
)
|
||||
|
||||
// suspendAmneziaWGConsumersOnWireGuardSwitch is called from Selector.SelectOutbound
|
||||
// BEFORE the switch is committed. If the member about to be selected is — or chains
|
||||
// down via detour to — a WireGuard-based endpoint (type "wireguard", covering plain
|
||||
// WG and AmneziaWG), it walks UP from this group to every AmneziaWG endpoint that
|
||||
// detours through it and suspends each one (brings its device down). Rationale:
|
||||
// AmneziaWG traffic encapsulated inside a WireGuard tunnel hangs the kernel on
|
||||
// Android; the static Start-guard cannot cover this because a selector's chosen
|
||||
// member is only known at runtime.
|
||||
//
|
||||
// Called before s.selected is updated, so the race is closed: once the group
|
||||
// points at the WireGuard member, the AmneziaWG consumers are already suspended
|
||||
// (started=false) and a concurrent reconnect fails with "not ready" instead of
|
||||
// sending a junk handshake into WireGuard.
|
||||
func suspendAmneziaWGConsumersOnWireGuardSwitch(outboundManager adapter.OutboundManager, groupTag string, selected adapter.Outbound) {
|
||||
if outboundManager == nil || groupTag == "" {
|
||||
return
|
||||
}
|
||||
if !chainReachesWireGuard(outboundManager, selected, make(map[string]bool)) {
|
||||
return
|
||||
}
|
||||
suspendAmneziaWGConsumers(outboundManager, groupTag, make(map[string]bool))
|
||||
}
|
||||
|
||||
// chainReachesWireGuard reports whether outbound is — or transitively detours
|
||||
// down to, or (being a group) contains a member that is — a WireGuard-based
|
||||
// endpoint. visited guards against cycles.
|
||||
func chainReachesWireGuard(outboundManager adapter.OutboundManager, outbound adapter.Outbound, visited map[string]bool) bool {
|
||||
if outbound == nil {
|
||||
return false
|
||||
}
|
||||
tag := outbound.Tag()
|
||||
if tag != "" {
|
||||
if visited[tag] {
|
||||
return false
|
||||
}
|
||||
visited[tag] = true
|
||||
}
|
||||
if outbound.Type() == C.TypeWireGuard {
|
||||
return true
|
||||
}
|
||||
// Down the detour chain (vless -> ... -> wireguard).
|
||||
for _, dependency := range outbound.Dependencies() {
|
||||
if member, loaded := outboundManager.Outbound(dependency); loaded {
|
||||
if chainReachesWireGuard(outboundManager, member, visited) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
// A nested group: any member reaching WireGuard counts.
|
||||
if group, isGroup := outbound.(adapter.OutboundGroup); isGroup {
|
||||
for _, memberTag := range group.All() {
|
||||
if member, loaded := outboundManager.Outbound(memberTag); loaded {
|
||||
if chainReachesWireGuard(outboundManager, member, visited) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// suspendAmneziaWGConsumers walks UP from tag via the reverse-dependency ledger
|
||||
// (ConsumersOf) and suspends every AmneziaWG endpoint that detours through it,
|
||||
// directly or transitively (e.g. AWG -> vless -> group). visited guards cycles.
|
||||
func suspendAmneziaWGConsumers(outboundManager adapter.OutboundManager, tag string, visited map[string]bool) {
|
||||
for _, consumerTag := range outboundManager.ConsumersOf(tag) {
|
||||
if visited[consumerTag] {
|
||||
continue
|
||||
}
|
||||
visited[consumerTag] = true
|
||||
consumer, loaded := outboundManager.Outbound(consumerTag)
|
||||
if !loaded {
|
||||
continue
|
||||
}
|
||||
if awg, isAWG := consumer.(adapter.AmneziaWGSuspendable); isAWG && awg.IsAmneziaWG() {
|
||||
awg.SuspendAmneziaWG()
|
||||
}
|
||||
// Keep walking up: a non-AWG hop (vless) or a parent group may itself have
|
||||
// an AmneziaWG consumer above it.
|
||||
suspendAmneziaWGConsumers(outboundManager, consumerTag, visited)
|
||||
}
|
||||
}
|
||||
|
||||
// lx:end awg
|
||||
@@ -1,120 +0,0 @@
|
||||
// lx:begin awg
|
||||
|
||||
package group
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/sagernet/sing-box/adapter"
|
||||
C "github.com/sagernet/sing-box/constant"
|
||||
)
|
||||
|
||||
// fakeOutbound is a minimal adapter.Outbound; only Type/Tag/Dependencies are read.
|
||||
type fakeOutbound struct {
|
||||
adapter.Outbound
|
||||
tag string
|
||||
outboundTyp string
|
||||
detour string
|
||||
}
|
||||
|
||||
func (o *fakeOutbound) Type() string { return o.outboundTyp }
|
||||
func (o *fakeOutbound) Tag() string { return o.tag }
|
||||
func (o *fakeOutbound) Dependencies() []string {
|
||||
if o.detour == "" {
|
||||
return nil
|
||||
}
|
||||
return []string{o.detour}
|
||||
}
|
||||
|
||||
// fakeAWG implements adapter.AmneziaWGSuspendable and records suspension.
|
||||
type fakeAWG struct {
|
||||
fakeOutbound
|
||||
awg bool
|
||||
suspended bool
|
||||
}
|
||||
|
||||
func (a *fakeAWG) IsAmneziaWG() bool { return a.awg }
|
||||
func (a *fakeAWG) SuspendAmneziaWG() { a.suspended = true }
|
||||
|
||||
// fakeManager resolves tags and reverse-deps (ConsumersOf) from fixed maps.
|
||||
type fakeManager struct {
|
||||
adapter.OutboundManager
|
||||
byTag map[string]adapter.Outbound
|
||||
consumers map[string][]string
|
||||
}
|
||||
|
||||
func (m *fakeManager) Outbound(tag string) (adapter.Outbound, bool) {
|
||||
ob, ok := m.byTag[tag]
|
||||
return ob, ok
|
||||
}
|
||||
func (m *fakeManager) ConsumersOf(tag string) []string { return m.consumers[tag] }
|
||||
|
||||
func TestChainReachesWireGuard(t *testing.T) {
|
||||
wg := &fakeOutbound{tag: "wg", outboundTyp: C.TypeWireGuard}
|
||||
vlessToWG := &fakeOutbound{tag: "v2wg", outboundTyp: C.TypeVLESS, detour: "wg"}
|
||||
vlessLeaf := &fakeOutbound{tag: "vleaf", outboundTyp: C.TypeVLESS}
|
||||
mgr := &fakeManager{byTag: map[string]adapter.Outbound{
|
||||
"wg": wg, "v2wg": vlessToWG, "vleaf": vlessLeaf,
|
||||
}}
|
||||
|
||||
if !chainReachesWireGuard(mgr, wg, map[string]bool{}) {
|
||||
t.Fatal("direct wireguard member must reach wireguard")
|
||||
}
|
||||
if !chainReachesWireGuard(mgr, vlessToWG, map[string]bool{}) {
|
||||
t.Fatal("vless detouring to wireguard must reach wireguard")
|
||||
}
|
||||
if chainReachesWireGuard(mgr, vlessLeaf, map[string]bool{}) {
|
||||
t.Fatal("plain vless must not reach wireguard")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSuspendAmneziaWGConsumers(t *testing.T) {
|
||||
// awg-direct detours through the group "sel"
|
||||
awgDirect := &fakeAWG{fakeOutbound: fakeOutbound{tag: "awg-direct", outboundTyp: C.TypeWireGuard, detour: "sel"}, awg: true}
|
||||
// awg-via-hop -> vless-hop -> sel
|
||||
awgViaHop := &fakeAWG{fakeOutbound: fakeOutbound{tag: "awg-hop", outboundTyp: C.TypeWireGuard, detour: "vless-hop"}, awg: true}
|
||||
vlessHop := &fakeOutbound{tag: "vless-hop", outboundTyp: C.TypeVLESS, detour: "sel"}
|
||||
// plain-wg detours through sel but is NOT amneziawg — must stay untouched
|
||||
plainWG := &fakeAWG{fakeOutbound: fakeOutbound{tag: "plain-wg", outboundTyp: C.TypeWireGuard, detour: "sel"}, awg: false}
|
||||
|
||||
mgr := &fakeManager{
|
||||
byTag: map[string]adapter.Outbound{
|
||||
"awg-direct": awgDirect, "awg-hop": awgViaHop,
|
||||
"vless-hop": vlessHop, "plain-wg": plainWG,
|
||||
},
|
||||
consumers: map[string][]string{
|
||||
"sel": {"awg-direct", "vless-hop", "plain-wg"},
|
||||
"vless-hop": {"awg-hop"},
|
||||
},
|
||||
}
|
||||
|
||||
suspendAmneziaWGConsumers(mgr, "sel", map[string]bool{})
|
||||
|
||||
if !awgDirect.suspended {
|
||||
t.Error("direct AmneziaWG consumer must be suspended")
|
||||
}
|
||||
if !awgViaHop.suspended {
|
||||
t.Error("transitive AmneziaWG consumer (via vless hop) must be suspended")
|
||||
}
|
||||
if plainWG.suspended {
|
||||
t.Error("plain (non-AmneziaWG) wireguard consumer must NOT be suspended")
|
||||
}
|
||||
}
|
||||
|
||||
// A switch to a non-wireguard member must suspend nothing.
|
||||
func TestSuspendSkippedForNonWireGuardSwitch(t *testing.T) {
|
||||
awg := &fakeAWG{fakeOutbound: fakeOutbound{tag: "awg", outboundTyp: C.TypeWireGuard, detour: "sel"}, awg: true}
|
||||
vlessLeaf := &fakeOutbound{tag: "vleaf", outboundTyp: C.TypeVLESS}
|
||||
mgr := &fakeManager{
|
||||
byTag: map[string]adapter.Outbound{"awg": awg, "vleaf": vlessLeaf},
|
||||
consumers: map[string][]string{"sel": {"awg"}},
|
||||
}
|
||||
|
||||
// selected member is plain vless (does not reach wireguard) → no suspension
|
||||
suspendAmneziaWGConsumersOnWireGuardSwitch(mgr, "sel", vlessLeaf)
|
||||
if awg.suspended {
|
||||
t.Error("must not suspend when the selected member does not reach wireguard")
|
||||
}
|
||||
}
|
||||
|
||||
// lx:end awg
|
||||
@@ -128,16 +128,6 @@ func (s *Selector) SelectOutbound(tag string) bool {
|
||||
if s.selected.Load() == detour {
|
||||
return true
|
||||
}
|
||||
// lx:begin awg
|
||||
// Suspend AmneziaWG consumers BEFORE switching: if the new member is (or chains
|
||||
// to) a WireGuard endpoint, any AmneziaWG endpoint that detours through this
|
||||
// group would tunnel AWG inside WireGuard and hang the kernel on Android. Doing
|
||||
// this before s.selected.Swap closes the race — by the time the group points at
|
||||
// the WireGuard member, those consumers are already down (started=false), so a
|
||||
// concurrent reconnect fails with "not ready" instead of sending a junk
|
||||
// handshake into WireGuard.
|
||||
suspendAmneziaWGConsumersOnWireGuardSwitch(s.outbound, s.Tag(), detour)
|
||||
// lx:end awg
|
||||
s.selected.Store(detour)
|
||||
invalidateReachability(s.ctx) // lx: SPEC 020 — active selection changed
|
||||
if s.Tag() != "" {
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
//go:build with_gvisor && with_awg
|
||||
|
||||
// lx: regression for the removal of the AmneziaWG-over-WireGuard start guard.
|
||||
//
|
||||
// The guard refused to bring up an AmneziaWG endpoint whose detour chain reached
|
||||
// a WireGuard-based endpoint — and refused *silently*: Start returned nil with
|
||||
// started=false, so the endpoint looked configured but every dial through it
|
||||
// failed with "WireGuard is not ready yet". The root cause it protected against
|
||||
// (a kernel hang on Android) is gone on this graft (ClientBind reserved-gate),
|
||||
// and Android is not a supported platform here at all.
|
||||
//
|
||||
// This test builds a real AmneziaWG endpoint (junk + ranged magic headers) whose
|
||||
// detour points at an outbound of type "wireguard", drives both start stages,
|
||||
// and asserts the endpoint reports itself started. With the guard in place the
|
||||
// first stage short-circuits and started stays false — this test fails.
|
||||
package wireguard
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/base64"
|
||||
"net"
|
||||
"net/netip"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/sagernet/sing-box/adapter"
|
||||
C "github.com/sagernet/sing-box/constant"
|
||||
"github.com/sagernet/sing-box/log"
|
||||
"github.com/sagernet/sing-box/option"
|
||||
"github.com/sagernet/sing/common/json/badoption"
|
||||
M "github.com/sagernet/sing/common/metadata"
|
||||
"github.com/sagernet/sing/service"
|
||||
"github.com/sagernet/sing/service/pause"
|
||||
)
|
||||
|
||||
// wgTypedOutbound is an adapter.Outbound that reports type "wireguard" — the hop
|
||||
// the guard used to refuse to start behind. Dialling through it always fails:
|
||||
// the point of the test is that the upper endpoint comes UP, not that it carries
|
||||
// traffic (that is the job of the transport-level e2e stand).
|
||||
type wgTypedOutbound struct {
|
||||
adapter.Outbound
|
||||
tag string
|
||||
}
|
||||
|
||||
func (o *wgTypedOutbound) Type() string { return C.TypeWireGuard }
|
||||
func (o *wgTypedOutbound) Tag() string { return o.tag }
|
||||
func (o *wgTypedOutbound) Dependencies() []string { return nil }
|
||||
|
||||
func (o *wgTypedOutbound) DialContext(ctx context.Context, network string, destination M.Socksaddr) (net.Conn, error) {
|
||||
return nil, os.ErrClosed
|
||||
}
|
||||
|
||||
func (o *wgTypedOutbound) ListenPacket(ctx context.Context, destination M.Socksaddr) (net.PacketConn, error) {
|
||||
return nil, os.ErrClosed
|
||||
}
|
||||
|
||||
// startChainManager resolves tags from a fixed map. adapter.OutboundManager is
|
||||
// embedded so this compiles against either shape of the interface.
|
||||
type startChainManager struct {
|
||||
adapter.OutboundManager
|
||||
byTag map[string]adapter.Outbound
|
||||
}
|
||||
|
||||
func (m *startChainManager) Outbound(tag string) (adapter.Outbound, bool) {
|
||||
ob, loaded := m.byTag[tag]
|
||||
return ob, loaded
|
||||
}
|
||||
|
||||
func randomKey(t *testing.T) string {
|
||||
t.Helper()
|
||||
var key [32]byte
|
||||
if _, err := rand.Read(key[:]); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Clamp so wireguard-go accepts it as a curve25519 private key.
|
||||
key[0] &= 248
|
||||
key[31] = (key[31] & 127) | 64
|
||||
return base64.StdEncoding.EncodeToString(key[:])
|
||||
}
|
||||
|
||||
// TestAmneziaWGOverWireGuardDetourStarts pins the invariant: an AmneziaWG
|
||||
// endpoint detouring through a WireGuard hop must come up like any other.
|
||||
func TestAmneziaWGOverWireGuardDetourStarts(t *testing.T) {
|
||||
ctx := pause.WithDefaultManager(context.Background())
|
||||
ctx = service.ContextWith[adapter.OutboundManager](ctx, &startChainManager{
|
||||
byTag: map[string]adapter.Outbound{
|
||||
"wg-hop": &wgTypedOutbound{tag: "wg-hop"},
|
||||
},
|
||||
})
|
||||
options := option.WireGuardEndpointOptions{
|
||||
MTU: 1280,
|
||||
Address: badoption.Listable[netip.Prefix]{netip.MustParsePrefix("10.7.0.2/32")},
|
||||
PrivateKey: randomKey(t),
|
||||
Peers: []option.WireGuardPeer{{
|
||||
Address: "10.9.9.9",
|
||||
Port: 51820,
|
||||
PublicKey: randomKey(t),
|
||||
AllowedIPs: badoption.Listable[netip.Prefix]{netip.MustParsePrefix("0.0.0.0/0")},
|
||||
}},
|
||||
AmneziaWGOptions: option.AmneziaWGOptions{
|
||||
Jc: 3,
|
||||
Jmin: 8,
|
||||
Jmax: 80,
|
||||
S4: 16,
|
||||
H1: "10-20",
|
||||
H2: "30-40",
|
||||
H3: "50-60",
|
||||
H4: "70-80",
|
||||
},
|
||||
}
|
||||
options.Detour = "wg-hop"
|
||||
|
||||
ep, err := NewEndpoint(ctx, nil, log.NewNOPFactory().NewLogger("wg-awg"), "wg-awg", options)
|
||||
if err != nil {
|
||||
t.Fatal("create amneziawg endpoint over a wireguard detour: ", err)
|
||||
}
|
||||
defer ep.Close()
|
||||
|
||||
if err = ep.Start(adapter.StartStateStart); err != nil {
|
||||
t.Fatal("start stage: ", err)
|
||||
}
|
||||
if err = ep.Start(adapter.StartStatePostStart); err != nil {
|
||||
t.Fatal("post-start stage: ", err)
|
||||
}
|
||||
if !ep.(*Endpoint).started.Load() {
|
||||
t.Fatal("an amneziawg endpoint behind a wireguard hop must start; it is silently held down")
|
||||
}
|
||||
}
|
||||
+20
-118
@@ -4,7 +4,6 @@ import (
|
||||
"context"
|
||||
"net"
|
||||
"net/netip"
|
||||
"strconv"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
@@ -45,28 +44,14 @@ type Endpoint struct {
|
||||
localAddresses []netip.Prefix
|
||||
endpoint *wireguard.Endpoint
|
||||
started atomic.Bool
|
||||
// lx:begin awg
|
||||
// awgActive marks this endpoint as running AmneziaWG (AmneziaWGOptions.IsSet());
|
||||
// detour is its configured upstream tag. Start uses them to refuse to bring up
|
||||
// an AmneziaWG-over-WireGuard chain, which hangs the kernel on Android — see
|
||||
// awgDetourChainReachesWireGuard. The ledger lives here (not just in the dialer
|
||||
// guard) because the hang happens synchronously in Start, before any dial.
|
||||
awgActive bool
|
||||
detour string
|
||||
// awgChainBlocked is set by Start when the AmneziaWG-over-WireGuard guard
|
||||
// fires: the device is left unstarted (started stays false) so no junk
|
||||
// handshake runs and the kernel cannot hang, while the rest of the instance
|
||||
// comes up. PostStart then skips this endpoint too.
|
||||
awgChainBlocked bool
|
||||
// lx:end awg
|
||||
// lx:begin idle-suspend
|
||||
// SPEC 020 idle-suspend state. lastActivity is the unix-nano timestamp of the
|
||||
// last dial through this endpoint, stamped at PostStart and on every dial entry.
|
||||
// idleAsleep is true while the endpoint is Down due to idle-suspend (distinct
|
||||
// from a guard-suspend, which sets started=false and clears idleAsleep, so a
|
||||
// guard-suspended endpoint fast-paths out of resumeOnDial and is never
|
||||
// idle-woken). resumeMu serialises the idle tick's suspend decision, a dial's
|
||||
// wake, and the AmneziaWG guard-suspend against one another.
|
||||
// from a deliberately-stopped endpoint, which has started=false and
|
||||
// idleAsleep=false, so it fast-paths out of resumeOnDial and is never
|
||||
// idle-woken). resumeMu serialises the idle tick's suspend decision against a
|
||||
// dial's wake.
|
||||
lastActivity atomic.Int64
|
||||
idleAsleep atomic.Bool
|
||||
resumeMu sync.Mutex
|
||||
@@ -74,6 +59,16 @@ type Endpoint struct {
|
||||
}
|
||||
|
||||
func NewEndpoint(ctx context.Context, router adapter.Router, logger log.ContextLogger, tag string, options option.WireGuardEndpointOptions) (adapter.Endpoint, error) {
|
||||
// lx: allow OS-level fragmentation of the OUTER UDP socket by default, the
|
||||
// same opt-out direct/hysteria/hysteria2/tuic already take. Without it the
|
||||
// dialer sets DF (IP_MTU_DISCOVER=IP_PMTUDISC_DO on linux), and an outer
|
||||
// datagram over the path MTU — routine once anything is encapsulated: WG's
|
||||
// own ~32 B header, AmneziaWG s4 transport junk, or this endpoint carrying a
|
||||
// nested tunnel — is dropped by the kernel ("message too long") instead of
|
||||
// fragmented, so the tunnel comes up and then carries nothing. An explicit
|
||||
// `udp_fragment: false` on the node still restores DF (UDPFragment wins over
|
||||
// UDPFragmentDefault in common/dialer).
|
||||
options.UDPFragmentDefault = true
|
||||
ep := &Endpoint{
|
||||
Adapter: endpoint.NewAdapterWithDialerOptions(C.TypeWireGuard, tag, []string{N.NetworkTCP, N.NetworkUDP, N.NetworkICMP}, options.DialerOptions),
|
||||
ctx: ctx,
|
||||
@@ -81,10 +76,6 @@ func NewEndpoint(ctx context.Context, router adapter.Router, logger log.ContextL
|
||||
dnsRouter: service.FromContext[adapter.DNSRouter](ctx),
|
||||
logger: logger,
|
||||
localAddresses: options.Address,
|
||||
// lx:begin awg
|
||||
awgActive: options.AmneziaWGOptions.IsSet(),
|
||||
detour: options.Detour,
|
||||
// lx:end awg
|
||||
}
|
||||
if options.Detour != "" && options.ListenPort != 0 {
|
||||
return nil, E.New("`listen_port` is conflict with `detour`")
|
||||
@@ -116,7 +107,8 @@ func NewEndpoint(ctx context.Context, router adapter.Router, logger log.ContextL
|
||||
Dialer: outboundDialer,
|
||||
CreateDialer: func(interfaceName string) N.Dialer {
|
||||
return common.Must1(dialer.NewDefault(ctx, option.DialerOptions{
|
||||
BindInterface: interfaceName,
|
||||
BindInterface: interfaceName,
|
||||
UDPFragmentDefault: true, // lx: same reason as above — this is the bind-to-interface twin of the outer socket
|
||||
}))
|
||||
},
|
||||
Name: options.Name,
|
||||
@@ -157,33 +149,6 @@ func NewEndpoint(ctx context.Context, router adapter.Router, logger log.ContextL
|
||||
}
|
||||
|
||||
func (w *Endpoint) Start(stage adapter.StartStage) error {
|
||||
// lx:begin awg
|
||||
// Refuse to bring up an AmneziaWG endpoint whose detour chain reaches a
|
||||
// WireGuard-based endpoint: encapsulating AWG (junk handshake) inside a
|
||||
// WireGuard tunnel hangs the kernel on Android. The hang happens here, in the
|
||||
// synchronous Start path (peer-domain resolution over the detour, then the
|
||||
// device's junk handshake) — before any dial — so the lazy DetourDialer guard
|
||||
// never gets a chance to fire. We must catch it at Start instead.
|
||||
//
|
||||
// Behaviour is "variant B": do NOT return an error (that would abort the whole
|
||||
// instance start). Instead log, skip device startup, and leave started=false
|
||||
// so the rest of the config comes up and every dial through this endpoint
|
||||
// fails cleanly with "WireGuard is not ready yet". A selector/urltest in the
|
||||
// middle hides the real target at start time, so the chain walk stops at a
|
||||
// group and that case is left to the lazy DetourDialer guard at dial time.
|
||||
if stage == adapter.StartStateStart && w.awgActive && w.detour != "" {
|
||||
if outboundManager := service.FromContext[adapter.OutboundManager](w.ctx); outboundManager != nil {
|
||||
if blockedBy := awgDetourChainReachesWireGuard(outboundManager, w.detour, make(map[string]bool)); blockedBy != "" {
|
||||
w.awgChainBlocked = true
|
||||
w.logger.Error("amneziawg endpoint will not start: its detour chain reaches wireguard-based endpoint ", strconv.Quote(blockedBy), " — amneziawg over wireguard is not supported. Use a non-wireguard detour (e.g. vless).")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
if w.awgChainBlocked {
|
||||
return nil
|
||||
}
|
||||
// lx:end awg
|
||||
switch stage {
|
||||
case adapter.StartStateStart:
|
||||
return w.endpoint.Start(false)
|
||||
@@ -200,69 +165,6 @@ func (w *Endpoint) Start(stage adapter.StartStage) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// lx:begin awg
|
||||
// awgDetourChainReachesWireGuard walks the transitive detour chain starting at
|
||||
// tag and returns the tag of the first WireGuard-based outbound it reaches
|
||||
// (type "wireguard", covering plain WireGuard and AmneziaWG), or "" if none. It
|
||||
// follows each outbound's detour dependency; it deliberately does NOT expand
|
||||
// selector/urltest groups, whose chosen member is only known at runtime — that
|
||||
// case is handled lazily by the DetourDialer guard. visited guards against cyclic
|
||||
// detour configs. All outbounds are registered before any Start, so every tag in
|
||||
// the chain is resolvable here even though some may not have started yet.
|
||||
func awgDetourChainReachesWireGuard(outboundManager adapter.OutboundManager, tag string, visited map[string]bool) string {
|
||||
if tag == "" || visited[tag] {
|
||||
return ""
|
||||
}
|
||||
visited[tag] = true
|
||||
outbound, loaded := outboundManager.Outbound(tag)
|
||||
if !loaded {
|
||||
return ""
|
||||
}
|
||||
if outbound.Type() == C.TypeWireGuard {
|
||||
return tag
|
||||
}
|
||||
if _, isGroup := outbound.(adapter.OutboundGroup); isGroup {
|
||||
// Runtime-resolved target — leave it to the lazy DetourDialer guard.
|
||||
return ""
|
||||
}
|
||||
for _, dependency := range outbound.Dependencies() {
|
||||
if blockedBy := awgDetourChainReachesWireGuard(outboundManager, dependency, visited); blockedBy != "" {
|
||||
return blockedBy
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// IsAmneziaWG reports whether this endpoint runs AmneziaWG. Implements
|
||||
// adapter.AmneziaWGSuspendable.
|
||||
func (w *Endpoint) IsAmneziaWG() bool {
|
||||
return w.awgActive
|
||||
}
|
||||
|
||||
// SuspendAmneziaWG brings the device down and marks the endpoint not-ready, so a
|
||||
// junk handshake is never sent and every dial fails with "WireGuard is not ready
|
||||
// yet". Called by the selector guard when a group this endpoint detours through
|
||||
// switches to a WireGuard member (AmneziaWG over WireGuard hangs the kernel on
|
||||
// Android). Idempotent. Implements adapter.AmneziaWGSuspendable.
|
||||
func (w *Endpoint) SuspendAmneziaWG() {
|
||||
// Take resumeMu so this is ordered against resumeOnDial/SuspendIfIdle: without
|
||||
// it, a dial that already passed resumeOnDial's idleAsleep checks could wake
|
||||
// the endpoint back up right after we clear the flag, defeating the guard.
|
||||
w.resumeMu.Lock()
|
||||
defer w.resumeMu.Unlock()
|
||||
if w.started.CompareAndSwap(true, false) {
|
||||
w.logger.Error("amneziawg endpoint suspended: a selector in its detour chain switched to a wireguard-based member — amneziawg over wireguard is not supported")
|
||||
}
|
||||
// Clear any idle-suspend state so resumeOnDial does not resurrect a
|
||||
// guard-suspended endpoint: if it was idle-asleep first, idleAsleep would still
|
||||
// be true and the next dial would wake it (SPEC 022 #2). With idleAsleep=false
|
||||
// resumeOnDial's fast path returns started (now false) and the endpoint stays down.
|
||||
w.idleAsleep.Store(false)
|
||||
w.endpoint.Suspend()
|
||||
}
|
||||
|
||||
// lx:end awg
|
||||
|
||||
// lx:begin idle-suspend
|
||||
|
||||
// stampActivity records the current time as the last dial through this endpoint.
|
||||
@@ -286,8 +188,8 @@ func (w *Endpoint) IdleSince() time.Duration {
|
||||
// holder — when it is unreachable from the active routing tree AND has been idle
|
||||
// past the threshold. Silent on every non-transition (edge-triggered logging).
|
||||
//
|
||||
// It never touches a guard-suspended endpoint: that one already has
|
||||
// started==false but idleAsleep==false, and the `!started` guard below short-
|
||||
// It never touches a deliberately-stopped endpoint: that one already has
|
||||
// started==false but idleAsleep==false, and the `!started` check below short-
|
||||
// circuits before the CAS. resumeMu mutually excludes this against resumeOnDial.
|
||||
func (w *Endpoint) SuspendIfIdle(reachable bool, threshold time.Duration) {
|
||||
w.resumeMu.Lock()
|
||||
@@ -296,7 +198,7 @@ func (w *Endpoint) SuspendIfIdle(reachable bool, threshold time.Duration) {
|
||||
return
|
||||
}
|
||||
if !w.started.Load() {
|
||||
// Already down some other way (guard-suspend, awg-chain-blocked, closed).
|
||||
// Already down some other way (deliberately stopped, closed).
|
||||
return
|
||||
}
|
||||
if w.idleAsleep.CompareAndSwap(false, true) {
|
||||
@@ -313,7 +215,7 @@ func (w *Endpoint) SuspendIfIdle(reachable bool, threshold time.Duration) {
|
||||
// session); that cost is on the first packet, as for any cold WG dial.
|
||||
//
|
||||
// Returns true if the endpoint is dialable (awake), false if it must stay down
|
||||
// (guard-suspend / chain-blocked — not an idle-suspend, so we do not resurrect it).
|
||||
// (deliberately stopped / closed — not an idle-suspend, so we do not resurrect it).
|
||||
func (w *Endpoint) resumeOnDial() bool {
|
||||
w.stampActivity()
|
||||
if !w.idleAsleep.Load() {
|
||||
|
||||
@@ -103,21 +103,21 @@ func TestSuspendIfIdle_idempotentCAS(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestSuspendIfIdle_guardSuspendedNotTouched is the §8 invariant verified live on
|
||||
// an AWG-over-WG endpoint (wg-3 in the prod run): a guard-suspended endpoint has
|
||||
// started=false WITHOUT idleAsleep. The idle tick must early-return on !started and
|
||||
// NOT flip idleAsleep — otherwise a later resumeOnDial would idle-wake it and
|
||||
// re-trigger the AWG-over-WG kernel hang the guard exists to prevent.
|
||||
func TestSuspendIfIdle_guardSuspendedNotTouched(t *testing.T) {
|
||||
// TestSuspendIfIdle_stoppedNotTouched is the §8 invariant: a deliberately-stopped
|
||||
// endpoint (Close, or a start that never completed) has started=false WITHOUT
|
||||
// idleAsleep. The idle tick must early-return on !started and NOT flip idleAsleep
|
||||
// — otherwise a later resumeOnDial would idle-wake a device that was
|
||||
// intentionally down.
|
||||
func TestSuspendIfIdle_stoppedNotTouched(t *testing.T) {
|
||||
w := newIdleTestEndpoint()
|
||||
w.started.Store(false) // guard-suspend (device.Down at Start), idleAsleep stays false
|
||||
w.started.Store(false) // stopped, idleAsleep stays false
|
||||
w.lastActivity.Store(time.Now().Add(-time.Hour).UnixNano())
|
||||
w.SuspendIfIdle(false, 30*time.Second)
|
||||
if w.idleAsleep.Load() {
|
||||
t.Fatal("a guard-suspended endpoint must NOT be flagged idleAsleep by the tick")
|
||||
t.Fatal("a stopped endpoint must NOT be flagged idleAsleep by the tick")
|
||||
}
|
||||
if w.started.Load() {
|
||||
t.Fatal("the tick must not change started for a guard-suspended endpoint")
|
||||
t.Fatal("the tick must not change started for a stopped endpoint")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -152,17 +152,17 @@ func TestResumeOnDial_dialBeforeTickRace(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestResumeOnDial_guardSuspendedNotWoken(t *testing.T) {
|
||||
// A guard-suspended endpoint has started=false but idleAsleep=false.
|
||||
// resumeOnDial must NOT wake it (returns started, i.e. false).
|
||||
func TestResumeOnDial_stoppedNotWoken(t *testing.T) {
|
||||
// A deliberately-stopped endpoint (Close / failed start) has started=false but
|
||||
// idleAsleep=false. resumeOnDial must NOT wake it (returns started, i.e. false).
|
||||
w := newIdleTestEndpoint()
|
||||
w.started.Store(false) // simulate guard/awg-chain suspend (not idle)
|
||||
w.started.Store(false) // stopped, not idle-suspended
|
||||
ok := w.resumeOnDial()
|
||||
if ok {
|
||||
t.Fatal("resumeOnDial must not resurrect a guard-suspended (non-idle) endpoint")
|
||||
t.Fatal("resumeOnDial must not resurrect a stopped (non-idle) endpoint")
|
||||
}
|
||||
if w.idleAsleep.Load() {
|
||||
t.Fatal("guard-suspended endpoint must not be flagged idleAsleep")
|
||||
t.Fatal("stopped endpoint must not be flagged idleAsleep")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"net"
|
||||
"net/netip"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/sagernet/sing/common"
|
||||
@@ -29,11 +30,15 @@ type ClientBind struct {
|
||||
dialer N.Dialer
|
||||
reservedForEndpoint map[netip.AddrPort][3]uint8
|
||||
connAccess sync.Mutex
|
||||
conn *wireConn
|
||||
done chan struct{}
|
||||
isConnect bool
|
||||
connectAddr netip.AddrPort
|
||||
reserved [3]uint8
|
||||
// lx: atomic — connect()'s fast-path read is lock-free while the store below
|
||||
// happens under connAccess, so a plain field is a data race between the send
|
||||
// and receive goroutines at tunnel start (upstream sing-box has the same
|
||||
// shape). No logic change: the double-check and the locking are unchanged.
|
||||
conn atomic.Pointer[wireConn]
|
||||
done chan struct{}
|
||||
isConnect bool
|
||||
connectAddr netip.AddrPort
|
||||
reserved [3]uint8
|
||||
}
|
||||
|
||||
func NewClientBind(ctx context.Context, logger logger.Logger, dialer N.Dialer, isConnect bool, connectAddr netip.AddrPort, reserved [3]uint8) *ClientBind {
|
||||
@@ -98,7 +103,7 @@ func (c *ClientBind) hasReserved() bool {
|
||||
}
|
||||
|
||||
func (c *ClientBind) connect() (*wireConn, error) {
|
||||
serverConn := c.conn
|
||||
serverConn := c.conn.Load()
|
||||
if serverConn != nil {
|
||||
select {
|
||||
case <-serverConn.done:
|
||||
@@ -114,7 +119,7 @@ func (c *ClientBind) connect() (*wireConn, error) {
|
||||
return nil, net.ErrClosed
|
||||
default:
|
||||
}
|
||||
serverConn = c.conn
|
||||
serverConn = c.conn.Load()
|
||||
if serverConn != nil {
|
||||
select {
|
||||
case <-serverConn.done:
|
||||
@@ -128,7 +133,7 @@ func (c *ClientBind) connect() (*wireConn, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.conn = &wireConn{
|
||||
serverConn = &wireConn{
|
||||
PacketConn: bufio.NewUnbindPacketConn(udpConn),
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
@@ -137,12 +142,13 @@ func (c *ClientBind) connect() (*wireConn, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.conn = &wireConn{
|
||||
serverConn = &wireConn{
|
||||
PacketConn: bufio.NewPacketConn(udpConn),
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
return c.conn, nil
|
||||
c.conn.Store(serverConn)
|
||||
return serverConn, nil
|
||||
}
|
||||
|
||||
func (c *ClientBind) Open(port uint16) (fns []conn.ReceiveFunc, actualPort uint16, err error) {
|
||||
@@ -206,7 +212,7 @@ func (c *ClientBind) Close() error {
|
||||
}
|
||||
c.connAccess.Lock()
|
||||
defer c.connAccess.Unlock()
|
||||
common.Close(common.PtrOrNil(c.conn))
|
||||
common.Close(common.PtrOrNil(c.conn.Load()))
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
// lx: -race regression for ClientBind.conn.
|
||||
//
|
||||
// connect() reads the field on a lock-free fast path while the double-checked
|
||||
// init writes it under connAccess. The send and receive goroutines both call
|
||||
// connect() as the tunnel comes up, so the plain field was a data race (upstream
|
||||
// sing-box has the same shape; fixed there by making it an atomic.Pointer).
|
||||
// The e2e AWG-over-detour stand hit it only occasionally — this test drives the
|
||||
// same two accesses deterministically, so `go test -race` sees them every run.
|
||||
package wireguard
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/sagernet/sing/common/logger"
|
||||
M "github.com/sagernet/sing/common/metadata"
|
||||
N "github.com/sagernet/sing/common/network"
|
||||
)
|
||||
|
||||
// rawUDPDialer hands out real (unconnected) UDP sockets on loopback, so
|
||||
// connect() takes its normal path without any network peer.
|
||||
type rawUDPDialer struct{}
|
||||
|
||||
func (rawUDPDialer) DialContext(ctx context.Context, network string, destination M.Socksaddr) (net.Conn, error) {
|
||||
return net.Dial(N.NetworkUDP, destination.String())
|
||||
}
|
||||
|
||||
func (rawUDPDialer) ListenPacket(ctx context.Context, destination M.Socksaddr) (net.PacketConn, error) {
|
||||
return net.ListenPacket(N.NetworkUDP, "127.0.0.1:0")
|
||||
}
|
||||
|
||||
func TestClientBindConnectConcurrent(t *testing.T) {
|
||||
// A real listener so the "connect" dial has somewhere to point.
|
||||
peer, err := net.ListenPacket(N.NetworkUDP, "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer peer.Close()
|
||||
|
||||
for _, isConnect := range []bool{true, false} {
|
||||
bind := NewClientBind(
|
||||
context.Background(),
|
||||
logger.NOP(),
|
||||
rawUDPDialer{},
|
||||
isConnect,
|
||||
M.SocksaddrFromNet(peer.LocalAddr()).Unwrap().AddrPort(),
|
||||
[3]uint8{},
|
||||
)
|
||||
if _, _, err = bind.Open(0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Eight goroutines racing the fast-path read against the locked store —
|
||||
// exactly the send/receive pair at tunnel start, only wider.
|
||||
var wg sync.WaitGroup
|
||||
for range 8 {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for range 32 {
|
||||
if _, connErr := bind.connect(); connErr != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if bind.conn.Load() == nil {
|
||||
t.Fatal("connect() must have installed a wireConn")
|
||||
}
|
||||
if err = bind.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -278,9 +278,8 @@ func (e *Endpoint) Close() error {
|
||||
|
||||
// lx:begin awg
|
||||
// Suspend brings the WireGuard device down without closing it, stopping the
|
||||
// (junk) handshake machinery. Used by the selector guard to neutralise an
|
||||
// AmneziaWG endpoint when a group it detours through switches to a WireGuard
|
||||
// member. Idempotent — a nil device (never started / already closed) is a no-op.
|
||||
// (junk) handshake machinery. Idempotent — a nil device (never started /
|
||||
// already closed) is a no-op.
|
||||
//
|
||||
// SPEC 020 reuses Suspend for idle-suspend: device.Down() closes the UDP socket,
|
||||
// which makes RoutineReceiveIncoming exit and release its bufsArrs (the dominant
|
||||
|
||||
Reference in New Issue
Block a user