fix(tlsfragment): one cut, in the label a blocklist keys on — and a budget for it
Follow-up to 815011dfb, which fixed WHICH label is cut but left "a cut in every
candidate label" as an unconditional rule. Measured on this tree, loopback peer,
product default fallbackDelay, one ClientHello per row:
cuts tls_fragment (*net.TCPConn) tls_fragment (proxy conn) tls_record_fragment
1 502 ms 500 ms <1 ms
2 1.004 s 1.001 s <1 ms
4 2.008 s 2.002 s <1 ms
8 4.015 s 4.003 s 539 us
21 10.540 s 10.509 s 525 us
So a cut in the PACKET modes costs half a second of connection setup, and it
costs that on BOTH branches — not only on the sleep path. writeAndWaitAck sleeps
the whole fallbackDelay whenever the ACK returns inside 20 ms (its "under
transparent proxy" case), and N.UnwrapReader reaches the *net.TCPConn only when
nothing in the chain transforms the stream, which a proxy protocol conn always
does. A proxied egress — every subscription node — therefore takes the flat
500 ms branch regardless of RTT. The number of labels is chosen by whoever picked
the hostname, and a 253-byte SNI is 85 of them: ~42 s of one connection's setup,
bought from the LAN.
In tls_record_fragment nothing waits: the ClientHello leaves in ONE write, split
into more records. 21 cuts cost 525 us and 105 bytes of record headers, and
1.1.1.1 completed the handshake with the ClientHello in 22 records in the same
77 ms it took with 2. That is the mode the field measurement was taken in, and
the mode where cutting every label was always affordable.
Hence two budgets rather than one rule: 1 cut for the packet modes, 4 for
record-only — the latter not a cost limit but a shape limit, since real names
carry one to three labels outside the public suffix and a hostile one must not
turn a ClientHello into 85 records no ordinary client emits.
One cut is enough because of WHERE it goes. Candidates are now ordered, most
worth cutting first, and first is the REGISTRABLE label — the one immediately
left of the public suffix. That is what a name-based blocklist keys on
("youtube" of youtube.com, www.youtube.com and studio.youtube.com alike,
"ytimg" of i9.ytimg.com, "example" of a.b.example.co.uk), and severing it also
breaks any match on the whole FQDN, so one cut covers both matchers. It is
chosen by STRUCTURE, from the public suffix list — not by length, which is the
same trap from the other side: in cdn-static-assets.youtube.com the longest
label is not the blocked one. The rest follow longest-first, on the argument
that among labels with no structural ranking a long one is likelier to be a
distinctive token than "www", "m" or "tv"; they are reached only when the budget
allows more, or when the registrable label is too short to cut.
The offset now comes from the label's MIDDLE THIRD. Every interior offset severs
the label, but one byte in leaves "outube" of "youtube" and a matcher keyed on a
substring still reads it. The draw stays random inside that third: a fixed point
would be a constant a middlebox vendor can special-case in one line, and this
whole family of tricks lives on making reassembly the only counter.
Also in this commit, and the reason it is not merely a tuning change: the panic
that shipped in v0.2.21 now has an instrument of its own.
TestWriteDoesNotPanicOnAServerNameChosenFromTheLAN drives real ClientHellos
carrying ".youtube.com", "youtube.com." (a legitimate FQDN with the root dot,
which curl and every browser will send), "..", an IP literal and non-ASCII bytes
through all three modes, and FuzzCutOffsets does the open half — 25.7 million
executions found nothing, and the fuzzer is shown able to find a planted defect
its seed corpus cannot reach, in one second. A hand-built ClientHello reaches
the shapes crypto/tls refuses to emit: a zero-length name, a 253-byte name, and
a server_name_list with a SECOND entry, which is why planning runs on
MyServerName.Length rather than on everything left in the extension.
Nine mutations, each failing by name with the numbers: the old dot arithmetic,
the old rand.Intn offset, the exact original expression (panic: invalid argument
to Intn, conn.go:208 <- Write conn.go:67), the empty-plan guard, the budget, the
priority order, the sort back into wire order, the first-entry truncation, the
middle third, and a one-byte corruption of a segment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BHw89tdWddzhjUc4bAH4tS
This commit is contained in:
+124
-27
@@ -7,6 +7,7 @@ import (
|
||||
"math/rand"
|
||||
"net"
|
||||
"net/netip"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -57,7 +58,13 @@ func (c *Conn) Write(b []byte) (n int, err error) {
|
||||
if serverName.Length >= 0 && serverName.Length < len(name) {
|
||||
name = name[:serverName.Length]
|
||||
}
|
||||
splitIndexes := cutOffsets(name, rand.Intn)
|
||||
// Packet fragmentation pays half a second per cut, record
|
||||
// fragmentation pays microseconds — see the budget constants.
|
||||
budget := recordCutBudget
|
||||
if c.splitPacket {
|
||||
budget = packetCutBudget
|
||||
}
|
||||
splitIndexes := cutOffsets(name, budget, rand.Intn)
|
||||
if len(splitIndexes) == 0 {
|
||||
// Nothing inside this name can be cut — it is empty or a single
|
||||
// byte, so there is no offset that leaves a non-empty piece on
|
||||
@@ -144,9 +151,47 @@ type labelSpan struct {
|
||||
end int
|
||||
}
|
||||
|
||||
// How many cuts Write may spend on one ClientHello. The two numbers differ
|
||||
// because the two modes cost completely different things per cut — MEASURED
|
||||
// 2026-07-27 on this tree, loopback peer, product default fallbackDelay
|
||||
// (500 ms), one ClientHello per row:
|
||||
//
|
||||
// cuts tls_fragment (*net.TCPConn) tls_fragment (proxy conn) tls_record_fragment
|
||||
// 1 502 ms 500 ms <1 ms
|
||||
// 2 1.004 s 1.001 s <1 ms
|
||||
// 4 2.008 s 2.002 s <1 ms
|
||||
// 8 4.015 s 4.003 s 539 µs
|
||||
// 21 10.540 s 10.509 s 525 µs
|
||||
//
|
||||
// So a cut in the PACKET modes costs half a second of connection setup, and it
|
||||
// costs that on BOTH branches: writeAndWaitAck sleeps the whole fallbackDelay
|
||||
// anyway whenever the ACK comes back inside 20 ms (its "under transparent
|
||||
// proxy" case), and N.UnwrapReader only reaches the *net.TCPConn when nothing
|
||||
// in the chain transforms the stream — which a proxy protocol conn always does,
|
||||
// so a proxied egress takes the flat-500 ms branch regardless of RTT. The
|
||||
// number of labels is chosen by whoever picked the hostname, so "a cut in every
|
||||
// label" made a 253-byte SNI worth ~42 s of one connection's setup.
|
||||
//
|
||||
// In tls_record_fragment nothing waits at all: the whole ClientHello leaves in
|
||||
// ONE write, split into more TLS records. 21 cuts cost 525 µs and 105 bytes of
|
||||
// record headers, and a real server (1.1.1.1) completed the handshake with the
|
||||
// ClientHello in 22 records in the same 77 ms it took with 2. That mode is
|
||||
// where "cut every label" was always affordable — and it is the mode the field
|
||||
// measurement that started this was taken in.
|
||||
//
|
||||
// recordCutBudget is therefore not a cost limit but a shape limit: real names
|
||||
// have one to three labels outside the public suffix, so 4 never binds on real
|
||||
// traffic, while a hostile 253-byte name cannot turn one ClientHello into 85
|
||||
// records that no ordinary client would ever emit.
|
||||
const (
|
||||
packetCutBudget = 1
|
||||
recordCutBudget = 4
|
||||
)
|
||||
|
||||
// cutOffsets plans where the ClientHello must be cut, in byte offsets relative
|
||||
// to the FIRST BYTE OF THE SERVER NAME. randIntn is math/rand's Intn in
|
||||
// production; a test hands in its own to make the plan deterministic.
|
||||
// to the FIRST BYTE OF THE SERVER NAME, spending at most budget cuts. randIntn
|
||||
// is math/rand's Intn in production; a test hands in its own to make the plan
|
||||
// deterministic.
|
||||
//
|
||||
// A cut is only a cut if it lands STRICTLY INSIDE a label. Offset 0 of a label
|
||||
// is that label's own boundary: it leaves the label — the very string the DPI
|
||||
@@ -155,37 +200,72 @@ type labelSpan struct {
|
||||
// offset 0, and on the measured provider (blocks by name in the handshake)
|
||||
// m.youtube.com, tv.youtube.com and www.youtube.com were all blocked with the
|
||||
// cut sitting uselessly at the start of "m"/"tv"/"www", while the name itself
|
||||
// travelled intact in one segment. Hence [1, len(label)-1], and hence a label
|
||||
// shorter than two bytes carries no cut at all.
|
||||
func cutOffsets(name string, randIntn func(n int) int) []int {
|
||||
// travelled intact in one segment. Hence a label shorter than two bytes carries
|
||||
// no cut at all.
|
||||
func cutOffsets(name string, budget int, randIntn func(n int) int) []int {
|
||||
var offsets []int
|
||||
for _, span := range cutLabels(name) {
|
||||
width := span.end - span.start
|
||||
if width < 2 {
|
||||
for _, span := range cutCandidates(name) {
|
||||
if span.end-span.start < 2 {
|
||||
continue // no interior offset exists
|
||||
}
|
||||
offsets = append(offsets, span.start+1+randIntn(width-1))
|
||||
offsets = append(offsets, cutInside(span, randIntn))
|
||||
if len(offsets) >= budget {
|
||||
break
|
||||
}
|
||||
}
|
||||
if len(offsets) == 0 && len(name) >= 2 {
|
||||
// Every candidate label was too short to cut on its own (a.b.co.uk).
|
||||
// No candidate label was long enough to cut on its own (a.b.co.uk).
|
||||
// Cut the name somewhere rather than hand it over in one piece: a
|
||||
// matcher looking for the whole FQDN still fails across the split, even
|
||||
// though no single label was severed.
|
||||
offsets = append(offsets, 1+randIntn(len(name)-1))
|
||||
offsets = append(offsets, cutInside(labelSpan{start: 0, end: len(name)}, randIntn))
|
||||
}
|
||||
slices.Sort(offsets) // candidates are returned by priority, the wire wants order
|
||||
return offsets
|
||||
}
|
||||
|
||||
// cutLabels returns the labels of name that a cut may land in.
|
||||
// cutInside draws an offset strictly inside span, from its MIDDLE THIRD.
|
||||
//
|
||||
// The public suffix is dropped because it is shared by everything under it and
|
||||
// carries none of the blocked word. Everything else stays: WIDENING the set of
|
||||
// labels needs no proof, NARROWING it does, so an input the public suffix list
|
||||
// has no opinion about (a trailing dot, an unmanaged TLD, a name that IS a
|
||||
// suffix) keeps every label and is cut everywhere. There is no branch here that
|
||||
// silently ends up cutting nothing — the only empty result is the empty name,
|
||||
// which has nothing to cut by construction.
|
||||
func cutLabels(name string) []labelSpan {
|
||||
// Every interior offset severs the label, but not equally well: a cut one byte
|
||||
// in leaves "outube" of "youtube", and a matcher keyed on a suffix or on a
|
||||
// six-byte substring still reads it. The middle leaves two short, unremarkable
|
||||
// halves. The draw stays random inside that third — a fixed point (say, exactly
|
||||
// the middle of the longest label) would be a constant a middlebox vendor can
|
||||
// special-case in one line, and the whole family of fragmentation tricks lives
|
||||
// on making reassembly the only counter.
|
||||
func cutInside(span labelSpan, randIntn func(n int) int) int {
|
||||
lo, hi := span.start+1, span.end-1 // the interior offsets, both inclusive
|
||||
if margin := (span.end - span.start - 1) / 3; margin > 0 {
|
||||
lo += margin
|
||||
hi -= margin
|
||||
}
|
||||
return lo + randIntn(hi-lo+1)
|
||||
}
|
||||
|
||||
// cutCandidates returns the labels of name that a cut may land in, MOST WORTH
|
||||
// CUTTING FIRST — which matters because the budget above is small.
|
||||
//
|
||||
// First is the registrable label: the one immediately left of the public
|
||||
// suffix. That is the label a name-based blocklist keys on ("youtube" of
|
||||
// youtube.com, www.youtube.com and studio.youtube.com alike, "ytimg" of
|
||||
// i9.ytimg.com, "example" of a.b.example.co.uk), and severing it also breaks
|
||||
// any match on the whole FQDN, so one cut covers both matchers. It is chosen by
|
||||
// STRUCTURE, from the public suffix list — not by length, which is the trap the
|
||||
// old code fell into from the other side: in cdn-static-assets.youtube.com the
|
||||
// longest label is not the blocked one.
|
||||
//
|
||||
// The rest follow longest-first: among labels we have no structural reason to
|
||||
// rank, a long one is likelier to be a distinctive token than "www", "m" or
|
||||
// "tv". They are only reached when the budget allows more than one cut, or when
|
||||
// the registrable label is too short to cut.
|
||||
//
|
||||
// The public suffix itself is dropped because it is shared by everything under
|
||||
// it and carries none of the blocked word. WIDENING this set needs no proof,
|
||||
// NARROWING it does, so an input the public suffix list has no opinion about (a
|
||||
// trailing dot, an unmanaged TLD, a name that IS a suffix) keeps every label.
|
||||
// No branch here ends up with nothing to cut except the empty name, which has
|
||||
// nothing to cut by construction.
|
||||
func cutCandidates(name string) []labelSpan {
|
||||
spans := labelSpans(name)
|
||||
suffix := publicsuffix.List.PublicSuffix(name)
|
||||
switch {
|
||||
@@ -198,12 +278,12 @@ func cutLabels(name string) []labelSpan {
|
||||
// about it — it returns the literal itself. Treat the whole literal as
|
||||
// one token: there is no name for a DPI box to read here, but the
|
||||
// caller asked for a fragmented handshake and gets one.
|
||||
spans = []labelSpan{{start: 0, end: len(name)}}
|
||||
return []labelSpan{{start: 0, end: len(name)}}
|
||||
|
||||
case suffix != "" && len(suffix) < len(name) && strings.HasSuffix(name, "."+suffix):
|
||||
// The ordinary case, and the one the old arithmetic got wrong: it
|
||||
// subtracted the number of dots in the WHOLE NAME, which — labels being
|
||||
// always one more than dots — left exactly one label, the first, for
|
||||
// always one more than dots — left exactly one label, the FIRST, for
|
||||
// every name in existence. Subtract the number of labels in the SUFFIX
|
||||
// instead: "com" is one ("www.youtube.com" keeps www + youtube),
|
||||
// "co.uk" is two ("a.b.co.uk" keeps a + b).
|
||||
@@ -214,11 +294,28 @@ func cutLabels(name string) []labelSpan {
|
||||
// Everything else — suffix == "" (a trailing dot, which the list
|
||||
// declines to parse), suffix == name (the name IS a public suffix:
|
||||
// "com", "co.uk", "localhost"), or a suffix that is somehow not a tail
|
||||
// of the name — keeps every label. Cutting inside a suffix costs one
|
||||
// extra segment and hides nothing that was not already hidden; NOT
|
||||
// cutting is the expensive mistake.
|
||||
// of the name — keeps every label. Cutting inside a suffix costs a
|
||||
// segment and hides nothing that was not already hidden; NOT cutting is
|
||||
// the expensive mistake.
|
||||
}
|
||||
return spans
|
||||
return byCutPriority(spans)
|
||||
}
|
||||
|
||||
// byCutPriority puts the registrable label first and orders the rest
|
||||
// longest-first. It never drops a span, so the budget — not this — decides how
|
||||
// many labels are actually cut.
|
||||
func byCutPriority(spans []labelSpan) []labelSpan {
|
||||
if len(spans) < 2 {
|
||||
return spans
|
||||
}
|
||||
out := make([]labelSpan, 0, len(spans))
|
||||
out = append(out, spans[len(spans)-1])
|
||||
rest := make([]labelSpan, len(spans)-1)
|
||||
copy(rest, spans[:len(spans)-1])
|
||||
slices.SortStableFunc(rest, func(a, b labelSpan) int {
|
||||
return (b.end - b.start) - (a.end - a.start)
|
||||
})
|
||||
return append(out, rest...)
|
||||
}
|
||||
|
||||
// labelSpans splits name on '.' and returns the byte range of each label.
|
||||
|
||||
+432
-158
@@ -1,6 +1,7 @@
|
||||
package tf
|
||||
|
||||
// Cut planning: which labels of the SNI get a cut, and where inside them.
|
||||
// Cut planning: which label of the SNI gets a cut, where inside it, and how
|
||||
// many cuts one ClientHello is allowed to cost.
|
||||
//
|
||||
// WHY THIS FILE EXISTS (2026-07-27)
|
||||
// Conn.Write used to compute the labels to cut as
|
||||
@@ -8,18 +9,22 @@ package tf
|
||||
// splits = splits[:len(splits)-strings.Count(serverName.ServerName, ".")]
|
||||
//
|
||||
// which is identically splits[:1] for EVERY name, labels being always one
|
||||
// more than dots. Exactly one label was ever cut — the first — so on a
|
||||
// more than dots. Exactly one label was ever cut — the LEFTMOST — so on a
|
||||
// provider that blocks by the name in the handshake, youtube.com passed (its
|
||||
// first label IS the blocked word) while m./tv./www./music./studio.youtube.com
|
||||
// were all blocked, the cut sitting uselessly inside "m"/"tv"/"www" while
|
||||
// "youtube" travelled whole in the next segment. Measured on the router.
|
||||
// were all blocked, the cut sitting inside "m"/"tv"/"www" while "youtube"
|
||||
// travelled whole in the next segment. Measured on the router.
|
||||
//
|
||||
// The second half of the same defect: the offset came from
|
||||
// rand.Intn(len(label)), whose 0 is the label's own boundary and severs
|
||||
// nothing. For a one-byte label that is the ONLY value it can take.
|
||||
// Two more halves of the same defect:
|
||||
// - the offset came from rand.Intn(len(label)), whose 0 is the label's own
|
||||
// boundary and severs nothing. For a one-byte label that is the ONLY
|
||||
// value it can take;
|
||||
// - an EMPTY label reached rand.Intn(0) and panicked the process. Reachable
|
||||
// from the LAN: route/conn.go wraps the outbound with this and the
|
||||
// ClientHello it fragments is the client's. See
|
||||
// TestWriteDoesNotPanicOnAServerNameChosenFromTheLAN.
|
||||
//
|
||||
// The tests below are the instrument for both halves. Every one of them fails
|
||||
// on the old expressions — see the mutation log in the task report.
|
||||
// Every test below fails on the old expressions — see the mutation log.
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
@@ -36,12 +41,12 @@ import (
|
||||
|
||||
// --- deterministic draws -----------------------------------------------------
|
||||
|
||||
// minRand always cuts at the first interior offset of a label, maxRand at the
|
||||
// last. Between them they pin BOTH ends of the range a cut may take, which is
|
||||
// the whole point: it is the ends that decide whether the label is severed or
|
||||
// merely touched.
|
||||
// minRand takes the lowest offset a label allows, maxRand the highest. Between
|
||||
// them they pin BOTH ends of the range, which is where the interesting failures
|
||||
// live: it is the ends that decide whether the label is severed or only touched.
|
||||
func minRand(int) int { return 0 }
|
||||
func maxRand(n int) int { return n - 1 }
|
||||
|
||||
func fixedRand(v int) func(int) int {
|
||||
return func(n int) int {
|
||||
if v >= n {
|
||||
@@ -51,167 +56,312 @@ func fixedRand(v int) func(int) int {
|
||||
}
|
||||
}
|
||||
|
||||
// --- which labels are cut, and where ----------------------------------------
|
||||
// severedLabel names the label that a cut at offset o splits in two, or says
|
||||
// why it splits none. This is the assertion vocabulary of the whole file: the
|
||||
// question is never "which number came out" but "which word did we break".
|
||||
func severedLabel(name string, o int) string {
|
||||
switch {
|
||||
case o <= 0 || o >= len(name):
|
||||
return "!outside the name"
|
||||
case name[o] == '.' || name[o-1] == '.':
|
||||
return "!a label boundary, nothing severed"
|
||||
}
|
||||
start := strings.LastIndexByte(name[:o], '.') + 1
|
||||
end := len(name)
|
||||
if i := strings.IndexByte(name[o:], '.'); i >= 0 {
|
||||
end = o + i
|
||||
}
|
||||
return name[start:end]
|
||||
}
|
||||
|
||||
func TestCutOffsetsSelectsEveryLabelOutsideThePublicSuffix(t *testing.T) {
|
||||
func severedLabels(name string, offsets []int) []string {
|
||||
var out []string
|
||||
for _, o := range offsets {
|
||||
out = append(out, severedLabel(name, o))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// --- which label is cut ------------------------------------------------------
|
||||
|
||||
func TestCutOffsetsCutTheRegistrableLabel(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
wantMin []int // offsets with the lowest draw in every label
|
||||
wantMax []int // offsets with the highest draw in every label
|
||||
why string
|
||||
name string
|
||||
packet []string // labels severed with the packet budget (1 cut)
|
||||
record []string // ... and with the record budget (4 cuts), in wire order
|
||||
why string
|
||||
}{
|
||||
{name: "youtube.com", wantMin: []int{1}, wantMax: []int{6}, why: "one label outside com"},
|
||||
{name: "www.youtube.com", wantMin: []int{1, 5}, wantMax: []int{2, 10}, why: "THE regression: youtube must be cut, not only www"},
|
||||
{name: "m.youtube.com", wantMin: []int{3}, wantMax: []int{8}, why: "m is one byte: no interior offset, so youtube carries the cut"},
|
||||
{name: "tv.youtube.com", wantMin: []int{1, 4}, wantMax: []int{1, 9}, why: "tv has exactly one interior offset"},
|
||||
{name: "music.youtube.com", wantMin: []int{1, 7}, wantMax: []int{4, 12}, why: ""},
|
||||
{name: "studio.youtube.com", wantMin: []int{1, 8}, wantMax: []int{5, 13}, why: ""},
|
||||
{name: "foo.bar.baz.youtube.com", wantMin: []int{1, 5, 9, 13}, wantMax: []int{2, 6, 10, 18}, why: "every label, not just the first"},
|
||||
{name: "example.co.uk", wantMin: []int{1}, wantMax: []int{6}, why: "co.uk is TWO labels of suffix"},
|
||||
{name: "a.b.example.co.uk", wantMin: []int{5}, wantMax: []int{10}, why: "a and b are one byte each; example carries the cut"},
|
||||
{name: "example.com.br", wantMin: []int{1}, wantMax: []int{6}, why: "com.br is two labels of suffix"},
|
||||
{name: "site.pp.ru", wantMin: []int{1}, wantMax: []int{3}, why: "pp.ru is a private two-label suffix"},
|
||||
{name: "co.uk", wantMin: []int{1, 4}, wantMax: []int{1, 4}, why: "the name IS the suffix: keep every label rather than cut nothing"},
|
||||
{name: "com", wantMin: []int{1}, wantMax: []int{2}, why: "same"},
|
||||
{name: "localhost", wantMin: []int{1}, wantMax: []int{8}, why: "unmanaged TLD: the list returns the whole name"},
|
||||
{name: "ab", wantMin: []int{1}, wantMax: []int{1}, why: "two bytes have exactly one interior offset"},
|
||||
{name: "a", wantMin: nil, wantMax: nil, why: "one byte cannot be cut at all"},
|
||||
{name: "", wantMin: nil, wantMax: nil, why: "no name, no cut"},
|
||||
{name: ".youtube.com", wantMin: []int{2}, wantMax: []int{7}, why: "leading dot: the empty label is skipped, NOT fed to rand.Intn(0)"},
|
||||
{name: "youtube.com.", wantMin: []int{1, 9}, wantMax: []int{6, 10}, why: "trailing dot: the list declines to parse it, so every label is cut"},
|
||||
{name: "a.b.co.uk", wantMin: []int{1}, wantMax: []int{8}, why: "no label is cuttable: fall back to cutting the name itself"},
|
||||
{name: "1.2.3.4", wantMin: []int{1}, wantMax: []int{6}, why: "IP literal: one opaque token"},
|
||||
{name: "::1", wantMin: []int{1}, wantMax: []int{2}, why: "IPv6 literal"},
|
||||
{name: "WWW.YouTube.COM", wantMin: []int{1, 5}, wantMax: []int{2, 10}, why: "the list preserves case"},
|
||||
{name: "youtube.com", packet: []string{"youtube"}, record: []string{"youtube"}},
|
||||
{name: "www.youtube.com", packet: []string{"youtube"}, record: []string{"www", "youtube"},
|
||||
why: "THE regression: the old code cut www and shipped youtube whole"},
|
||||
{name: "m.youtube.com", packet: []string{"youtube"}, record: []string{"youtube"},
|
||||
why: "a one-byte label has no interior offset and carries no cut"},
|
||||
{name: "tv.youtube.com", packet: []string{"youtube"}, record: []string{"tv", "youtube"}},
|
||||
{name: "music.youtube.com", packet: []string{"youtube"}, record: []string{"music", "youtube"}},
|
||||
{name: "studio.youtube.com", packet: []string{"youtube"}, record: []string{"studio", "youtube"}},
|
||||
{name: "cdn-static-assets.youtube.com", packet: []string{"youtube"}, record: []string{"cdn-static-assets", "youtube"},
|
||||
why: "the LONGEST label is not the blocked one — structure decides, not length"},
|
||||
{name: "foo.bar.baz.youtube.com", packet: []string{"youtube"}, record: []string{"foo", "bar", "baz", "youtube"},
|
||||
why: "four candidates, four cuts, and the budget stops there"},
|
||||
{name: "a.b.c.d.e.youtube.com", packet: []string{"youtube"}, record: []string{"youtube"},
|
||||
why: "five one-byte labels: the budget is never even reached"},
|
||||
{name: "i9.ytimg.com", packet: []string{"ytimg"}, record: []string{"i9", "ytimg"}},
|
||||
{name: "example.co.uk", packet: []string{"example"}, record: []string{"example"},
|
||||
why: "co.uk is TWO labels of public suffix"},
|
||||
{name: "a.b.example.co.uk", packet: []string{"example"}, record: []string{"example"}},
|
||||
{name: "example.com.br", packet: []string{"example"}, record: []string{"example"}},
|
||||
{name: "site.pp.ru", packet: []string{"site"}, record: []string{"site"},
|
||||
why: "pp.ru is a private two-label suffix"},
|
||||
{name: "localhost", packet: []string{"localhost"}, record: []string{"localhost"},
|
||||
why: "unmanaged TLD: the list returns the whole name, so cut it"},
|
||||
{name: "com", packet: []string{"com"}, record: []string{"com"}},
|
||||
{name: "co.uk", packet: []string{"uk"}, record: []string{"co", "uk"},
|
||||
why: "the name IS the suffix: keep every label rather than cut nothing"},
|
||||
{name: ".youtube.com", packet: []string{"youtube"}, record: []string{"youtube"},
|
||||
why: "leading dot: the empty label is skipped, NOT fed to rand.Intn(0)"},
|
||||
{name: "youtube.com.", packet: []string{"youtube"}, record: []string{"youtube", "com"},
|
||||
why: "trailing dot: the list declines to parse it, so every label stays a candidate"},
|
||||
{name: "WWW.YouTube.COM", packet: []string{"YouTube"}, record: []string{"WWW", "YouTube"}},
|
||||
{name: "ab", packet: []string{"ab"}, record: []string{"ab"}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
require.Equal(t, tc.wantMin, cutOffsets(tc.name, minRand), "lowest draw: %s", tc.why)
|
||||
require.Equal(t, tc.wantMax, cutOffsets(tc.name, maxRand), "highest draw: %s", tc.why)
|
||||
for _, draw := range []struct {
|
||||
label string
|
||||
fn func(int) int
|
||||
}{{"lowest", minRand}, {"highest", maxRand}} {
|
||||
got := severedLabels(tc.name, cutOffsets(tc.name, packetCutBudget, draw.fn))
|
||||
require.Equal(t, tc.packet, got, "%s draw, packet budget: %s", draw.label, tc.why)
|
||||
got = severedLabels(tc.name, cutOffsets(tc.name, recordCutBudget, draw.fn))
|
||||
require.Equal(t, tc.record, got, "%s draw, record budget: %s", draw.label, tc.why)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCutOffsetsSeverTheBlockedLabel is the field measurement turned into an
|
||||
// instrument. On the measured provider these six names differ only in the label
|
||||
// in front of "youtube", and five of the six were blocked. What has to hold —
|
||||
// for every draw, not for most of them — is that the byte range of "youtube"
|
||||
// ends up straddling a cut.
|
||||
// instrument. On the measured provider these names differ only in the label in
|
||||
// front of "youtube", and five of the six were blocked. What has to hold — for
|
||||
// every draw and both budgets, not for most of them — is that the byte range of
|
||||
// the blocked word straddles a cut.
|
||||
func TestCutOffsetsSeverTheBlockedLabel(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, name := range []string{
|
||||
"youtube.com",
|
||||
"m.youtube.com",
|
||||
"tv.youtube.com",
|
||||
"www.youtube.com",
|
||||
"music.youtube.com",
|
||||
"studio.youtube.com",
|
||||
"i9.ytimg.com",
|
||||
for _, tc := range []struct{ name, blocked string }{
|
||||
{"youtube.com", "youtube"},
|
||||
{"m.youtube.com", "youtube"},
|
||||
{"tv.youtube.com", "youtube"},
|
||||
{"www.youtube.com", "youtube"},
|
||||
{"music.youtube.com", "youtube"},
|
||||
{"studio.youtube.com", "youtube"},
|
||||
{"cdn-static-assets.youtube.com", "youtube"},
|
||||
{"i9.ytimg.com", "ytimg"},
|
||||
{"a.b.example.co.uk", "example"},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
target := "youtube"
|
||||
if strings.Contains(name, "ytimg") {
|
||||
target = "ytimg"
|
||||
}
|
||||
start := strings.Index(name, target)
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
start := strings.Index(tc.name, tc.blocked)
|
||||
require.GreaterOrEqual(t, start, 0)
|
||||
end := start + len(target)
|
||||
// Every draw the label can take, not a sample of them: n is small
|
||||
// enough to enumerate, so there is no "it passed 1000 times" to
|
||||
// argue with.
|
||||
for draw := 0; draw < len(name); draw++ {
|
||||
offsets := cutOffsets(name, fixedRand(draw))
|
||||
severed := false
|
||||
for _, o := range offsets {
|
||||
if o > start && o < end {
|
||||
severed = true
|
||||
end := start + len(tc.blocked)
|
||||
// Every draw the label can take, not a sample: the range is small
|
||||
// enough to enumerate, so there is no "it passed 1000 times" here.
|
||||
for draw := 0; draw < len(tc.name); draw++ {
|
||||
for _, budget := range []int{packetCutBudget, recordCutBudget} {
|
||||
offsets := cutOffsets(tc.name, budget, fixedRand(draw))
|
||||
severed := false
|
||||
for _, o := range offsets {
|
||||
if o > start && o < end {
|
||||
severed = true
|
||||
}
|
||||
}
|
||||
require.True(t, severed,
|
||||
"draw %d, budget %d: %q got cuts at %v (%v), none inside %q [%d,%d)",
|
||||
draw, budget, tc.name, offsets, severedLabels(tc.name, offsets), tc.blocked, start, end)
|
||||
}
|
||||
require.True(t, severed,
|
||||
"draw %d: %q got cuts at %v, none of them inside %q [%d,%d)",
|
||||
draw, name, offsets, target, start, end)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCutOffsetsNeverLandOnALabelBoundary states the property the old code
|
||||
// broke: an offset at a label's own edge is not a cut. It is checked over every
|
||||
// draw of every label of every name.
|
||||
func TestCutOffsetsNeverLandOnALabelBoundary(t *testing.T) {
|
||||
// TestCutOffsetsStayInTheMiddleThird: every interior offset severs the label,
|
||||
// but not equally well — one byte in leaves "outube" of "youtube", which a
|
||||
// matcher keyed on a substring still reads. Both halves must keep at least
|
||||
// (width-1)/3 + 1 bytes.
|
||||
func TestCutOffsetsStayInTheMiddleThird(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, name := range []string{
|
||||
"youtube.com", "www.youtube.com", "m.youtube.com", "music.youtube.com",
|
||||
"example.co.uk", "a.b.example.co.uk", "foo.bar.baz.youtube.com",
|
||||
"localhost", "com", "co.uk", ".youtube.com", "youtube.com.",
|
||||
"youtube.com", "www.youtube.com", "cdn-static-assets.youtube.com",
|
||||
"music.youtube.com", "example.co.uk", "ab.example.com", "localhost",
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
for draw := 0; draw < 64; draw++ {
|
||||
offsets := cutOffsets(name, fixedRand(draw))
|
||||
require.NotEmpty(t, offsets, "draw %d: %q was handed over in one piece", draw, name)
|
||||
prev := -1
|
||||
for _, o := range offsets {
|
||||
require.Greater(t, o, 0, "draw %d: cut at the very start of %q hides nothing", draw, name)
|
||||
require.Less(t, o, len(name), "draw %d: cut past the end of %q", draw, name)
|
||||
require.NotEqual(t, byte('.'), name[o], "draw %d: cut at %d sits on a label boundary of %q", draw, o, name)
|
||||
require.NotEqual(t, byte('.'), name[o-1], "draw %d: cut at %d sits on a label boundary of %q", draw, o, name)
|
||||
require.Greater(t, o, prev, "draw %d: offsets of %q are not strictly increasing: %v", draw, name, offsets)
|
||||
prev = o
|
||||
for _, budget := range []int{packetCutBudget, recordCutBudget} {
|
||||
for _, o := range cutOffsets(name, budget, fixedRand(draw)) {
|
||||
label := severedLabel(name, o)
|
||||
require.NotContains(t, label, "!", "draw %d: cut at %d in %q severed nothing", draw, o, name)
|
||||
start := strings.Index(name, label)
|
||||
width := len(label)
|
||||
margin := (width-1)/3 + 1
|
||||
require.GreaterOrEqual(t, o-start, margin,
|
||||
"draw %d: cut at %d leaves only %d byte(s) of %q on the left", draw, o, o-start, label)
|
||||
require.GreaterOrEqual(t, start+width-o, margin,
|
||||
"draw %d: cut at %d leaves only %d byte(s) of %q on the right", draw, o, start+width-o, label)
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCutOffsetsFallbackStaysInBounds covers the names where NO label is
|
||||
// cuttable and the plan falls back to cutting the name itself. Boundary
|
||||
// interiority cannot hold here — there is no label to be interior to — so what
|
||||
// is checked is that the cut exists and can still be applied to a buffer.
|
||||
func TestCutOffsetsFallbackStaysInBounds(t *testing.T) {
|
||||
// TestCutOffsetsRespectTheBudget: the budget is what bounds a hostile name's
|
||||
// cost — a measured 500 ms of connection setup per cut in the packet modes.
|
||||
func TestCutOffsetsRespectTheBudget(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, name := range []string{"a.b.co.uk", "x.pp.ru", "a.b.c.d", "1.2.3.4"} {
|
||||
var long strings.Builder
|
||||
for i := 0; i < 40; i++ {
|
||||
long.WriteString("lb.")
|
||||
}
|
||||
long.WriteString("example.com") // 40 cuttable labels plus the registrable one
|
||||
for _, budget := range []int{1, 2, 3, 4} {
|
||||
require.Len(t, cutOffsets(long.String(), budget, rand.Intn), budget, "budget %d", budget)
|
||||
}
|
||||
require.Len(t, cutOffsets(long.String(), packetCutBudget, rand.Intn), 1,
|
||||
"a 253-byte SNI must not be able to buy more than one 500 ms wait")
|
||||
require.Len(t, cutOffsets(long.String(), recordCutBudget, rand.Intn), 4,
|
||||
"nor more than five records")
|
||||
}
|
||||
|
||||
// TestCutOffsetsFallBackWhenNoLabelCanBeCut covers the names where NO candidate
|
||||
// label has an interior offset. Severing a label is impossible there, so what
|
||||
// is checked is that a cut still happens and still lands inside the buffer: a
|
||||
// matcher keyed on the whole FQDN fails across it.
|
||||
func TestCutOffsetsFallBackWhenNoLabelCanBeCut(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, name := range []string{"a.b.co.uk", "x.pp.ru", "a.b.c.d", "1.2.3.4", "::1", "x.com"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
for draw := 0; draw < len(name)+4; draw++ {
|
||||
offsets := cutOffsets(name, fixedRand(draw))
|
||||
require.Len(t, offsets, 1, "%q should fall back to exactly one cut", name)
|
||||
require.Greater(t, offsets[0], 0)
|
||||
require.Less(t, offsets[0], len(name))
|
||||
for _, budget := range []int{packetCutBudget, recordCutBudget} {
|
||||
offsets := cutOffsets(name, budget, fixedRand(draw))
|
||||
require.NotEmpty(t, offsets, "%q went out in one piece", name)
|
||||
require.Greater(t, offsets[0], 0)
|
||||
require.Less(t, offsets[len(offsets)-1], len(name))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCutOffsetsSurviveRealRandomness runs the production draw source over the
|
||||
// degenerate names that used to reach rand.Intn(0) — an empty label panics it,
|
||||
// and ".youtube.com" / "youtube.com." are the two ways an ordinary client
|
||||
// produces one.
|
||||
func TestCutOffsetsSurviveRealRandomness(t *testing.T) {
|
||||
// TestCutOffsetsAreOrderedAndDistinct: the write loop slices b between
|
||||
// consecutive offsets, so anything out of order or repeated is an empty or
|
||||
// negative segment on the wire. Candidates come back in PRIORITY order, which
|
||||
// is not wire order — this is the test that the sort is not forgotten.
|
||||
func TestCutOffsetsAreOrderedAndDistinct(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, name := range []string{
|
||||
"", "a", ".", "..", "...", ".com", "com.", ".youtube.com", "youtube.com.",
|
||||
".youtube.com.", "a..b.example.com", "-.-.-.-", "xn--p1ai", "test.xn--p1ai",
|
||||
"www.youtube.com", "foo.bar.baz.youtube.com", "cdn-static-assets.youtube.com",
|
||||
"a.bb.ccc.dddd.example.com", "youtube.com.", ".youtube.com", "co.uk",
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
for i := 0; i < 200; i++ {
|
||||
offsets := cutOffsets(name, rand.Intn) // must not panic
|
||||
prev := -1
|
||||
offsets := cutOffsets(name, recordCutBudget, rand.Intn)
|
||||
prev := 0
|
||||
for _, o := range offsets {
|
||||
require.Greater(t, o, prev)
|
||||
require.Greater(t, o, 0)
|
||||
require.Less(t, o, len(name))
|
||||
require.Greater(t, o, prev, "%q: offsets %v are not strictly increasing", name, offsets)
|
||||
prev = o
|
||||
}
|
||||
require.Less(t, prev, len(name))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// --- the arithmetic must not panic on anything ------------------------------
|
||||
|
||||
// adversarialNames is the closed list of shapes that reach the arithmetic from
|
||||
// outside: empty and one-byte names, every position a dot can take, names that
|
||||
// are nothing but dots, names at the 253-byte limit, and bytes that are not
|
||||
// ASCII at all. The unit test, the fuzz seed corpus and the end-to-end test all
|
||||
// draw from it, so all three see the same inputs.
|
||||
func adversarialNames() []string {
|
||||
return []string{
|
||||
"", "a", ".", "..", "...", "....",
|
||||
".com", "com.", ".com.", ".youtube.com", "youtube.com.", ".youtube.com.",
|
||||
"a..b.example.com", "..youtube..com..", "-.-.-.-", "-", "--",
|
||||
"xn--p1ai", "test.xn--p1ai", "xn--", ".xn--p1ai.",
|
||||
"\xff\xfe.example.com", "\x00\x00.com", "пример.рф", "\xff",
|
||||
strings.Repeat("a", 253),
|
||||
strings.Repeat("ab.", 84) + "a", // 253 bytes, 85 labels
|
||||
strings.Repeat(".", 253),
|
||||
strings.Repeat("a.", 126) + "a",
|
||||
"1.2.3.4", "::1", "::ffff:1.2.3.4", "fe80::1%eth0", "0.0.0.0",
|
||||
}
|
||||
}
|
||||
|
||||
func TestCutOffsetsSurviveEveryAdversarialName(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, name := range adversarialNames() {
|
||||
t.Run(strings.ToValidUTF8(name, "?"), func(t *testing.T) {
|
||||
for i := 0; i < 100; i++ {
|
||||
for _, budget := range []int{packetCutBudget, recordCutBudget} {
|
||||
offsets := cutOffsets(name, budget, rand.Intn) // must not panic
|
||||
require.LessOrEqual(t, len(offsets), budget)
|
||||
if len(name) >= 2 {
|
||||
require.NotEmpty(t, offsets, "%q is long enough to cut and was not cut", name)
|
||||
} else {
|
||||
require.Empty(t, offsets, "%q has no offset that leaves bytes on both sides", name)
|
||||
}
|
||||
prev := 0
|
||||
for _, o := range offsets {
|
||||
require.Greater(t, o, prev)
|
||||
require.Less(t, o, len(name))
|
||||
prev = o
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// FuzzCutOffsets is the open half of the audit above: the closed list says what
|
||||
// we thought of, this says whether anything else reaches rand.Intn with a
|
||||
// non-positive argument or produces an offset the write loop cannot slice at.
|
||||
// Under plain `go test` it runs the seed corpus, which is that closed list.
|
||||
func FuzzCutOffsets(f *testing.F) {
|
||||
for _, name := range adversarialNames() {
|
||||
f.Add(name, packetCutBudget)
|
||||
f.Add(name, recordCutBudget)
|
||||
}
|
||||
for _, name := range []string{"www.youtube.com", "a.b.example.co.uk", "localhost"} {
|
||||
f.Add(name, 1)
|
||||
f.Add(name, 4)
|
||||
}
|
||||
f.Fuzz(func(t *testing.T, name string, budget int) {
|
||||
if budget < 0 {
|
||||
budget = -budget
|
||||
}
|
||||
budget = budget%recordCutBudget + 1 // 1..4, never zero or negative
|
||||
offsets := cutOffsets(name, budget, rand.Intn)
|
||||
if len(offsets) > budget {
|
||||
t.Fatalf("%q: %d offsets for a budget of %d", name, len(offsets), budget)
|
||||
}
|
||||
if len(name) >= 2 && len(offsets) == 0 {
|
||||
t.Fatalf("%q (%d bytes) was handed over in one piece", name, len(name))
|
||||
}
|
||||
prev := 0
|
||||
for _, o := range offsets {
|
||||
if o <= prev || o >= len(name) {
|
||||
t.Fatalf("%q: offsets %v are not strictly increasing inside [1,%d)", name, offsets, len(name))
|
||||
}
|
||||
prev = o
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// --- end to end: what actually goes out on the wire -------------------------
|
||||
|
||||
// fakeConn records every Write. It is deliberately NOT a *net.TCPConn, which is
|
||||
// also the common production case (the outbound is usually a proxy stream), so
|
||||
// Conn.Write takes the sleep-instead-of-ACK path — hence the 1ns fallback delay
|
||||
// the tests below pass to NewConn.
|
||||
// also the common production case (the outbound is usually a proxy stream whose
|
||||
// reader transforms the bytes, so N.UnwrapReader stops there), so Conn.Write
|
||||
// takes the sleep-instead-of-ACK path — hence the 1ns fallback delay the tests
|
||||
// below pass to NewConn.
|
||||
type fakeConn struct {
|
||||
writes [][]byte
|
||||
}
|
||||
@@ -245,10 +395,53 @@ func clientHelloFor(t *testing.T, serverName string) []byte {
|
||||
return hello
|
||||
}
|
||||
|
||||
// buildClientHello assembles a ClientHello by hand around a server_name_list of
|
||||
// the given entries. crypto/tls will not emit a name with a leading or trailing
|
||||
// dot, an IP literal, an empty name or a second list entry — hostnameInSNI
|
||||
// rewrites or refuses all of them — and those are exactly the shapes a
|
||||
// forwarded ClientHello from a LAN client can carry.
|
||||
func buildClientHello(t *testing.T, entries ...string) []byte {
|
||||
t.Helper()
|
||||
var list []byte
|
||||
for _, e := range entries {
|
||||
list = append(list, sniNameDNSHostnameType)
|
||||
list = binary.BigEndian.AppendUint16(list, uint16(len(e)))
|
||||
list = append(list, e...)
|
||||
}
|
||||
extBody := binary.BigEndian.AppendUint16(nil, uint16(len(list)))
|
||||
extBody = append(extBody, list...)
|
||||
|
||||
ext := binary.BigEndian.AppendUint16(nil, sniExtensionType)
|
||||
ext = binary.BigEndian.AppendUint16(ext, uint16(len(extBody)))
|
||||
ext = append(ext, extBody...)
|
||||
|
||||
extensions := binary.BigEndian.AppendUint16(nil, uint16(len(ext)))
|
||||
extensions = append(extensions, ext...)
|
||||
|
||||
body := []byte{0x03, 0x03} // client_version TLS 1.2
|
||||
body = append(body, make([]byte, 32)...) // random
|
||||
body = append(body, 0x00) // session_id length
|
||||
body = append(body, 0x00, 0x02, 0x13, 0x01) // cipher_suites
|
||||
body = append(body, 0x01, 0x00) // compression_methods
|
||||
body = append(body, extensions...)
|
||||
handshake := []byte{handshakeType, byte(len(body) >> 16), byte(len(body) >> 8), byte(len(body))}
|
||||
handshake = append(handshake, body...)
|
||||
|
||||
record := []byte{contentType, 0x03, 0x01}
|
||||
record = binary.BigEndian.AppendUint16(record, uint16(len(handshake)))
|
||||
record = append(record, handshake...)
|
||||
|
||||
// Control on the instrument: this hand-built record must parse the way a
|
||||
// real one does, or the tests below are testing a straw man.
|
||||
sni := IndexTLSServerName(record)
|
||||
require.NotNil(t, sni, "hand-built ClientHello did not parse")
|
||||
require.Equal(t, len(entries[0]), sni.Length, "Length must be the FIRST entry")
|
||||
require.Equal(t, entries[0], string(record[sni.Index:sni.Index+sni.Length]))
|
||||
return record
|
||||
}
|
||||
|
||||
// patchSNI rewrites the server name inside a ClientHello in place. from and to
|
||||
// must be the same length, so every length field in the record stays valid —
|
||||
// this is how names crypto/tls refuses to emit (a leading or trailing dot, an
|
||||
// IP literal) get tested against the real parser.
|
||||
// must be the same length, so every length field in the record stays valid.
|
||||
func patchSNI(t *testing.T, hello []byte, from, to string) []byte {
|
||||
t.Helper()
|
||||
require.Equal(t, len(from), len(to), "patchSNI cannot change the length")
|
||||
@@ -305,12 +498,13 @@ type writeMode struct {
|
||||
splitRecord bool
|
||||
recordFraming bool
|
||||
segmentPerCall bool // one Write call per segment
|
||||
budget int
|
||||
}
|
||||
|
||||
var writeModes = []writeMode{
|
||||
{name: "tls_fragment", splitPacket: true, splitRecord: false, recordFraming: false, segmentPerCall: true},
|
||||
{name: "tls_record_fragment", splitPacket: false, splitRecord: true, recordFraming: true, segmentPerCall: false},
|
||||
{name: "both", splitPacket: true, splitRecord: true, recordFraming: true, segmentPerCall: true},
|
||||
{name: "tls_fragment", splitPacket: true, segmentPerCall: true, budget: packetCutBudget},
|
||||
{name: "tls_record_fragment", splitRecord: true, recordFraming: true, budget: recordCutBudget},
|
||||
{name: "both", splitPacket: true, splitRecord: true, recordFraming: true, segmentPerCall: true, budget: packetCutBudget},
|
||||
}
|
||||
|
||||
// TestWriteSeversTheBlockedLabelOnTheWire is the end-to-end control: not "the
|
||||
@@ -320,23 +514,20 @@ var writeModes = []writeMode{
|
||||
func TestWriteSeversTheBlockedLabelOnTheWire(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, mode := range writeModes {
|
||||
for _, serverName := range []string{
|
||||
"youtube.com", // the ONE name the old code got right
|
||||
"www.youtube.com", // the regression
|
||||
"m.youtube.com", // one-byte label in front
|
||||
"music.youtube.com", //
|
||||
"a.b.example.co.uk", // two-label public suffix
|
||||
for _, tc := range []struct{ name, blocked string }{
|
||||
{"youtube.com", "youtube"}, // the ONE name the old code got right
|
||||
{"www.youtube.com", "youtube"}, // the regression
|
||||
{"m.youtube.com", "youtube"}, // one-byte label in front
|
||||
{"music.youtube.com", "youtube"},
|
||||
{"cdn-static-assets.youtube.com", "youtube"},
|
||||
{"a.b.example.co.uk", "example"}, // two-label public suffix
|
||||
} {
|
||||
t.Run(mode.name+"/"+serverName, func(t *testing.T) {
|
||||
t.Run(mode.name+"/"+tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
hello := clientHelloFor(t, serverName)
|
||||
hello := clientHelloFor(t, tc.name)
|
||||
sniAt := IndexTLSServerName(hello).Index
|
||||
target := "youtube"
|
||||
if strings.Contains(serverName, "example") {
|
||||
target = "example"
|
||||
}
|
||||
start := sniAt + strings.Index(serverName, target)
|
||||
end := start + len(target)
|
||||
start := sniAt + strings.Index(tc.name, tc.blocked)
|
||||
end := start + len(tc.blocked)
|
||||
for i := 0; i < 100; i++ {
|
||||
out := &fakeConn{}
|
||||
n, err := NewConn(out, t.Context(), mode.splitPacket, mode.splitRecord, time.Nanosecond).Write(hello)
|
||||
@@ -344,6 +535,7 @@ func TestWriteSeversTheBlockedLabelOnTheWire(t *testing.T) {
|
||||
require.Equal(t, len(hello), n, "Write must report the length of the buffer it was given")
|
||||
_, cuts := segments(t, hello, out.writes, mode.recordFraming)
|
||||
require.NotEmpty(t, cuts, "the ClientHello went out in one piece")
|
||||
require.LessOrEqual(t, len(cuts), mode.budget, "more cuts than this mode's budget")
|
||||
severed := false
|
||||
for _, c := range cuts {
|
||||
if c > start && c < end {
|
||||
@@ -352,7 +544,7 @@ func TestWriteSeversTheBlockedLabelOnTheWire(t *testing.T) {
|
||||
}
|
||||
require.True(t, severed,
|
||||
"run %d: %q left with cuts at %v, none inside %q [%d,%d)",
|
||||
i, serverName, cuts, target, start, end)
|
||||
i, tc.name, cuts, tc.blocked, start, end)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -367,6 +559,7 @@ func TestWriteReassemblesToTheOriginalClientHello(t *testing.T) {
|
||||
for _, mode := range writeModes {
|
||||
for _, serverName := range []string{
|
||||
"www.youtube.com", "youtube.com", "a.b.example.co.uk", "localhost", "a",
|
||||
"foo.bar.baz.youtube.com",
|
||||
} {
|
||||
t.Run(mode.name+"/"+serverName, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
@@ -401,34 +594,46 @@ func TestWriteReassemblesToTheOriginalClientHello(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestWriteHandlesDegenerateServerNames feeds Conn.Write the names crypto/tls
|
||||
// refuses to emit but a forwarded ClientHello can carry. Before the fix the
|
||||
// first two of these reached rand.Intn(0) and took the process down with
|
||||
// "invalid argument to Intn"; the ClientHello here comes from a LAN client, so
|
||||
// that was reachable from the LAN.
|
||||
func TestWriteHandlesDegenerateServerNames(t *testing.T) {
|
||||
// TestWriteDoesNotPanicOnAServerNameChosenFromTheLAN is the regression for a
|
||||
// PROCESS DEATH that shipped in v0.2.21.
|
||||
//
|
||||
// route/conn.go wraps the outbound connection with this Conn and fragments the
|
||||
// ClientHello the LAN client sent, so the server name is chosen by the client,
|
||||
// not by us. An empty label made cutOffsets call rand.Intn(0) — "panic: invalid
|
||||
// argument to Intn" — and a Go panic in a connection goroutine takes the whole
|
||||
// daemon with it. Two ordinary ways to produce one:
|
||||
//
|
||||
// "youtube.com." a fully qualified name with the root dot, which curl and
|
||||
// every browser will happily send, and for which the public
|
||||
// suffix list returns "" so the trailing empty label survived;
|
||||
// ".youtube.com" a leading dot, which nothing legitimate sends but nothing
|
||||
// stops a client from writing into its own ClientHello.
|
||||
//
|
||||
// With the kill switch armed the daemon's death is not a slow connection, it is
|
||||
// a dark LAN until procd restarts it — into the same request.
|
||||
func TestWriteDoesNotPanicOnAServerNameChosenFromTheLAN(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, tc := range []struct {
|
||||
from, to string
|
||||
}{
|
||||
{from: "xyoutube.com", to: ".youtube.com"},
|
||||
{from: "youtube.comx", to: "youtube.com."},
|
||||
{from: "xyoutube.comx", to: ".youtube.com."},
|
||||
{from: "ax.example.com", to: "a..example.com"},
|
||||
{from: "1x2x3x4", to: "1.2.3.4"},
|
||||
{from: "localhost", to: "localhost"},
|
||||
{from: "ab", to: "ab"},
|
||||
{from: "a", to: "a"},
|
||||
for _, tc := range []struct{ from, to, why string }{
|
||||
{from: "youtube.comx", to: "youtube.com.", why: "the FQDN root dot — a legitimate name"},
|
||||
{from: "xyoutube.com", to: ".youtube.com", why: "leading dot"},
|
||||
{from: "xyoutube.comx", to: ".youtube.com.", why: "both"},
|
||||
{from: "ax.example.com", to: "a..example.com", why: "a doubled dot mid-name"},
|
||||
{from: "xxxxxxxxxxxx", to: "............", why: "nothing but dots"},
|
||||
{from: "1x2x3x4", to: "1.2.3.4", why: "an IP literal, which RFC 6066 forbids in SNI"},
|
||||
{from: "xxx", to: "::1", why: "an IPv6 literal"},
|
||||
{from: "a", to: "a", why: "one byte: no cut exists, and the empty plan must not be indexed"},
|
||||
{from: "ab", to: "ab", why: "two bytes: exactly one interior offset"},
|
||||
{from: "\xff\xfe.example.com", to: "\xff\xfe.example.com", why: "bytes that are not ASCII"},
|
||||
} {
|
||||
t.Run(tc.to, func(t *testing.T) {
|
||||
t.Run(strings.ToValidUTF8(tc.to, "?"), func(t *testing.T) {
|
||||
t.Parallel()
|
||||
hello := patchSNI(t, clientHelloFor(t, tc.from), tc.from, tc.to)
|
||||
for _, mode := range writeModes {
|
||||
for i := 0; i < 50; i++ {
|
||||
out := &fakeConn{}
|
||||
n, err := NewConn(out, t.Context(), mode.splitPacket, mode.splitRecord, time.Nanosecond).Write(hello)
|
||||
require.NoError(t, err, "%s", mode.name)
|
||||
require.Equal(t, len(hello), n, "%s", mode.name)
|
||||
require.NoError(t, err, "%s: %s", mode.name, tc.why)
|
||||
require.Equal(t, len(hello), n, "%s: %s", mode.name, tc.why)
|
||||
payloads, _ := segments(t, hello, out.writes, mode.recordFraming)
|
||||
var joined []byte
|
||||
for _, p := range payloads {
|
||||
@@ -439,6 +644,54 @@ func TestWriteHandlesDegenerateServerNames(t *testing.T) {
|
||||
if mode.recordFraming {
|
||||
want = hello[recordLayerHeaderLen:]
|
||||
}
|
||||
require.Equal(t, want, joined, "%s run %d: %s", mode.name, i, tc.why)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestWriteHandlesServerNameListsCryptoTLSWillNotEmit reaches the shapes that
|
||||
// need a hand-built record: a zero-length name, a name at the 253-byte limit,
|
||||
// and a list carrying a SECOND entry — which MyServerName.ServerName includes
|
||||
// and MyServerName.Length does not, so cut planning must run on the first entry
|
||||
// alone or it feeds the public suffix list bytes that belong to no name.
|
||||
func TestWriteHandlesServerNameListsCryptoTLSWillNotEmit(t *testing.T) {
|
||||
t.Parallel()
|
||||
for _, tc := range []struct {
|
||||
title string
|
||||
entries []string
|
||||
cut bool // must the ClientHello leave in more than one piece?
|
||||
}{
|
||||
{title: "empty name", entries: []string{""}, cut: false},
|
||||
{title: "one byte", entries: []string{"a"}, cut: false},
|
||||
{title: "253 bytes, 85 labels", entries: []string{strings.Repeat("ab.", 84) + "a"}, cut: true},
|
||||
{title: "253 bytes, one label", entries: []string{strings.Repeat("a", 253)}, cut: true},
|
||||
{title: "two entries", entries: []string{"www.youtube.com", "evil.example.com"}, cut: true},
|
||||
{title: "two entries, first empty", entries: []string{"", "www.youtube.com"}, cut: false},
|
||||
{title: "trailing dot", entries: []string{"youtube.com."}, cut: true},
|
||||
} {
|
||||
t.Run(tc.title, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
hello := buildClientHello(t, tc.entries...)
|
||||
for _, mode := range writeModes {
|
||||
for i := 0; i < 20; i++ {
|
||||
out := &fakeConn{}
|
||||
n, err := NewConn(out, t.Context(), mode.splitPacket, mode.splitRecord, time.Nanosecond).Write(hello)
|
||||
require.NoError(t, err, mode.name)
|
||||
require.Equal(t, len(hello), n, mode.name)
|
||||
payloads, cuts := segments(t, hello, out.writes, mode.recordFraming)
|
||||
require.Equal(t, tc.cut, len(cuts) > 0, "%s: expected cut=%v, got %d cut(s)", mode.name, tc.cut, len(cuts))
|
||||
require.LessOrEqual(t, len(cuts), mode.budget, mode.name)
|
||||
var joined []byte
|
||||
for _, p := range payloads {
|
||||
require.NotEmpty(t, p)
|
||||
joined = append(joined, p...)
|
||||
}
|
||||
want := hello
|
||||
if mode.recordFraming {
|
||||
want = hello[recordLayerHeaderLen:]
|
||||
}
|
||||
require.Equal(t, want, joined, "%s run %d", mode.name, i)
|
||||
}
|
||||
}
|
||||
@@ -446,6 +699,27 @@ func TestWriteHandlesDegenerateServerNames(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestWriteCutsTheFirstEntryOfTheServerNameList: with two entries the cut must
|
||||
// land inside "youtube" of the FIRST one. Planning over the whole remainder of
|
||||
// the extension would hand the public suffix list a string that is not a name
|
||||
// and put the cut somewhere else entirely.
|
||||
func TestWriteCutsTheFirstEntryOfTheServerNameList(t *testing.T) {
|
||||
t.Parallel()
|
||||
hello := buildClientHello(t, "www.youtube.com", "cdn-static-assets.example.com")
|
||||
sni := IndexTLSServerName(hello)
|
||||
start := sni.Index + strings.Index("www.youtube.com", "youtube")
|
||||
end := start + len("youtube")
|
||||
for i := 0; i < 200; i++ {
|
||||
out := &fakeConn{}
|
||||
_, err := NewConn(out, t.Context(), true, false, time.Nanosecond).Write(hello)
|
||||
require.NoError(t, err)
|
||||
_, cuts := segments(t, hello, out.writes, false)
|
||||
require.Len(t, cuts, 1)
|
||||
require.Greater(t, cuts[0], start, "run %d: cut at %d is outside youtube [%d,%d)", i, cuts[0], start, end)
|
||||
require.Less(t, cuts[0], end, "run %d: cut at %d is outside youtube [%d,%d)", i, cuts[0], start, end)
|
||||
}
|
||||
}
|
||||
|
||||
// TestWriteWithoutSNIIsUntouched: the fast path must stay a straight pass, and
|
||||
// the second and later writes must never be re-planned.
|
||||
func TestWriteWithoutSNIIsUntouched(t *testing.T) {
|
||||
|
||||
Reference in New Issue
Block a user