omarandClaude Opus 4.8
8980a25a59
Audit of the run-51 logs showed actions/cache@v3.3.2 works on the act_runner
(cold: "Cache saved" x4; next job: "Cache restored" in ~2s, npm --fast skip,
usign/dl reused) and the sdk-cache mirror seeds correctly — but the single
biggest recurring cost was NOT cached: `scripts/feeds update -a` re-cloned
base+packages+luci+routing+telephony every run (~7.8 min warm x 4 SDK jobs on
the serial runner ≈ ~28 min/run wasted; github ~1 MB/s from this host).
Cache .cache/feeds/{opkg,apk} (workspace dir, actions/cache-persisted, visible
in the SDK container via --volumes-from) symlinked over the SDK's empty feeds/:
`feeds update` now git-fetches deltas (seconds) instead of full clones, always
checking out feeds.conf's pins. Fail-safe: any error on the cached checkouts
wipes the cache and clones fresh. Key by SDK release (feeds-opkg-24.10.4 /
feeds-apk-25.12.1) — stable across runs, invalidates on an SDK bump; both arch
jobs of a lane share one entry (identical pins, serial runner).
Steady-state warm run: ~60+ min -> ~20-22 min. Also documented in the workflow
header: never key a cache on github.sha — each cache SAVE stalls the act_runner
~3 min, so per-run-changing keys would add +3 min/entry every run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
93 lines
4.5 KiB
Bash
93 lines
4.5 KiB
Bash
#!/bin/sh
|
|
# ci/build-feed.sh — build the signed opkg feed for ONE arch.
|
|
#
|
|
# Usage: ci/build-feed.sh <ARCH> <SDK_DOCKER_TAG> <OUTDIR>
|
|
# e.g. ci/build-feed.sh x86_64 x86_64-24.10.4 out/x86_64
|
|
# ci/build-feed.sh aarch64_cortex-a53 mediatek-filogic-24.10.4 out/aarch64_cortex-a53
|
|
#
|
|
# This is the reusable per-arch entrypoint the Gitea workflow calls. It runs on
|
|
# the CI RUNNER and:
|
|
# 1. asserts the prebuilt shaterd binary for this arch was already staged by
|
|
# scripts/build-shaterd.sh (into openwrt/shaterd/files/) — proving artifact
|
|
# order: SPA+shaterd build BEFORE the SDK package build;
|
|
# 2. drives the arch-matched `openwrt/sdk` docker image to compile all 4
|
|
# packages (ci/sdk-build.sh) and collect their .ipk into OUTDIR;
|
|
# 3. builds + usign-signs the opkg `Packages` index over OUTDIR
|
|
# (ci/install-usign.sh + ci/make-index.sh; signs iff $KEY_BUILD is set).
|
|
#
|
|
# Env:
|
|
# KEY_BUILD usign SECRET key (Gitea repo secret). If set, the feed index is
|
|
# signed and verifiable by dist/shater-feed.pub (fp 5ac4b177689cb8e0).
|
|
# If unset, an UNSIGNED feed is produced (make-index warns).
|
|
set -eu
|
|
|
|
ARCH="${1:?arch required (x86_64 | aarch64_cortex-a53)}"
|
|
SDK_TAG="${2:?sdk docker tag required (e.g. x86_64-24.10.4)}"
|
|
OUT="${3:?output dir required}"
|
|
|
|
REPO="$(cd "$(dirname "$0")/.." && pwd)"
|
|
mkdir -p "$OUT"; OUT="$(cd "$OUT" && pwd)"
|
|
# $OUT is created here as ROOT on the runner, but the nested `openwrt/sdk`
|
|
# container runs as the unprivileged `buildbot` (uid 1000) — so it must be able
|
|
# to write the collected .ipk into $OUT. World-writable is set HERE (a chmod
|
|
# from inside the container, as buildbot, cannot fix a root-owned dir).
|
|
chmod 0777 "$OUT"
|
|
|
|
# --- 0) the prebuilt shaterd binary must already be staged for this arch ------
|
|
case "$ARCH" in
|
|
x86_64) sfx=amd64 ;;
|
|
aarch64_cortex-a53) sfx=arm64 ;;
|
|
*) echo "[feed] ERROR: unsupported ARCH '$ARCH'"; exit 2 ;;
|
|
esac
|
|
if [ ! -f "$REPO/openwrt/shaterd/files/shaterd-$sfx.upx" ]; then
|
|
echo "[feed] ERROR: openwrt/shaterd/files/shaterd-$sfx.upx not staged."
|
|
echo " Run scripts/build-shaterd.sh BEFORE ci/build-feed.sh." >&2
|
|
exit 3
|
|
fi
|
|
|
|
chmod +x "$REPO"/ci/*.sh 2>/dev/null || true
|
|
|
|
# --- 0.5) persistent dl/ (package source tarballs) ----------------------------
|
|
# Workspace dir restored/saved by actions/cache in the workflow and shared into
|
|
# the nested SDK container via --volumes-from; becomes CONFIG_DOWNLOAD_FOLDER
|
|
# there (ci/sdk-build.sh). PKG_HASH still verifies every file, so a stale cache
|
|
# can never produce a wrong build. Must be writable by the container's
|
|
# unprivileged buildbot user (same reason as the $OUT chmod above).
|
|
DL_DIR="$REPO/.cache/dl"
|
|
mkdir -p "$DL_DIR"
|
|
chmod -R a+rwX "$DL_DIR" 2>/dev/null || true
|
|
|
|
# --- 0.6) persistent feeds/ git checkouts -------------------------------------
|
|
# Workspace dir restored/saved by actions/cache (key: feeds-opkg-<release>) and
|
|
# symlinked over the SDK's feeds/ inside the container (ci/sdk-build.sh), so
|
|
# `scripts/feeds update -a` fetches deltas instead of re-cloning base+packages+
|
|
# luci from scratch (~7 min/run on this runner's slow github.com link).
|
|
# Top-level chmod only: the contents are created by the container's uid-1000
|
|
# build user and restored with the same ownership (tar-as-root preserves it).
|
|
FEEDS_CACHE="$REPO/.cache/feeds/opkg"
|
|
mkdir -p "$FEEDS_CACHE"
|
|
chmod a+rwX "$REPO/.cache" "$REPO/.cache/feeds" "$FEEDS_CACHE" 2>/dev/null || true
|
|
|
|
# --- 1) SDK package build (4 packages) in the arch-matched SDK image ----------
|
|
# We drive the `openwrt/sdk` docker image directly (not openwrt/gh-action-sdk):
|
|
# on a self-hosted Gitea act_runner the marketplace action fetch can be
|
|
# unavailable, and we need a CLEAN single-feed layout. `--volumes-from
|
|
# $(hostname)` shares THIS job container's workspace volume into the nested SDK
|
|
# container — a bare `-v $PWD:...` points at a host path that does not exist
|
|
# under the act_runner DinD setup. (Requires the job to run inside a container,
|
|
# which Gitea Actions does by default.)
|
|
echo "[feed] SDK build arch=$ARCH image=openwrt/sdk:$SDK_TAG"
|
|
docker pull "openwrt/sdk:$SDK_TAG"
|
|
docker run --rm --volumes-from "$(hostname)" \
|
|
-e ARCH="$ARCH" -e REPO="$REPO" -e OUT="$OUT" -e DL_DIR="$DL_DIR" \
|
|
-e FEEDS_CACHE="$FEEDS_CACHE" \
|
|
"openwrt/sdk:$SDK_TAG" \
|
|
sh "$REPO/ci/sdk-build.sh"
|
|
|
|
# --- 2) index + sign the per-arch feed (usign, KEY_BUILD passed through) -------
|
|
sh "$REPO/ci/install-usign.sh"
|
|
KEY_BUILD="${KEY_BUILD:-}" bash "$REPO/ci/make-index.sh" "$OUT"
|
|
|
|
echo "[feed] done arch=$ARCH -> $OUT"
|
|
ls -l "$OUT"
|