omarandClaude Opus 4.8
8980a25a59
Audit of the run-51 logs showed actions/cache@v3.3.2 works on the act_runner
(cold: "Cache saved" x4; next job: "Cache restored" in ~2s, npm --fast skip,
usign/dl reused) and the sdk-cache mirror seeds correctly — but the single
biggest recurring cost was NOT cached: `scripts/feeds update -a` re-cloned
base+packages+luci+routing+telephony every run (~7.8 min warm x 4 SDK jobs on
the serial runner ≈ ~28 min/run wasted; github ~1 MB/s from this host).
Cache .cache/feeds/{opkg,apk} (workspace dir, actions/cache-persisted, visible
in the SDK container via --volumes-from) symlinked over the SDK's empty feeds/:
`feeds update` now git-fetches deltas (seconds) instead of full clones, always
checking out feeds.conf's pins. Fail-safe: any error on the cached checkouts
wipes the cache and clones fresh. Key by SDK release (feeds-opkg-24.10.4 /
feeds-apk-25.12.1) — stable across runs, invalidates on an SDK bump; both arch
jobs of a lane share one entry (identical pins, serial runner).
Steady-state warm run: ~60+ min -> ~20-22 min. Also documented in the workflow
header: never key a cache on github.sha — each cache SAVE stalls the act_runner
~3 min, so per-run-changing keys would add +3 min/entry every run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
109 lines
5.5 KiB
Bash
Executable File
109 lines
5.5 KiB
Bash
Executable File
#!/bin/sh
|
|
# ci/build-feed-apk.sh — build the signed **apk** feed for ONE arch (the 25.12
|
|
# lane — additive next to ci/build-feed.sh, which stays the opkg/24.10 lane).
|
|
#
|
|
# Usage: ci/build-feed-apk.sh <ARCH> <SDK_URL> <OUTDIR>
|
|
# e.g. ci/build-feed-apk.sh aarch64_cortex-a53 \
|
|
# https://downloads.immortalwrt.org/releases/25.12.1/targets/mediatek/filogic/immortalwrt-sdk-25.12.1-mediatek-filogic_gcc-14.3.0_musl.Linux-x86_64.tar.zst \
|
|
# out-apk/aarch64_cortex-a53
|
|
#
|
|
# This is the per-arch entrypoint the Gitea workflow's `build-apk` job calls.
|
|
# It runs on the CI RUNNER and:
|
|
# 1. asserts the prebuilt shaterd binary for this arch was already staged by
|
|
# scripts/build-shaterd.sh (same artifact-order contract as the opkg lane);
|
|
# 2. drives a plain `debian:bookworm` container (workspace shared via
|
|
# `--volumes-from`, same trick as ci/build-feed.sh) that downloads the
|
|
# ImmortalWrt 25.12 apk-SDK tarball and runs ci/sdk-build-apk.sh in it:
|
|
# compile the 4 packages as .apk, then `apk mkndx --sign` the per-arch
|
|
# `packages.adb` index. Unlike the usign lane (index signed on the runner),
|
|
# apk indexing NEEDS the SDK's host `apk` tool, so index+sign happen inside
|
|
# the container.
|
|
#
|
|
# Why the ImmortalWrt SDK (not openwrt/sdk images): the 25.12 fleet runs
|
|
# BananaWRT 25.12-mtk-vendor = ImmortalWrt 25.12 base (target mediatek/filogic,
|
|
# pkg arch aarch64_cortex-a53), and Docker Hub has no immortalwrt/sdk
|
|
# mediatek-filogic 25.12 tag — hence the official SDK tarball.
|
|
#
|
|
# Env:
|
|
# KEY_APK EC (prime256v1) PRIVATE key PEM (Gitea repo secret — the apk analog
|
|
# of KEY_BUILD). If set, packages.adb carries an embedded signature
|
|
# verifiable by dist/shater-apk.pem (routers: /etc/apk/keys/).
|
|
# If unset, an UNSIGNED index is produced (warning; not shippable —
|
|
# apk signatures are effectively mandatory).
|
|
set -eu
|
|
|
|
ARCH="${1:?arch required (x86_64 | aarch64_cortex-a53)}"
|
|
SDK_URL="${2:?ImmortalWrt SDK tarball URL required (.tar.zst)}"
|
|
OUT="${3:?output dir required}"
|
|
|
|
REPO="$(cd "$(dirname "$0")/.." && pwd)"
|
|
mkdir -p "$OUT"; OUT="$(cd "$OUT" && pwd)"
|
|
|
|
# --- 0) the prebuilt shaterd binary must already be staged for this arch ------
|
|
case "$ARCH" in
|
|
x86_64) sfx=amd64 ;;
|
|
aarch64_cortex-a53) sfx=arm64 ;;
|
|
*) echo "[apk-feed] ERROR: unsupported ARCH '$ARCH'"; exit 2 ;;
|
|
esac
|
|
if [ ! -f "$REPO/openwrt/shaterd/files/shaterd-$sfx.upx" ]; then
|
|
echo "[apk-feed] ERROR: openwrt/shaterd/files/shaterd-$sfx.upx not staged."
|
|
echo " Run scripts/build-shaterd.sh BEFORE ci/build-feed-apk.sh." >&2
|
|
exit 3
|
|
fi
|
|
|
|
[ -n "${KEY_APK:-}" ] || echo "[apk-feed] WARNING: KEY_APK not set — the apk index will be UNSIGNED"
|
|
|
|
chmod +x "$REPO"/ci/*.sh 2>/dev/null || true
|
|
|
|
# --- 0.5) runner-side caches --------------------------------------------------
|
|
# All under $REPO/.cache so (a) actions/cache in the workflow can persist them
|
|
# between runs and (b) the nested container sees them via --volumes-from.
|
|
# sdk/ SDK tarballs (keyed in the workflow by tarball basename)
|
|
# dl/ package source tarballs — becomes CONFIG_DOWNLOAD_FOLDER inside the
|
|
# SDK; PKG_HASH still verifies every file, so stale = re-downloaded.
|
|
# apt/ debian:bookworm .deb archives for the host-deps install.
|
|
# The nested container runs the build as an unprivileged user -> must be writable
|
|
# (same reason as the chmod 0777 "$OUT" in ci/build-feed.sh).
|
|
CACHE="$REPO/.cache"
|
|
mkdir -p "$CACHE/sdk" "$CACHE/dl" "$CACHE/apt"
|
|
chmod -R a+rwX "$CACHE/dl" "$CACHE/apt" 2>/dev/null || true
|
|
|
|
# feeds/ git checkouts (actions/cache key: feeds-apk-<release>) — symlinked
|
|
# over the SDK's feeds dir inside the container (ci/sdk-build-apk.sh) so
|
|
# `scripts/feeds update -a` fetches deltas instead of re-cloning the
|
|
# ImmortalWrt feeds every run. Top-level chmod only: contents are created and
|
|
# owned by the container's uid-1000 build user (restore preserves ownership).
|
|
FEEDS_CACHE="$CACHE/feeds/apk"
|
|
mkdir -p "$FEEDS_CACHE"
|
|
chmod a+rwX "$CACHE" "$CACHE/feeds" "$FEEDS_CACHE" 2>/dev/null || true
|
|
|
|
# Fetch the SDK tarball ON THE RUNNER (restored cache -> own Gitea release-asset
|
|
# mirror -> upstream with stall-kill + retries) instead of the old bare
|
|
# `wget` inside the container, which hung whole runs when
|
|
# downloads.immortalwrt.org stalled mid-transfer.
|
|
SDK_TAR="$CACHE/sdk/$(basename "$SDK_URL")"
|
|
sh "$REPO/ci/fetch-sdk.sh" "$SDK_URL" "$SDK_TAR"
|
|
|
|
# --- 1) SDK build + index + sign inside a debian container -------------------
|
|
# `--volumes-from $(hostname)` shares THIS job container's workspace volume into
|
|
# the nested container (see ci/build-feed.sh for why a bare -v does not work on
|
|
# the act_runner DinD setup).
|
|
echo "[apk-feed] SDK build arch=$ARCH (ImmortalWrt 25.12 apk-SDK)"
|
|
docker pull -q debian:bookworm
|
|
docker run --rm --volumes-from "$(hostname)" \
|
|
-e ARCH="$ARCH" -e REPO="$REPO" -e OUT="$OUT" -e SDK_URL="$SDK_URL" \
|
|
-e SDK_TAR="$SDK_TAR" -e DL_DIR="$CACHE/dl" -e APT_CACHE="$CACHE/apt" \
|
|
-e FEEDS_CACHE="$FEEDS_CACHE" -e KEY_APK="${KEY_APK:-}" \
|
|
debian:bookworm bash "$REPO/ci/sdk-build-apk.sh"
|
|
|
|
# --- 2) sanity: the per-arch apk repo dir must be complete -------------------
|
|
[ -s "$OUT/packages.adb" ] || { echo "[apk-feed] ERROR: $OUT/packages.adb missing/empty" >&2; exit 4; }
|
|
apks=$(find "$OUT" -maxdepth 1 -name '*.apk' | wc -l)
|
|
[ "$apks" -ge 4 ] || { echo "[apk-feed] ERROR: expected >=4 .apk in $OUT, found $apks" >&2; exit 5; }
|
|
if [ -n "${KEY_APK:-}" ] && [ ! -s "$OUT/shater-apk.pem" ]; then
|
|
echo "[apk-feed] ERROR: signed feed but shater-apk.pem missing from $OUT" >&2; exit 6
|
|
fi
|
|
|
|
echo "[apk-feed] done arch=$ARCH -> $OUT"
|
|
ls -l "$OUT"
|