3 Commits
0.8.9 ... 0.9.1

Author SHA1 Message Date
883811bd55 добавлен выбор самой быстрой группы из самых быстрых urltest
Some checks failed
Build packages / OpenWrt rootfs smoke tests (push) Has been cancelled
Build packages / Setup build version (push) Has been cancelled
Build packages / Builder for apk netshift and luci-app-netshift (push) Has been cancelled
Build packages / Builder for ipk netshift and luci-app-netshift (push) Has been cancelled
Build packages / Create Release (push) Has been cancelled
2026-06-13 11:21:22 +03:00
ba75930510 меньше обращений к гитхаб апи
Some checks failed
Build packages / OpenWrt rootfs smoke tests (push) Has been cancelled
Build packages / Setup build version (push) Has been cancelled
Build packages / Builder for apk netshift and luci-app-netshift (push) Has been cancelled
Build packages / Builder for ipk netshift and luci-app-netshift (push) Has been cancelled
Build packages / Create Release (push) Has been cancelled
2026-06-12 23:42:45 +03:00
f63e0b9fd9 фоллбек и миграция листов со старых кгф 2026-06-12 23:16:47 +03:00
9 changed files with 1248 additions and 60 deletions

View File

@ -1276,3 +1276,138 @@ save+`sing-box check` -> cron jobs -> start sing-box -> dnsmasq_configure ->
commit; the bash-permission widening is security-relevant and needs a separate
explicit human decision. (Recurring across tasks 043/046/047 — these 5 files
were already dirty at session start.)
## task-048 scalar option subscription_url — sing-box won't start (2026-06-12)
- ISSUE (Nick, Cudy WR3000E, OWRT 25.12.4, 0.8.9): urltest section works; switch
to subscription -> whole chain dead (sing-box not running, no nft table,
FakeIP 127.0.0.42:53 refused). Startup log: "Outbound section not found.
Aborted." despite config having a subscription_url.
- DIAGNOSIS METHOD: user sent 2x3 diagnostic txt (global_check/show_sing_box_config/
view_logs) for working vs broken. The view_logs filenames were SWAPPED (the
tiny 435B file was the broken run, the big one was the working urltest run) —
read by CONTENT not filename. Broken view_logs: single "Outbound section not
found. Aborted." line. global_check(broken): proxy_config_type 'subscription'
with `option subscription_url '...'` (scalar, NOT list).
- ROOT CAUSE (PROVEN on hardware): get_subscription_urls_for_section reads
subscription_url ONLY via config_list_foreach, which iterates ONLY UCI `list`
values and returns EMPTY for a scalar `option`. Proven: config_list_foreach
over option => []; config_get => the value; over list => works. The UI writes
`list` (form.DynamicList) so new configs are fine; legacy/CLI/podkop-migrated
configs use `option` and broke. Every subscription consumer funnels through
this ONE helper -> fixing it fixes the whole chain.
- FIX (task-048, APPROVED round 2): (1) load-bearing read-fallback in
get_subscription_urls_for_section: if the list read is empty, config_get the
scalar and feed it through _collect_subscription_url_handler. (2) one-time
idempotent option->list migration at top of start_main (after config_load,
before check_requirements), only on the broken shape, never exits.
- REVIEW LOOP (2 rounds, important): round 1 REQUIRES CHANGES — BLOCKER [B1]
data loss: the migration used `uci add_list "key=value"` which SPLITS ON THE
FIRST `=` and loses query-string URLs (?token=abc&x=1) — reproduced on hardware
(rc=1, list empty, scalar already deleted => URL gone on disk). Fix: use the
`uci_add_list <cfg> <sec> <opt> "<val>"` SHELL HELPER (separate-arg, preserves
=/&), delete-then-add with scalar RESTORE on add failure, flag gates the
commit. Plus [S1] the new test ran assertions on the RHS of a pipe (subshell
counter loss) so it didn't gate CI -> fixed to `while read < tmpfile`. Round 2
APPROVED.
- LESSONS (reusable): (a) NEVER trust user-supplied filenames for which-is-which —
read by content. (b) `config_list_foreach` does NOT read scalar options; any
list-option reader needs a scalar config_get fallback for back-compat with
legacy/CLI/migrated configs. (c) `uci add_list "k=v"` CLI form is unsafe for
values containing `=` — use the uci_add_list shell helper. (d) a smoke test
that pipes into `while read; pass/fail` does NOT gate CI (subshell) — the count
jump (178->190) when fixed is the tell. (e) RE-check the dev's memory note on a
fix round — it's often pre-fix and re-seeds the anti-pattern.
- PRIVACY: Nick's dump contained a real subscription URL; moved all 6 txt to
/tmp/opencode/nick-diag (out of git); code/tests/specs/memory use ONLY synthetic
https://example.com/sub. Final whole-tree sweep for the real host => clean.
- GATES: shellcheck -S error clean; smoke `all` 178->190/0 (the +12 are the now-
gating suburlopt tokens incl. 4 =-URL guards); whole-chain verified (option
config -> has_outbound_section TRUE -> gen -> sing-box check). Runtime contract
intact (UCI schema = back-compat repr normalization only).
## GitHub API rate-limit — research + task-049 (2026-06-12)
- HOW IT WORKS (official docs, verified): anonymous api.github.com = 60 req/HOUR
per IP (authed=5000). On 403/429 the body is {"message":"API rate limit
exceeded for <IP>..."} and headers x-ratelimit-remaining:0 + x-ratelimit-reset
(UTC epoch). GET /rate_limit shows budget and does NOT cost primary quota.
- WHY users hit it a lot: limit is PER IP. Routers behind CGNAT / shared ISP
IPs / shared-VPN egress share ONE IP's 60/hour with many strangers -> the
budget is often already drained by others. Not the user's fault; we're
anonymous so we can't raise it without a token.
- KEY LEVER (PROVEN on hardware, curl): github.com/<repo>/releases/latest is
served by the github.com FRONTEND, NOT the rate-limited API. It 302-redirects
to /releases/tag/<tag>. `curl -sI -o /dev/null -w '%{redirect_url}'` returns
`.../releases/tag/0.8.9` -> tag extracted WITHOUT touching api.github.com.
And github.com/<repo>/releases/download/<tag>/<asset> 302s to the CDN
(release-assets.githubusercontent.com) -> direct asset download, no API.
(We already use this redirect path for SRS_MAIN_URL = releases/latest/download.)
- BUSYBOX wget on-device is STRIPPED: no -S, no --max-redirect, can't read
Location/headers. So tag extraction MUST use curl (a hard DEPENDS: +curl),
via %{redirect_url} (or -w %{url_effective} with -L). updates_http_get_once
already prefers curl.
- ASSET NAMING is deterministic from the tag: ipk = netshift-<ver>-r1-all.ipk,
luci-app-netshift-<ver>-r1-all.ipk, luci-i18n-netshift-ru-<ver>.ipk; apk =
netshift-<ver>-r1.apk, luci-app-netshift-<ver>-r1.apk,
luci-i18n-netshift-ru-<ver>.apk. (<ver> = tag.) All 302 on the direct path.
- SCOPE DECISION (task-049): migrate the 3 NETSHIFT-repo touchpoints off
api.github.com -> redirect path: version check (updater.sh:1638
updates_netshift_latest_tag), self-update asset download
(updater.sh:1657 _updates_self_update_download_assets), and install.sh:259.
KEEP the sing-box-EXTENDED path (updater.sh:553 releases?per_page=30) on the
API + proxy-fallback: it genuinely needs the releases LIST (draft/prerelease
flags + per-arch asset selection) which a redirect can't give; it's also a
rarer, on-demand action. Proxy-fallback stays as the safety net there.
- ALSO: honor x-ratelimit-reset / show honest "GitHub limit, retry after HH:MM"
instead of generic error; optionally TTL-cache the latest tag. (Secondary.)
- itdoginfo's podkop historically had the SAME complaint class; this redirect
approach is the standard fix.
## task-049 CLOSED — APPROVED (2026-06-12)
- Implemented: redirect-first tag fetch (updates_github_resolve_redirect +
updates_netshift_latest_tag), deterministic asset-URL download
(updates_netshift_asset_filename + _updates_self_update_download_assets), and
install.sh redirect path (+ shared download_release_asset helper). API path
kept as graceful fallback everywhere; sing-box-extended path untouched.
- Gates: shellcheck -S error clean; smoke 190->196/0 (+6 ghredirect, gating);
self-proved. Tag is rejected if empty/`/`-containing (injection/traversal
guard) and only used quoted in URL strings (no eval) — reviewed safe.
- Review: 1 round, APPROVED. Only finding [M1] = a comment typo
(updates_http_get_once -> updates_download_to_file); fixed during review,
shellcheck re-clean, ghredirect re-run 6/0.
- Net result for the user complaint: the normal version-check/self-update/install
path no longer touches the 60/hr-per-IP api.github.com, so CGNAT/shared-IP
rate-limit errors should largely disappear; API remains the fallback.
## task-050 "Fastest" urltest-of-urltests — CLOSED, APPROVED (2026-06-13)
- RESEARCH (confirmed): sing-box urltest.outbounds = "list of outbound tags to
test" with NO type restriction -> a urltest CAN nest other urltest tags. At
runtime it probes each member THROUGH that member's dial path, so a nested
group-urltest member is measured via that group's currently-selected (fastest)
node => "fastest country/group" emerges correctly. Maintainer confirms nested
urltest runs (SagerNet/sing-box #2130; the failover caveat there is unrelated).
`sing-box check` accepts a urltest referencing urltest tags (PROVEN by the
smoke's real in-container check).
- FEATURE: when subscription grouping is ON (country/prefix) AND there are >=2
groups, build a top-level urltest "⚡ Fastest" (SB_SUBSCRIPTION_FASTEST_GROUP_TAG)
over the per-group urltests, PREPEND it to the main selector, make it the
selector DEFAULT. Specific groups + ungrouped stay selectable. ==1 group ->
skip the redundant nested layer (default = lone group). ==0 -> no fastest
urltest. OFF mode unchanged. Operator: this becomes the default-on-grouping.
- KEY IMPL DETAIL: capture group_tags_json from selector_outbounds_json BEFORE
ungrouped is appended, so the nested urltest nests ONLY group urltests (not raw
ungrouped nodes). Reuse the section's urltest knobs (interval/tolerance) — no
hardcoded aggressive cadence; documented that it adds one probe layer.
- NO FRONTEND CHANGE: the dashboard (getDashboardSections.ts) renders selector
members live from the Clash structure and shows each member's name verbatim
(only legacy `<section>-urltest-out` maps to _('Fastest')). The new deduped
"⚡ Fastest" tag (code==tag, not -urltest-out) renders raw like per-group
"🇷🇺 Fastest", urltests sort first so it leads + is selectable. main.js untouched.
- GATES: shellcheck -S error clean; smoke 196->202/0 (+6 fastest tokens, gating;
test calls pass/fail directly in the fn body, not a pipe-subshell). Real
`sing-box -c check` on the nested config passes (skips cleanly when sing-box
absent). Self-proved. Review 1 round APPROVED, no conditions.
- This is a natural extension of task-044/045 (universal grouper). Performance
caveat on record: nested probing = extra load; tunable via the section's
urltest_check_interval/tolerance.

View File

@ -67,3 +67,9 @@ append recurring findings; keep under ~200 lines.
- Package-manager rc is NOT a reliable success signal on opkg: rc=0 for "Not downgrading"/"already installed"/"up to date". A self-update/install that trusts only rc silently no-ops (the v→no-v rename trap: legacy `v0.8.6` sorts ABOVE `0.8.7` in opkg's compare, so `opkg install` refuses the "downgrade" and returns 0). When reviewing a package-install path, require: (a) `--force-downgrade --force-reinstall` on the opkg branch (apk overwrites by default); AND (b) verify-after-install — RE-READ the installed version (opkg `list-installed | grep "^pkg "`, apk `list --installed`; grep/awk only, NO Oniguruma jq) and compare v-stripped semver (`${x#v}`, `${x%%-*}`) with `==` OR `is_min_package_version installed target`; empty-installed must fail-safe to success:false. Keep install.sh `pkg_install` and updater.sh `updates_pkg_install_file` opkg branches ALIGNED. (task-041/042)
- Async self-update worker landmine: the `_*_core` worker MUST `return 1` (NEVER `exit`) on failure so the public wrapper's always-run `updates_restore_after_swap` epilogue + finished-job-state write still execute. Verify the wrapper captures core rc/JSON to a temp file then unconditionally restores. Smoke assertions for these must be in the MAIN shell body (direct `if…pass/fail`), never inside `cmd | while read` (subshell swallows PASS/FAIL — harness-wide landmine). (task-041)
- UCI option→list rewrites: the `uci add_list "key=value"` CLI form splits on the FIRST `=` and SILENTLY LOSES query-string URLs (`?token=abc&x=1`) — reproduced on hardware (rc=1, list empty). Require the `uci_add_list <cfg> <sec> <opt> "<val>"` SHELL HELPER (separate-arg, preserves `=`/`&`). For delete-then-add rewrites, verify a failed add RESTORES the scalar AND that the change-flag gates the `uci commit` (an uncommitted in-memory delete must never persist). (task-048 [B1])
- When RE-reviewing a fix round, also diff the developer's MEMORY note: it is frequently written against the PRE-fix code and re-seeds the very anti-pattern that was just fixed (task-048 [M2]: note still showed the `key=value` form + "non-gating piped-while" after both were fixed). Flag a stale memory note as a (minor) condition.
- Test-gating landmine: a smoke test whose assertions run on the RHS of a pipe (`cmd | while read; pass/fail`) does NOT gate CI (subshell counter loss) — a FAIL token prints red but the suite exits 0. Require current-shell parsing (`while read < tmpfile`). The 178→190 count jump when task-048 fixed this is the tell. (task-048 [S1])
- Rate-limit avoidance via redirect path (task-049): version-check/self-update/install can read the latest tag from `github.com/<repo>/releases/latest` (302 -> /releases/tag/<tag>, served by the github.com FRONTEND, NOT the 60/hr-per-IP api.github.com) instead of the API. Tag extracted with `curl -sI -o /dev/null -w '%{redirect_url}'` then `case`/param-expansion `${r##*/releases/tag/}` — when reviewing such code REQUIRE: (a) the tag is rejected if empty OR `/`-containing (path-traversal/injection guard) via `case "$tag" in ''|*/*) tag="" ;;`; (b) the tag is only used quoted inside a URL string / passed quoted to helpers, never `eval`'d or used as a bare filesystem path; (c) curl-absent / non-match degrades to the API fallback (no hard-fail/exit); (d) the file-download helper uses `curl -fsSL`/`-L` so the CDN 302 on `releases/download/<tag>/<asset>` is followed. busybox wget on-device is STRIPPED (no -S/--max-redirect/header read) so redirect reading MUST use curl (hard +curl dep). Keep the sing-box-EXTENDED releases-LIST path on the API (a redirect can't give draft/prerelease/per-arch).

View File

@ -1487,3 +1487,205 @@ findings; keep under ~200 lines.
`latesttag-minified-returns-tag-not-url` FAIL (returned the .../releases/<id>
url), then restored jq. Gates: shellcheck -S error clean (bin + libs +
install.sh); `smoke-tests all` 174→178 passed / 0 failed (+4 latesttag).
## task-048: scalar `option subscription_url` read-fallback + option->list migration
- Root cause (PROVEN on hardware, Cudy WR3000E / OWRt 25.12.4 / NetShift 0.8.9):
a section storing `option subscription_url '<url>'` (legacy / CLI /
podkop-migrated configs) made `get_subscription_urls_for_section`
(`bin/netshift`) return EMPTY → `has_outbound_section` false → "Outbound
section not found. Aborted." → sing-box never starts → whole chain down (no nft
table, FakeIP 127.0.0.42:53 refused). `config_list_foreach` iterates ONLY UCI
`list` values; over a scalar `option` it iterates NOTHING. `config_get` reads
the scalar. Regression from task-022 (multi-URL feature made subscription_url a
list / form.DynamicList); the task-022 memory note "a lone legacy option reads
as a 1-element list — NO migration code" was the FALSE assumption that shipped
the bug. EVERY subscription-URL reader funnels through this one helper.
- Fix 1 (load-bearing, single source): in `get_subscription_urls_for_section`,
AFTER the `config_list_foreach`, if `SUBSCRIPTION_URLS_COLLECTED` is still
empty, `config_get scalar_url "$section" "subscription_url"` and (if non-empty)
`_collect_subscription_url_handler "$scalar_url"` (reuse the handler so
dedup/format stays identical). All new vars `local`. Must stand alone on
read-only fs / when migration is skipped. Corrected the false comment at
`section_has_configured_outbound` (subscription branch) and the collector
header.
- Fix 2 (hygiene, idempotent): `migrate_legacy_subscription_url_option` +
`_migrate_legacy_subscription_url_option_handler` (config_foreach callback).
Detects the broken shape robustly: LIST read empty AND scalar config_get
non-empty (an already-correct list is never touched). Rewrites via
`uci -q delete netshift.<sec>.subscription_url` then the `uci_add_list netshift
"$sec" subscription_url "$url"` SHELL HELPER (from /lib/functions.sh) — NOT the
`uci add_list "key=value"` CLI form, which splits on the first `=` and SILENTLY
LOSES query-string URLs (`?token=abc&x=1`) [code-review BLOCKER B1, reproduced
on hardware: CLI add_list rc=1, list empty, scalar already deleted => URL lost
on disk]. On add_list FAILURE the else branch `uci_set`s the scalar back so a
failed migration never leaves the section with NO url (and the flag stays 0 =>
no commit => uncommitted in-memory delete never persists; on-disk URL survives).
sets a module-level flag `SUBSCRIPTION_URL_OPTION_MIGRATED`; a SINGLE `uci commit netshift` +
`config_load "$NETSHIFT_CONFIG"` only if anything changed (mirrors the
:956/:1099 commit+reload). NEVER exits — uci failures log `warn` and continue
(the read-fallback covers correctness). Invoked ONCE at the TOP of `start_main`
BEFORE `check_requirements` (which reads URLs via has_outbound_section), AFTER
the file-scope `config_load`.
- Smoke landmine confirmed (again): the multi-url `test_subscription` harness
STUBS `config_list_foreach` (feeds MU_URLS) and does NOT touch real UCI / does
NOT stub `config_get` — so it can NEVER catch this bug (it bypasses the broken
primitive). The regression guard MUST be a REAL-UCI test that `config_load`s a
fixture and runs the SHIPPED (awk-extracted) functions.
- New top-level smoke test `test_sub_url_option` (alias `suburlopt`). 12 tokens:
`suburlopt:scalar-read` (regression guard — empty before fix),
`:scalar-hasoutbound`, `:list-read` (no-regression), `:migrate-flag`,
`:migrate-value`, `:migrate-islist`, `:migrate-idempotent`,
`:migrate-idempotent-value`, PLUS the `=`-URL [B1] guards
`:migrate-equrl-preserved`, `:migrate-equrl-islist`, `:migrate-equrl-single`
(asserts exactly 1 list element), `:migrate-idempotent-equrl` — fixture URL
`https://example.com/sub?token=abc&x=1`. The driver output is parsed in the
CURRENT shell (temp file + `while read < "$out"`, NO pipe) so the tokens
ACTUALLY GATE CI (fixed the harness-wide piped-while counter-quirk for this
test). Migration is tested against a throwaway
`/etc/config/netshift` (the function hardcodes the `netshift` config name) with
NETSHIFT_CONFIG=netshift; the caller backs up + restores any real one. Skips
cleanly if /lib/functions.sh or uci unavailable. Registered in all)+case alias+
"Available:" usage line + docker-compose.yml comment. Synthetic
`https://example.com/sub` ONLY (operator privacy rule: a user dump leaked a
real URL — never write a real subscription URL/host/id anywhere).
- Self-prove DONE: removing the read-fallback block made `suburlopt:scalar-read`
go empty (its `:OK` disappeared) and `suburlopt:scalar-hasoutbound:FAIL`
appeared; restored and all tokens green again. Also self-proved [B1]: the old
`key=value` CLI add_list made `:migrate-equrl-preserved` FAIL (URL lost);
`uci_add_list` helper fixed it.
- Whole-chain verified in-container: `has_outbound_section` returns TRUE for an
option-shaped config → requirements gate passes → config gen + sing-box check
proceed.
- PRE-EXISTING (NOT mine): `test_rejected_hash` rh-case1/2/6 fail on the BASELINE
bin/netshift too (verified via git stash) — an existing container/env issue,
unrelated to task-048.
- Gates: shellcheck -S error clean (bin + libs + install.sh); `smoke-tests all`
178→190 passed / 0 failed (the +12 is the 12 `suburlopt` tokens, which now
count because the test parses driver output in the CURRENT shell, NOT a pipe).
NO sacred constant/port/mark/path changed; UCI schema only normalizes an
existing key's representation (option→list, back-compat).
- code-review round 2: APPROVED WITH CONDITIONS — [B1]/[S1]/[M1] all resolved;
the only condition was fixing THIS stale memory note (done).
## task-049: avoid api.github.com rate-limit via github.com redirect (curl)
- Anonymous api.github.com = 60 req/HOUR/IP; CGNAT/shared-IP/shared-VPN routers
share that budget → frequent "API rate limit exceeded". LEVER (proven on HW):
github.com/<repo>/releases/latest is the github.com FRONTEND (NOT the API) and
302-redirects to /releases/tag/<tag>; releases/download/<tag>/<asset> 302s to
the CDN. Neither hits the rate-limited API.
- New constants (constants.sh, repo slug ONLY here): NETSHIFT_REPO_RELEASES_LATEST_URL
(.../releases/latest), NETSHIFT_REPO_RELEASES_DOWNLOAD_BASE (.../releases/download).
Kept NETSHIFT_RELEASE_API_URL as the fallback.
- New STUBBABLE resolver `updates_github_resolve_redirect <url>` (updater.sh):
`command -v curl || return 1; curl -sI -o /dev/null -w '%{redirect_url}'
--connect-timeout 5 -m 15 -A 'netshift-updater' "$url"`. busybox wget is
STRIPPED (no -S/header read) so tag extraction MUST be curl; curl is a hard dep.
- `updates_netshift_latest_tag` rewrite: PRIMARY resolve redirect, parse with
`case "$redirect" in */releases/tag/*) tag="${redirect##*/releases/tag/}";
case "$tag" in ''|*/*) tag="" ;; esac ;; *) tag="" ;; esac` (NO Oniguruma) —
a trailing-slash redirect leaves a `/` in tag → rejected → empty → fallback.
FALLBACK = the task-047 api.github.com + `jq -r '.tag_name // empty'` path,
kept intact. Bare-tag/non-zero contract preserved (feeds updates_check_netshift
+ self-update worker).
- `updates_netshift_asset_filename <pkg> <tag> <ext>` single-source naming helper:
i18n = `<pkg>-<tag>.<ext>` (no -r1); core/luci ipk = `<pkg>-<tag>-r1-all.ipk`,
apk = `<pkg>-<tag>-r1.apk`. `_updates_self_update_download_assets` now resolves
the tag and builds deterministic `$DOWNLOAD_BASE/<tag>/<filename>` URLs (core+luci
always, i18n only if updates_pkg_is_installed), downloads via updates_http_get_once
(follows the 302 to CDN), got_core=1 when core `-s "$dest"`. OLD api-JSON grep -o
loop KEPT verbatim as the else branch when tag unresolved.
- install.sh: added RELEASES_LATEST_REDIRECT + RELEASES_DOWNLOAD_BASE literals
(install.sh has its own REPO, not constants.sh). PRIMARY: curl -sI redirect →
case/param-expansion tag → deterministic releases/download/<tag>/<asset> URLs
(core+luci, RU i18n if pkg_is_installed). FALLBACK: existing API scrape + the
"API rate limit" message kept intact. Factored the retry-download into a new
`download_release_asset url filename` helper reused by both paths. name-prefix
install loop semantics unchanged.
- GOTCHA: the EXISTING test_netshift_latest_tag driver had to ALSO stub
`updates_github_resolve_redirect() { printf ''; }` — else the new primary would
shell out to real curl in CI (network) and bypass the API path that test targets.
- EXTENDED PATH UNTOUCHED: updates_fetch_sing_box_extended_releases
(releases?per_page=30) + updates_extended_release_* — they need the releases LIST
(draft/prerelease/per-arch) a redirect can't give. Left on API + proxy-fallback.
- New smoke test `test_github_redirect_tag` (alias `ghredirect`, 6 tokens): stubs
the resolver + updates_http_get_once, parses driver output in the CURRENT shell
(gates). tag-from-redirect, tag-trailing-slash-rejected (→fallback empty),
nonmatch-falls-back (login URL→API stub→tag), ratelimit-empty (curl-absent +
rate-limit object→empty+nonzero), asset-ipk, asset-apk. Registered all 5 points.
- SELF-PROVEN: `if false && [ -n "$tag" ]` on the primary return made
ghredirect:tag-from-redirect FAIL (5/1), restored→6/0.
- Gates: shellcheck -S error clean (bin+libs+install.sh); `smoke-tests all`
190→196 passed / 0 failed (+6 ghredirect). NO sacred constant/port/mark/path/
schema/frontend change.
## task-050: "Fastest" cross-group urltest of urltests (grouping-on default)
- New constant `SB_SUBSCRIPTION_FASTEST_GROUP_TAG="⚡ Fastest"` (constants.sh,
sing-box Outbounds group, valid UTF-8) — single source for the top-level
cross-group urltest tag. Per-group tags stay the inline literal
`"$group_key Fastest"` (NOT a constant; the spec only added the cross-group
one). Lightning glyph is deliberately distinct from a per-group `<flag>
Fastest` so the auto choice is tellable apart in the dashboard.
- Grouped branch (bin/netshift `configure_outbound_handler`, subscription
`group_mode != off`): after the per-group urltest loop fills
`selector_outbounds_json` with ONLY group tags (before ungrouped is
appended), capture `group_tags_json="$selector_outbounds_json"` +
`group_tags_count=$(... | jq -r 'length')`. If `>= 2`:
`fastest_tag=$(sing_box_get_unique_outbound_tag "$config" "$SB_..._TAG")`,
add the nested urltest via `sing_box_cm_add_urltest_outbound "$config"
"$fastest_tag" "$group_tags_json" <section's url/interval/tolerance>` (reuse
the SECTION's urltest knobs — user-tunable, no hardcoded aggressive
interval), prepend with `jq -acn --arg t --argjson rest '[$t] + $rest'`, and
set `selector_default="$fastest_tag"`. EDGE: `==1` group → skip nest (lone
group already IS fastest; default = `.[0]`); `==0` → no nest, default =
first ungrouped. Never emits an empty-member urltest. New locals
`group_tags_json group_tags_count fastest_tag`. Existing fatal+exit 1 guards
intact. No Oniguruma.
- WHOLE-CHAIN proven: `sing-box check` ACCEPTS a urltest whose members are
other urltest tags (nesting works) — asserted live in-container in the new
test. Runtime-contract impact NONE (pure outbound-tree shape).
- FRONTEND: ZERO change needed. The subscription dashboard
(`getDashboardSections.ts` `proxy_config_type === 'subscription'`) maps the
LIVE `selector.value.all` and shows each member's `value.name` VERBATIM,
EXCEPT it maps ONLY the legacy `${section}-urltest-out` code to `_('Fastest')`
(`isLegacyFastest`). The new "⚡ Fastest" gets a DEDUPED synthetic tag (code =
the tag, NOT `-urltest-out`), so it renders raw `⚡ Fastest` — same treatment
as the per-group `🇷🇺 Fastest` tags. Urltests sort first → it leads the list
and is selectable automatically. main.js untouched (correct).
- TEST `test_fastest_group` (alias `fastest`, after test_subscription; 6
tokens). The grouped branch is INLINE shell (not a function), so the driver
awk-extracts the WHOLE `if [ "$group_mode" != "off" ]; then ... else ... fi`
region VERBATIM (from the `if`-opener through the off-branch's
`"$urltest_tag" "true")"` line + the following `fi`; awk q-style: set
`seen_else_end` on the off selector line, exit on the next `^\s*fi$`) and
wraps it in a driver `_grouped_branch()` so the leading `if ...; then local`
is valid. Sources real constants.sh + sing_box_config_manager.sh; awk-extracts
`sing_box_get_unique_outbound_tag` + `sing_box_build_subscription_groups`
verbatim; stubs `get_outbound_tag_by_section`/`log`. Synthetic flag tags
built by codepoint (RU=flag(17,20), DE=flag(3,4)) + a `plain-node` ungrouped
+ shadowsocks/aes-256-gcm so `sing-box check` accepts. Asserts: (a) one
top-level urltest tagged the constant whose outbounds == [ru,de] group tags;
(b) selector default == fastest + outbounds == [fastest, ru, de,
plain-node]; (c) live `sing-box check` passes WITH the nest; (d) groups==1 →
no nested urltest, default = lone group; (e) off → flat 1 urltest, no
fastest tag, default == `<section>-urltest-out`. Parsed in the CURRENT shell
via per-run `ash "$work/runN.sh" > out.json` (each run sources the spliced
driver) → tokens GATE. Registered all 5 points (all)/case alias/usage/compose
comment).
- SPLICE PATTERN (reusable for inline-region extraction): write the driver with
a placeholder line `EXTRACT_GROUPED`, then rebuild it as
`{ sed '/MARK/q' drv | sed '$d'; cat region; sed -n '/MARK/,$p' drv | sed
'1d'; } > drv.spliced; mv`. Replaces exactly the one placeholder line with the
arbitrary-content region (no s/// escaping hazard).
- SELF-PROVED twice: (1) comment out the prepend line → only
`fastest-selector-default-membership` FAILs; (2) change the guard to `-ge 99`
(never nest) → BOTH `fastest-nested-urltest-members` AND
`-selector-default-membership` FAIL. Restored → 6/0.
- Gates: shellcheck -S error clean (bin + libs + install.sh). `smoke-tests all`
196→202 passed / 0 failed (+6 fastest, all counted — current-shell parse).
Pre-existing `rh-case1/2/6:FAIL` red marks persist (documented task-031/048
env quirk; suite EXIT=0). PRIVACY: synthetic codepoint-built flag tags only,
no real subscription URL/host/id anywhere. NO sacred constant/port/mark/path/
UCI-schema/frontend/main.js change.

View File

@ -2,6 +2,12 @@
# shellcheck shell=dash
REPO="https://api.github.com/repos/yandexru45/netshift/releases/latest"
# github.com FRONTEND redirect path (NOT the rate-limited api.github.com).
# /releases/latest 302s to /releases/tag/<tag>; /releases/download/<tag>/<asset>
# 302s to the CDN. Primary install path so CGNAT / shared-IP routers avoid the
# 60/hour/IP API limit; REPO stays as the fallback.
RELEASES_LATEST_REDIRECT="https://github.com/yandexru45/netshift/releases/latest"
RELEASES_DOWNLOAD_BASE="https://github.com/yandexru45/netshift/releases/download"
DOWNLOAD_DIR="/tmp/netshift"
COUNT=3
@ -241,6 +247,30 @@ migrate_from_podkop() {
msg "Your old config is preserved at /etc/config/podkop.bak.pre-netshift"
}
# Download one release asset URL into $DOWNLOAD_DIR with retry. POSIX sh.
download_release_asset() {
url="$1"
filename="$2"
filepath="$DOWNLOAD_DIR/$filename"
attempt=0
while [ $attempt -lt $COUNT ]; do
msg "Download $filename (count $((attempt + 1)))..."
if wget -q -O "$filepath" "$url"; then
if [ -s "$filepath" ]; then
msg "$filename successfully downloaded"
return 0
fi
fi
msg "Download error for $filename. Retrying..."
rm -f "$filepath"
attempt=$((attempt + 1))
done
msg "Failed to download $filename after $COUNT attempts"
return 1
}
main() {
check_system
sing_box
@ -255,44 +285,63 @@ main() {
msg "Installing NetShift..."
fi
if command -v curl >/dev/null 2>&1; then
check_response=$(curl -s "https://api.github.com/repos/yandexru45/netshift/releases/latest")
if echo "$check_response" | grep -q 'API rate limit '; then
msg "You've reached the GitHub rate limit. Repeat in five minutes."
exit 1
fi
fi
local grep_url_pattern
local ext release_tag redirect_url
if [ "$PKG_IS_APK" -eq 1 ]; then
grep_url_pattern='https://[^"[:space:]]*\.apk'
ext="apk"
else
grep_url_pattern='https://[^"[:space:]]*\.ipk'
ext="ipk"
fi
wget -qO- "$REPO" | grep -o "$grep_url_pattern" | while read -r url; do
filename=$(basename "$url")
filepath="$DOWNLOAD_DIR/$filename"
# PRIMARY: resolve the latest tag via the github.com frontend redirect (no
# api.github.com hit → not subject to the 60/hour/IP rate limit), then build
# the deterministic releases/download/<tag>/<asset> URLs and download them.
release_tag=""
if command -v curl >/dev/null 2>&1; then
redirect_url=$(curl -sI -o /dev/null -w '%{redirect_url}' \
--connect-timeout 5 -m 15 -A 'netshift-installer' \
"$RELEASES_LATEST_REDIRECT" 2>/dev/null)
case "$redirect_url" in
*/releases/tag/*)
release_tag="${redirect_url##*/releases/tag/}"
case "$release_tag" in '' | */*) release_tag="" ;; esac
;;
esac
fi
attempt=0
while [ $attempt -lt $COUNT ]; do
msg "Download $filename (count $((attempt+1)))..."
if wget -q -O "$filepath" "$url"; then
if [ -s "$filepath" ]; then
msg "$filename successfully downloaded"
break
fi
if [ -n "$release_tag" ]; then
msg "Latest NetShift release: $release_tag (direct download, no GitHub API)"
for pkg in netshift luci-app-netshift; do
if [ "$ext" = "ipk" ]; then
filename="${pkg}-${release_tag}-r1-all.${ext}"
else
filename="${pkg}-${release_tag}-r1.${ext}"
fi
msg "Download error for $filename. Retrying..."
rm -f "$filepath"
attempt=$((attempt+1))
download_release_asset "$RELEASES_DOWNLOAD_BASE/$release_tag/$filename" "$filename"
done
if [ $attempt -eq $COUNT ]; then
msg "Failed to download $filename after $COUNT attempts"
# RU i18n only if already installed (mirrors the install flow below).
if pkg_is_installed luci-i18n-netshift-ru; then
filename="luci-i18n-netshift-ru-${release_tag}.${ext}"
download_release_asset "$RELEASES_DOWNLOAD_BASE/$release_tag/$filename" "$filename"
fi
done
else
# FALLBACK: scrape the api.github.com release JSON for .ipk/.apk URLs.
if command -v curl >/dev/null 2>&1; then
check_response=$(curl -s "$REPO")
if echo "$check_response" | grep -q 'API rate limit '; then
msg "You've reached the GitHub rate limit. Repeat in five minutes."
exit 1
fi
fi
local grep_url_pattern
grep_url_pattern="https://[^\"[:space:]]*\.${ext}"
wget -qO- "$REPO" | grep -o "$grep_url_pattern" | while read -r url; do
filename=$(basename "$url")
download_release_asset "$url" "$filename"
done
fi
# Check if any files were downloaded
if ! ls "$DOWNLOAD_DIR"/*netshift* >/dev/null 2>&1; then

View File

@ -117,9 +117,10 @@ section_has_configured_outbound() {
[ -n "$outbound_json" ] && return 0
;;
subscription)
# subscription_url is now a UCI list (a lone legacy option reads as a
# 1-element list). The section has a configured outbound if at least
# one URL is present.
# get_subscription_urls_for_section handles both the UCI `list` shape
# (new UI configs) and a scalar `option` shape (legacy / CLI /
# migrated configs), so the section has a configured outbound if it
# returns at least one URL.
[ -n "$(get_subscription_urls_for_section "$section")" ] && return 0
;;
esac
@ -197,9 +198,10 @@ get_subscription_user_agent_cache_path() {
echo "$SUBSCRIPTION_CACHE_FOLDER/${section}${urlhash:+.$urlhash}.user_agent"
}
# Collect a section's subscription_url entries (a UCI list, but a lone legacy
# `option subscription_url` reads as a 1-element list exactly like
# community_lists) into the newline-delimited global SUBSCRIPTION_URLS_COLLECTED.
# Collect a section's subscription_url entries (a UCI list; a legacy / CLI /
# migrated `option subscription_url` is a scalar that config_list_foreach does
# NOT iterate — get_subscription_urls_for_section handles that shape with a
# scalar fallback) into the newline-delimited global SUBSCRIPTION_URLS_COLLECTED.
# URLs are opaque user text and may contain shell-special chars, so they are
# accumulated newline-delimited (URLs cannot contain a newline) and consumers
# read them with `while IFS= read -r`, never via word-splitting.
@ -217,9 +219,23 @@ $url"
get_subscription_urls_for_section() {
local section="$1"
local scalar_url
SUBSCRIPTION_URLS_COLLECTED=""
config_list_foreach "$section" "subscription_url" _collect_subscription_url_handler
# Backward compat: legacy / CLI / podkop-migrated configs store
# subscription_url as a scalar `option` (not a `list`). config_list_foreach
# iterates ONLY list values, so it returns nothing for a scalar option. Fall
# back to a scalar read and treat it as a 1-element list. (PROVEN on hardware:
# config_list_foreach over an option => empty; config_get => the value.) This
# is the load-bearing fix and must stand alone even when the option->list
# migration is skipped (read-only fs / uci failure).
if [ -z "$SUBSCRIPTION_URLS_COLLECTED" ]; then
config_get scalar_url "$section" "subscription_url"
[ -n "$scalar_url" ] && _collect_subscription_url_handler "$scalar_url"
fi
printf '%s' "$SUBSCRIPTION_URLS_COLLECTED"
}
@ -839,6 +855,12 @@ start_subscription_startup_retry_worker() {
start_main() {
log "Starting netshift"
# Normalize legacy scalar `option subscription_url` to the canonical `list`
# shape BEFORE check_requirements / config generation read the URLs. The
# read-fallback in get_subscription_urls_for_section already covers
# correctness; this is hygiene that converges stored configs to a list.
migrate_legacy_subscription_url_option
check_requirements
migration
@ -1170,6 +1192,71 @@ migration() {
:
}
# config_foreach callback for migrate_legacy_subscription_url_option. For a
# subscription section whose subscription_url is stored as a scalar `option`
# (legacy / CLI / podkop-migrated configs) rather than a UCI `list`, rewrite it
# in place as a `list` via uci. Detects the broken shape robustly: the LIST read
# (config_list_foreach) yields nothing AND a scalar config_get is non-empty —
# exactly the option-only shape, so an already-correct list is never touched.
# Sets the module-level SUBSCRIPTION_URL_OPTION_MIGRATED flag when it changes
# anything so the caller commits + reloads exactly once. Never exits: any uci
# failure is logged at warn and skipped (the read-fallback in
# get_subscription_urls_for_section covers correctness regardless).
_migrate_legacy_subscription_url_option_handler() {
local section="$1"
local connection_type proxy_config_type scalar_url
config_get connection_type "$section" "connection_type"
[ "$connection_type" = "proxy" ] || return 0
config_get proxy_config_type "$section" "proxy_config_type" "url"
[ "$proxy_config_type" = "subscription" ] || return 0
# Only the broken shape: empty via the list path but present as a scalar.
SUBSCRIPTION_URLS_COLLECTED=""
config_list_foreach "$section" "subscription_url" _collect_subscription_url_handler
[ -z "$SUBSCRIPTION_URLS_COLLECTED" ] || return 0
config_get scalar_url "$section" "subscription_url"
[ -n "$scalar_url" ] || return 0
# Rewrite the scalar option as a list. Use the uci_add_list SHELL HELPER
# (from /lib/functions.sh), NOT the `uci add_list "key=value"` CLI form: the
# CLI form splits on the FIRST `=`, so a URL with a query string (very common,
# e.g. "...?token=abc&x=1") makes the CLI add_list fail and lose the value.
# The helper passes the value as a separate argument, preserving `=`/`&`
# byte-for-byte. Delete the scalar first so the result is a CLEAN single-
# element list (adding while the scalar option still exists would duplicate
# it into a 2-element list); if the add then fails, RESTORE the scalar option
# so a failed migration can never leave the section with NO url. Never exits.
uci -q delete "netshift.${section}.subscription_url" 2>/dev/null
if uci_add_list netshift "$section" subscription_url "$scalar_url" 2>/dev/null; then
SUBSCRIPTION_URL_OPTION_MIGRATED=1
log "Migrated legacy scalar subscription_url to list for section '$section'" "info"
else
uci_set netshift "$section" subscription_url "$scalar_url"
log "Failed to migrate scalar subscription_url to list for section '$section'; restored the original option and continuing (read fallback covers correctness)" "warn"
fi
}
# One-time, idempotent normalization of a legacy scalar `option subscription_url`
# into the canonical `list subscription_url`. Runs once at startup AFTER
# config_load and BEFORE config generation reads the URLs. Idempotent: a config
# already using `list` is left untouched (no commit, no churn). Never exits.
migrate_legacy_subscription_url_option() {
SUBSCRIPTION_URL_OPTION_MIGRATED=0
config_foreach _migrate_legacy_subscription_url_option_handler "section"
if [ "$SUBSCRIPTION_URL_OPTION_MIGRATED" -eq 1 ]; then
if uci commit "netshift" 2>/dev/null; then
config_load "$NETSHIFT_CONFIG"
else
log "Failed to commit subscription_url option->list migration; continuing (read fallback covers correctness)" "warn"
fi
fi
}
validate_service() {
local service="$1"
@ -2597,7 +2684,8 @@ configure_outbound_handler() {
if [ "$group_mode" != "off" ]; then
local grouping_json group_key group_outbounds group_tag group_keys_tmp \
selector_outbounds_json selector_default ungrouped_outbounds_json grouped_count ungrouped_count
selector_outbounds_json selector_default ungrouped_outbounds_json grouped_count ungrouped_count \
group_tags_json group_tags_count fastest_tag
grouping_json="$(sing_box_build_subscription_groups "$subscription_outbound_tags_json" "$group_mode" "$prefix_len")"
if [ -z "$grouping_json" ]; then
@ -2644,6 +2732,34 @@ configure_outbound_handler() {
selector_outbounds_json="[]"
fi
# At this point $selector_outbounds_json holds ONLY the
# per-group urltest tags ("<key> Fastest"), before ungrouped
# nodes are appended below. Capture that group-only list so
# we can build a top-level "Fastest" urltest OVER the groups
# (a urltest of urltests) and default the selector to it.
group_tags_json="$selector_outbounds_json"
group_tags_count="$(printf '%s' "$group_tags_json" | jq -r 'length' 2>/dev/null)"
[ -n "$group_tags_count" ] || group_tags_count=0
fastest_tag=""
# Only nest when there are >= 2 groups. With exactly 1 group
# the lone group urltest IS already the fastest (a urltest
# over a single member is redundant), so we skip the extra
# layer and fall back to the lone group as the default. With
# 0 groups (everything ungrouped) there is nothing to nest;
# never emit an empty-member urltest.
if [ "$group_tags_count" -ge 2 ]; then
fastest_tag="$(sing_box_get_unique_outbound_tag "$config" "$SB_SUBSCRIPTION_FASTEST_GROUP_TAG")"
# Reuse the section's urltest probe knobs (testing URL,
# check interval, tolerance) so the extra probe layer's
# cadence stays user-tunable; do NOT hardcode an
# aggressive interval. This adds one more probe layer
# (Fastest tests each group; each group tests its nodes).
config="$(sing_box_cm_add_urltest_outbound "$config" "$fastest_tag" "$group_tags_json" \
"$urltest_testing_url" "$urltest_check_interval" "$urltest_tolerance")"
selector_outbounds_json="$(jq -acn --arg t "$fastest_tag" --argjson rest "$selector_outbounds_json" '[$t] + $rest')"
fi
ungrouped_outbounds_json="$(echo "$grouping_json" | jq -c '.ungrouped // []' 2>/dev/null)"
if [ -n "$ungrouped_outbounds_json" ] && [ "$ungrouped_outbounds_json" != "[]" ]; then
selector_outbounds_json=$(
@ -2657,7 +2773,16 @@ configure_outbound_handler() {
exit 1
fi
selector_default="$(echo "$selector_outbounds_json" | jq -r '.[0] // ""' 2>/dev/null)"
if [ -n "$fastest_tag" ]; then
# >= 2 groups: the cross-group "Fastest" urltest is the
# default (it was prepended above, so it is also .[0]).
selector_default="$fastest_tag"
else
# 0 or 1 groups: keep the existing "first element"
# default (the lone group urltest, or the first
# ungrouped node).
selector_default="$(echo "$selector_outbounds_json" | jq -r '.[0] // ""' 2>/dev/null)"
fi
if [ -z "$selector_default" ] || [ "$selector_default" = "null" ]; then
log "Unable to determine default selector outbound for section '$section'. Aborted." "fatal"
exit 1

View File

@ -92,6 +92,15 @@ UPDATES_LIBCRONET_LIB="/usr/lib/libcronet.so"
# API for NetShift itself (same endpoint install.sh and get_system_info use);
# the self-update worker downloads the release .ipk/.apk assets from it.
NETSHIFT_RELEASE_API_URL="https://api.github.com/repos/yandexru45/netshift/releases/latest"
# GitHub FRONTEND (github.com, NOT the rate-limited api.github.com) redirect path
# for the NetShift repo. /releases/latest 302-redirects to /releases/tag/<tag>
# (resolve with curl -w '%{redirect_url}' — no API hit, not subject to the
# 60/hour/IP anonymous API limit); /releases/download/<tag>/<asset> 302s to the
# CDN for direct asset download. Primary path for version-check + self-update;
# NETSHIFT_RELEASE_API_URL stays as the graceful fallback. Repo slug lives here
# only — do not hardcode it elsewhere.
NETSHIFT_REPO_RELEASES_LATEST_URL="https://github.com/yandexru45/netshift/releases/latest"
NETSHIFT_REPO_RELEASES_DOWNLOAD_BASE="https://github.com/yandexru45/netshift/releases/download"
# tmpfs scratch dir for the self-update download (release packages) — RAM, never
# the tiny overlay; reaped on success and on reboot.
UPDATES_NETSHIFT_DOWNLOAD_DIR="/tmp/netshift/selfupdate"
@ -130,6 +139,12 @@ SB_DIRECT_OUTBOUND_TAG="direct-out"
# Subscription grouping (task-044). Default codepoint count for prefix-mode
# grouping when subscription_group_prefix_len is unset/invalid.
SUBSCRIPTION_GROUP_DEFAULT_PREFIX_LEN=2
# Subscription grouping (task-050). Tag/label for the top-level "Fastest"
# urltest that probes ACROSS the per-group urltests (a urltest of urltests)
# when grouping is on. Valid UTF-8 emoji + English; deliberately distinct from
# a per-group "<flag> Fastest" tag so the cross-group auto choice is tellable
# apart in the dashboard. Single source for the tag (keep this file UTF-8).
SB_SUBSCRIPTION_FASTEST_GROUP_TAG="⚡ Fastest"
# Route
SB_REJECT_RULE_TAG="reject-rule-tag"
SB_EXCLUSION_RULE_TAG="exclusion-rule-tag"

View File

@ -1626,15 +1626,45 @@ updates_self_update_netshift() {
return "$rc"
}
# Echoes the GitHub latest-release tag for NetShift (e.g. "0.8.8"), or nothing.
# Reuses the same API endpoint as get_system_info / install.sh. Parsed with jq
# (NOT grep/cut): GitHub may return the release object pretty-printed OR minified
# (single line); a field-positional grep|cut grabs the first key's value (the
# release "url") on minified JSON, which caused a false "outdated" + a self-update
# that downloaded a garbage "version". jq is format-independent.
updates_netshift_latest_tag() {
local response tag
# Resolve a URL's HTTP redirect target via curl WITHOUT hitting the rate-limited
# API or downloading the body. Echoes the redirect URL (empty if curl absent or
# no redirect). Stubbable in tests.
updates_github_resolve_redirect() {
local url="$1"
command -v curl >/dev/null 2>&1 || return 1
curl -sI -o /dev/null -w '%{redirect_url}' --connect-timeout 5 -m 15 -A 'netshift-updater' "$url" 2>/dev/null
}
# Echoes the GitHub latest-release tag for NetShift (e.g. "0.8.8"), or nothing.
# PRIMARY: resolve the github.com frontend redirect of /releases/latest — it
# 302s to /releases/tag/<tag>. That frontend is NOT the 60/hour-per-IP
# api.github.com, so it sidesteps the anonymous rate limit entirely (the common
# failure on CGNAT / shared-IP / shared-VPN-egress routers). FALLBACK: the
# api.github.com release object parsed with jq (task-047) so a curl-less box or a
# changed-redirect github still degrades gracefully instead of hard-failing.
# jq is format-independent (minified or pretty); a field-positional grep|cut
# grabbed the wrong key on minified JSON, causing a false "outdated".
# Bare tag on success / non-zero otherwise (contract consumed by
# updates_check_netshift and the self-update worker).
updates_netshift_latest_tag() {
local response tag redirect
# PRIMARY: github.com/<repo>/releases/latest 302-redirects to
# /releases/tag/<tag>. Parse with case/param-expansion (no Oniguruma).
redirect="$(updates_github_resolve_redirect "$NETSHIFT_REPO_RELEASES_LATEST_URL")"
case "$redirect" in
*/releases/tag/*)
tag="${redirect##*/releases/tag/}"
case "$tag" in '' | */*) tag="" ;; esac
;;
*) tag="" ;;
esac
if [ -n "$tag" ]; then
printf '%s' "$tag"
return 0
fi
# FALLBACK: api.github.com (rate-limited) parsed with jq.
response="$(updates_http_get_once "$NETSHIFT_RELEASE_API_URL" "")"
if [ -z "$response" ]; then
return 1
@ -1645,25 +1675,75 @@ updates_netshift_latest_tag() {
printf '%s' "$tag"
}
# Downloads the NetShift release assets matching the package-name prefixes for
# the active package manager into $dir. Echoes nothing; returns 0 if at least
# the core "netshift" package was downloaded, non-zero otherwise. The asset URL
# list comes from the same latest-release JSON, filtered to .ipk or .apk by the
# package manager (busybox grep -o, no jq array walk required).
# Echo the deterministic release asset filename for a package + tag + ext.
# ipk core/luci carry "-r1-all"; apk core/luci carry "-r1"; the i18n package
# carries neither suffix (just "<pkg>-<tag>.<ext>"). Single source of the asset
# naming pattern so it lives in one place, not scattered.
updates_netshift_asset_filename() {
local pkg="$1" tag="$2" ext="$3"
case "$pkg" in
"$UPDATES_NETSHIFT_PKG_I18N_RU") printf '%s-%s.%s' "$pkg" "$tag" "$ext" ;;
*)
if [ "$ext" = "ipk" ]; then
printf '%s-%s-r1-all.%s' "$pkg" "$tag" "$ext"
else
printf '%s-%s-r1.%s' "$pkg" "$tag" "$ext"
fi
;;
esac
}
# Downloads the NetShift release assets for the active package manager into $dir.
# Echoes nothing; returns 0 if at least the core "netshift" package was
# downloaded, non-zero otherwise.
# PRIMARY: resolve the latest tag (redirect-based, rate-limit-free) and build the
# deterministic github.com/<repo>/releases/download/<tag>/<asset> URLs — the
# CDN 302 is followed by updates_download_to_file (curl -L / wget both follow it).
# FALLBACK: if the tag can't be resolved, scrape the api.github.com release JSON
# for .ipk/.apk URLs (busybox grep -o) as before, so a curl-less box still works.
_updates_self_update_download_assets() {
local dir="$1"
local response ext pattern url filename dest attempt got_core=0
response="$(updates_http_get_once "$NETSHIFT_RELEASE_API_URL" "")"
if [ -z "$response" ]; then
return 1
fi
local tag pkg
if updates_pkg_is_apk; then
ext="apk"
else
ext="ipk"
fi
tag="$(updates_netshift_latest_tag)"
if [ -n "$tag" ]; then
# Direct deterministic asset URLs (no API). Core + luci always; the RU
# i18n package only if already installed.
for pkg in "$UPDATES_NETSHIFT_PKG_CORE" "$UPDATES_NETSHIFT_PKG_LUCI" "$UPDATES_NETSHIFT_PKG_I18N_RU"; do
if [ "$pkg" = "$UPDATES_NETSHIFT_PKG_I18N_RU" ]; then
updates_pkg_is_installed "$UPDATES_NETSHIFT_PKG_I18N_RU" || continue
fi
filename="$(updates_netshift_asset_filename "$pkg" "$tag" "$ext")"
url="$NETSHIFT_REPO_RELEASES_DOWNLOAD_BASE/$tag/$filename"
dest="$dir/$filename"
attempt=0
while [ "$attempt" -lt 3 ]; do
if updates_download_to_file "$url" "$dest"; then
break
fi
rm -f "$dest" 2>/dev/null
attempt=$((attempt + 1))
done
if [ "$pkg" = "$UPDATES_NETSHIFT_PKG_CORE" ] && [ -s "$dest" ]; then
got_core=1
fi
done
[ "$got_core" -eq 1 ]
return $?
fi
# FALLBACK: scrape the API release JSON for direct asset URLs.
response="$(updates_http_get_once "$NETSHIFT_RELEASE_API_URL" "")"
if [ -z "$response" ]; then
return 1
fi
pattern="https://[^\"[:space:]]*\.${ext}"
# Iterate the matching browser_download_url values. Only keep assets whose

View File

@ -9,9 +9,10 @@
# docker compose -f tests/docker-compose.yml run --rm netshift-test <test-name>
#
# Test names: all, deps, syntax, config, helpers, jq, cm, sb, nft,
# nftv6, selmark, isolation, monfd, unsupported, diagnostics, subscription, insecure, rejected,
# jobstate, selfheal, dnsdetour, globalproxy, stablecheck,
# extcheck, netshiftcheck, latesttag, selfupdate, backupguard
# nftv6, selmark, isolation, monfd, unsupported, diagnostics, subscription, fastest, insecure, rejected,
# jobstate, selfheal, dnsdetour, suburlopt, globalproxy, stablecheck,
# extcheck, netshiftcheck, latesttag, ghredirect, selfupdate,
# backupguard
# ──────────────────────────────────────────────────────────────────
services:

View File

@ -3744,6 +3744,259 @@ CCEOF
rm -f "$cc" "$cc_out"
}
# ─────────────────────────────────────────────────────────────────
# Test: "Fastest" cross-group urltest of urltests (task-050)
#
# When subscription grouping is ON (country/prefix) and there are >= 2 groups,
# the grouped branch in bin/netshift adds a top-level urltest tagged
# $SB_SUBSCRIPTION_FASTEST_GROUP_TAG whose members are the per-group urltests
# ("<key> Fastest"), PREPENDS it to the main selector, and makes it the selector
# default. Groups + ungrouped stay selectable. groups==1 -> no nested layer
# (default = lone group). off -> flat urltest+selector unchanged.
#
# The grouped branch is inline shell inside configure_outbound_handler (not its
# own function), so we awk-extract that exact code region VERBATIM out of the
# live bin (from the branch's `local grouping_json ...` decl through the final
# grouped selector build) and wrap it in a driver function — the test exercises
# the SHIPPED logic, not a copy. We seed $config with synthetic flag-tagged
# shadowsocks outbounds (no real subscription data) so the generated config can
# be fed to `sing-box check`. Tokens use the name:OK/FAIL convention; the driver
# output is parsed in the CURRENT shell (no pipe) so the tokens GATE CI.
# ─────────────────────────────────────────────────────────────────
test_fastest_group() {
header "Fastest Cross-Group urltest (task-050)"
if ! command -v jq > /dev/null 2>&1; then
skip "jq not available"
return
fi
local bin="${NETSHIFT_SRC}/usr/bin/netshift"
local constants="${NETSHIFT_LIB_DIR}/constants.sh"
local manager="${NETSHIFT_LIB_DIR}/sing_box_config_manager.sh"
if [ ! -r "$bin" ] || [ ! -r "$constants" ] || [ ! -r "$manager" ]; then
skip "netshift bin / constants.sh / config_manager.sh not found"
return
fi
local work="/tmp/netshift-fastest-$$"
mkdir -p "$work"
local drv="$work/driver.sh"
cat > "$drv" << 'FGEOF'
# Quiet logger (the grouped branch logs at info/debug/fatal; never let a fatal
# log mask the real exit code — the branch calls `exit 1` itself on failure).
log() { :; }
echolog() { :; }
nolog() { :; }
# Real constant ($SB_SUBSCRIPTION_FASTEST_GROUP_TAG) + cm primitives.
. "CONSTANTS_PATH"
. "MANAGER_PATH"
# Pull the shipped helpers VERBATIM out of the live bin so we test shipped code.
eval "$(awk '/^sing_box_get_unique_outbound_tag\(\) \{/{p=1} p{print} p&&/^\}/{exit}' "BIN_PATH")"
eval "$(awk '/^sing_box_build_subscription_groups\(\) \{/{p=1} p{print} p&&/^\}/{exit}' "BIN_PATH")"
# Extract the WHOLE grouping if/else region VERBATIM (the grouped `then` branch
# AND the flat `off` `else` branch) and wrap it as a function so we can drive
# both modes against the SHIPPED code. The leading `if ...; then local ...`
# line is valid inside this wrapper.
_grouped_branch() {
EXTRACT_GROUPED
}
# The off branch derives its urltest tag via get_outbound_tag_by_section; the
# grouped branch derives only $selector_tag (which we set ourselves). Stub it
# deterministically (synthetic, no real data).
get_outbound_tag_by_section() { printf '%s-out' "$1"; }
run_case() {
# $1 = group_mode, $2 = prefix_len, $3 = tags-json, $4 = base config (with
# the synthetic outbounds), $5 = selector tag. Echoes the resulting config.
group_mode="$1"
prefix_len="$2"
subscription_outbound_tags_json="$3"
config="$4"
selector_tag="$5"
section="syn"
urltest_testing_url="https://www.gstatic.com/generate_204"
urltest_check_interval="3m0s"
urltest_tolerance="50"
selector_outbounds=""
selector_default=""
_grouped_branch
printf '%s' "$config"
}
FGEOF
# Build the synthetic outbound set: two country groups (RU/DE flags) with two
# nodes each + one ungrouped node. Flags are regional-indicator pairs built
# by codepoint so NO real subscription identifiers appear anywhere.
local synth_json
synth_json="$(jq -cn '
def flag($a; $b): ([127462 + $a, 127462 + $b] | implode);
(flag(17; 20)) as $ru # RU
| (flag(3; 4)) as $de # DE
| {
outbounds: [
{type:"shadowsocks", tag:($ru + " N1"), server:"10.0.0.1", server_port:443, method:"aes-256-gcm", password:"p"},
{type:"shadowsocks", tag:($ru + " N2"), server:"10.0.0.2", server_port:443, method:"aes-256-gcm", password:"p"},
{type:"shadowsocks", tag:($de + " N1"), server:"10.0.0.3", server_port:443, method:"aes-256-gcm", password:"p"},
{type:"shadowsocks", tag:($de + " N2"), server:"10.0.0.4", server_port:443, method:"aes-256-gcm", password:"p"},
{type:"shadowsocks", tag:"plain-node", server:"10.0.0.5", server_port:443, method:"aes-256-gcm", password:"p"},
{type:"direct", tag:"direct-out"}
]
}')"
local tags_json
tags_json="$(printf '%s' "$synth_json" | jq -c '[.outbounds[] | select(.type=="shadowsocks") | .tag]')"
# Single-group set: only RU nodes (no DE, no ungrouped).
local synth1_json synth1_tags
synth1_json="$(jq -cn '
def flag($a; $b): ([127462 + $a, 127462 + $b] | implode);
(flag(17; 20)) as $ru
| {
outbounds: [
{type:"shadowsocks", tag:($ru + " N1"), server:"10.0.1.1", server_port:443, method:"aes-256-gcm", password:"p"},
{type:"shadowsocks", tag:($ru + " N2"), server:"10.0.1.2", server_port:443, method:"aes-256-gcm", password:"p"},
{type:"direct", tag:"direct-out"}
]
}')"
synth1_tags="$(printf '%s' "$synth1_json" | jq -c '[.outbounds[] | select(.type=="shadowsocks") | .tag]')"
# Substitute the awk-extracted grouped-branch region into the driver. The
# region is plain shell statements; sed reads it from the live bin between
# the unique markers and writes it where EXTRACT_GROUPED sits.
local region="$work/region.sh"
# Capture from the `if [ "$group_mode" != "off" ]; then` opener through the
# off-branch's final selector build line and the immediately following `fi`
# that closes the if/else (q-flag stops after that fi).
awk '
/if \[ "\$group_mode" != "off" \]; then/{p=1}
p{print}
p && /"\$urltest_tag" "true"\)"/{seen_else_end=1; next}
seen_else_end && /^[[:space:]]*fi$/{exit}
' "$bin" > "$region"
# Confirm the region captured BOTH branches: the fastest prepend (grouped),
# the cm urltest call, and the off-branch closing.
if grep -q 'SB_SUBSCRIPTION_FASTEST_GROUP_TAG' "$region" \
&& grep -q 'sing_box_cm_add_urltest_outbound' "$region" \
&& grep -q 'Create urltest + selector' "$region"; then
pass "fastest-region-extracted:OK"
else
fail "fastest-region-extracted:FAIL" "$(head -5 "$region" 2>/dev/null)"
fi
# Splice region into the driver in place of the EXTRACT_GROUPED placeholder
# (use an r-command via a temp because the region contains arbitrary chars).
{
sed '/EXTRACT_GROUPED/q' "$drv" | sed '$d'
cat "$region"
sed -n '/EXTRACT_GROUPED/,$p' "$drv" | sed '1d'
} > "$drv.spliced"
mv "$drv.spliced" "$drv"
sed -i "s|CONSTANTS_PATH|$constants|g;s|MANAGER_PATH|$manager|g;s|BIN_PATH|$bin|g" "$drv"
# ── >= 2 groups (country mode): nested Fastest urltest + selector default ──
local out2="$work/out2.json"
{
echo ". \"$drv\""
echo "run_case country 2 '$tags_json' '$synth_json' 'syn-out'"
} > "$work/run2.sh"
ash "$work/run2.sh" > "$out2" 2>/dev/null || true
# The deduped fastest tag (the constant; no collision in our synthetic set).
local fastest_expected ru_tag de_tag
fastest_expected="$(. "$constants"; printf '%s' "$SB_SUBSCRIPTION_FASTEST_GROUP_TAG")"
ru_tag="$(printf '%s' "$synth_json" | jq -r '.outbounds[0].tag' | sed 's/ N1$//') Fastest"
de_tag="$(printf '%s' "$synth_json" | jq -r '.outbounds[2].tag' | sed 's/ N1$//') Fastest"
# (a) Top-level urltest tagged the fastest tag whose outbounds are EXACTLY
# the per-group urltest tags.
if jq -e --arg t "$fastest_expected" --arg g1 "$ru_tag" --arg g2 "$de_tag" '
([.outbounds[] | select(.type=="urltest" and .tag==$t)]) as $f
| ($f | length) == 1
and ($f[0].outbounds == [$g1, $g2])
' "$out2" > /dev/null 2>&1; then
pass "fastest-nested-urltest-members:OK"
else
fail "fastest-nested-urltest-members:FAIL" "$(jq -c '[.outbounds[]|select(.type=="urltest")|{tag,outbounds}]' "$out2" 2>/dev/null)"
fi
# (b) Main selector default == fastest tag, and outbounds ==
# [fastest, group1, group2, ungrouped...].
if jq -e --arg t "$fastest_expected" --arg g1 "$ru_tag" --arg g2 "$de_tag" '
([.outbounds[] | select(.type=="selector" and .tag=="syn-out")]) as $s
| ($s | length) == 1
and ($s[0].default == $t)
and ($s[0].outbounds == [$t, $g1, $g2, "plain-node"])
' "$out2" > /dev/null 2>&1; then
pass "fastest-selector-default-membership:OK"
else
fail "fastest-selector-default-membership:FAIL" "$(jq -c '.outbounds[]|select(.type=="selector")|{tag,default,outbounds}' "$out2" 2>/dev/null)"
fi
# (c) sing-box check PASSES on the generated config WITH the nested urltest.
if command -v sing-box > /dev/null 2>&1; then
local chk2="$work/check2.json"
# Wrap the outbounds into a minimal full config sing-box can validate.
jq '{
log: {disabled:true},
dns: {servers: [], rules: [], final: "direct"},
inbounds: [{type:"direct", tag:"dns-in", listen:"127.0.0.42", listen_port:53}],
outbounds: .outbounds,
route: {rules: [], rule_set: [], final: "direct-out", auto_detect_interface: true}
}' "$out2" > "$chk2" 2>/dev/null
if sing-box -c "$chk2" check > /dev/null 2>&1; then
pass "fastest-singbox-check-passes:OK"
else
fail "fastest-singbox-check-passes:FAIL" "$(sing-box -c "$chk2" check 2>&1 | head -3)"
fi
else
skip "fastest-singbox-check-passes (sing-box not installed)"
fi
# (d) groups==1 -> NO redundant nested urltest; default = lone group.
local out1="$work/out1.json"
{
echo ". \"$drv\""
echo "run_case country 2 '$synth1_tags' '$synth1_json' 'syn1-out'"
} > "$work/run1.sh"
ash "$work/run1.sh" > "$out1" 2>/dev/null || true
local lone_group
lone_group="$(printf '%s' "$synth1_json" | jq -r '.outbounds[0].tag' | sed 's/ N1$//') Fastest"
if jq -e --arg t "$fastest_expected" --arg lone "$lone_group" '
([.outbounds[] | select(.type=="urltest" and .tag==$t)] | length) == 0
and ([.outbounds[] | select(.type=="selector" and .tag=="syn1-out")][0].default == $lone)
' "$out1" > /dev/null 2>&1; then
pass "fastest-single-group-no-nest:OK"
else
fail "fastest-single-group-no-nest:FAIL" "$(jq -c '[.outbounds[]|select(.type=="urltest" or .type=="selector")|{type,tag,default}]' "$out1" 2>/dev/null)"
fi
# (e) off mode unchanged (regression): flat urltest + selector, NO fastest
# tag, selector default == the flat urltest tag (<section>-urltest-out).
local outoff="$work/outoff.json"
{
echo ". \"$drv\""
echo "run_case off 2 '$tags_json' '$synth_json' 'syn-out'"
} > "$work/runoff.sh"
ash "$work/runoff.sh" > "$outoff" 2>/dev/null || true
if jq -e --arg t "$fastest_expected" '
([.outbounds[] | select(.type=="urltest" and .tag==$t)] | length) == 0
and ([.outbounds[] | select(.type=="urltest")] | length) == 1
and ([.outbounds[] | select(.type=="selector" and .tag=="syn-out")][0].default
== "syn-urltest-out")
' "$outoff" > /dev/null 2>&1; then
pass "fastest-off-mode-unchanged:OK"
else
fail "fastest-off-mode-unchanged:FAIL" "$(jq -c '[.outbounds[]|select(.type=="urltest" or .type=="selector")|{type,tag,default}]' "$outoff" 2>/dev/null)"
fi
rm -rf "$work"
}
# ─────────────────────────────────────────────────────────────────
# Test: Insecure subscription fetch flag (task-021b)
#
@ -4836,6 +5089,187 @@ DDEOF
rm -f "$drv"
}
# ─────────────────────────────────────────────────────────────────
# Test: scalar `option subscription_url` read-fallback + option->list migration
# (task-048)
# ─────────────────────────────────────────────────────────────────
# REAL-UCI regression guard for the hardware bug: a section storing
# subscription_url as a scalar UCI `option` (legacy / CLI / podkop-migrated
# configs) made get_subscription_urls_for_section return EMPTY (config_list_foreach
# iterates ONLY list values), so has_outbound_section failed and sing-box never
# started. This must use the SHIPPED functions against an actual config_load — NOT
# the stubbed config_list_foreach in test_subscription (which honors MU_URLS
# directly and therefore cannot catch the broken primitive). Synthetic URL only.
test_sub_url_option() {
header "Scalar option subscription_url read-fallback + migration (task-048)"
local bin="${NETSHIFT_SRC}/usr/bin/netshift"
if [ ! -r "$bin" ]; then
skip "suburlopt — bin/netshift not found"
return
fi
if [ ! -r /lib/functions.sh ] || [ ! -r /lib/config/uci.sh ] || ! command -v uci > /dev/null 2>&1; then
skip "suburlopt — LuCI config_load / uci not available"
return
fi
local lib="${NETSHIFT_LIB_DIR}"
local drv="/tmp/netshift-suburlopt-$$.sh"
cat > "$drv" << 'SUBOPTEOF'
BIN="BIN_PATH_PLACEHOLDER"
LIB="LIB_DIR_PLACEHOLDER"
. /lib/functions.sh
. /lib/config/uci.sh 2>/dev/null || true
# shellcheck disable=SC1090
. "$LIB/constants.sh"
# shellcheck disable=SC1090
. "$LIB/helpers.sh"
log() { :; }
echolog() { :; }
nolog() { :; }
# Exercise the SHIPPED functions verbatim (awk-extracted) against a real
# config_load — this is the whole point: the real LuCI config_list_foreach /
# config_get primitives, not a stub.
for fn in get_subscription_urls_for_section _collect_subscription_url_handler \
section_has_configured_outbound \
migrate_legacy_subscription_url_option \
_migrate_legacy_subscription_url_option_handler; do
eval "$(awk -v f="$fn" '$0 ~ "^"f"\\(\\) \\{"{p=1} p{print} p&&/^\}/{exit}' "$BIN")"
done
mkdir -p /etc/config
# ── Fixture A: SCALAR option subscription_url (the exact broken shape) ──
cat > /etc/config/netshift_suboptscalar <<'CFGEOF'
config section 'main'
option connection_type 'proxy'
option proxy_config_type 'subscription'
option subscription_url 'https://example.com/sub'
CFGEOF
config_load netshift_suboptscalar
urls="$(get_subscription_urls_for_section main)"
[ "$urls" = "https://example.com/sub" ] && echo 'suburlopt:scalar-read:OK' || echo "suburlopt:scalar-read:FAIL [$urls]"
if section_has_configured_outbound main; then
echo 'suburlopt:scalar-hasoutbound:OK'
else
echo 'suburlopt:scalar-hasoutbound:FAIL'
fi
rm -f /etc/config/netshift_suboptscalar
# ── Fixture B: LIST subscription_url (must still work — no regression) ──
cat > /etc/config/netshift_suboptlist <<'CFGEOF'
config section 'main'
option connection_type 'proxy'
option proxy_config_type 'subscription'
list subscription_url 'https://example.com/sub'
CFGEOF
config_load netshift_suboptlist
urls="$(get_subscription_urls_for_section main)"
[ "$urls" = "https://example.com/sub" ] && echo 'suburlopt:list-read:OK' || echo "suburlopt:list-read:FAIL [$urls]"
rm -f /etc/config/netshift_suboptlist
# ── Migration: option -> list, idempotent. The migration function hardcodes
# the `netshift` config name, so write a throwaway /etc/config/netshift (the
# caller backs up + restores any real one). Two sections: a plain URL AND a URL
# with a query string containing `=`/`&`/`?` — the latter is the [B1] regression
# guard: the old `uci add_list "key=value"` CLI form splits on the first `=` and
# LOSES the value, while uci_add_list preserves it byte-for-byte. ──
NETSHIFT_CONFIG="netshift"
EQ_URL='https://example.com/sub?token=abc&x=1'
cat > /etc/config/netshift <<CFGEOF
config section 'main'
option connection_type 'proxy'
option proxy_config_type 'subscription'
option subscription_url 'https://example.com/sub'
config section 'query'
option connection_type 'proxy'
option proxy_config_type 'subscription'
option subscription_url '$EQ_URL'
CFGEOF
config_load netshift
# First run: must migrate both scalar options -> lists and flip the flag.
migrate_legacy_subscription_url_option
if [ "$SUBSCRIPTION_URL_OPTION_MIGRATED" = "1" ]; then
echo 'suburlopt:migrate-flag:OK'
else
echo "suburlopt:migrate-flag:FAIL [$SUBSCRIPTION_URL_OPTION_MIGRATED]"
fi
# The stored values must be preserved.
migrated_val="$(uci -q get netshift.main.subscription_url)"
[ "$migrated_val" = "https://example.com/sub" ] && echo 'suburlopt:migrate-value:OK' || echo "suburlopt:migrate-value:FAIL [$migrated_val]"
# [B1] regression guard: the `=`/`&` URL survives byte-for-byte.
migrated_eq="$(uci -q get netshift.query.subscription_url)"
[ "$migrated_eq" = "$EQ_URL" ] && echo 'suburlopt:migrate-equrl-preserved:OK' || echo "suburlopt:migrate-equrl-preserved:FAIL [$migrated_eq]"
# After a fresh config_load the LIST path (config_list_foreach) returns each URL,
# and the committed state must be a CLEAN single-element list (no leftover scalar
# option and no duplicate element).
config_load netshift
SUBSCRIPTION_URLS_COLLECTED=""
config_list_foreach main subscription_url _collect_subscription_url_handler
[ "$SUBSCRIPTION_URLS_COLLECTED" = "https://example.com/sub" ] && echo 'suburlopt:migrate-islist:OK' || echo "suburlopt:migrate-islist:FAIL [$SUBSCRIPTION_URLS_COLLECTED]"
SUBSCRIPTION_URLS_COLLECTED=""
config_list_foreach query subscription_url _collect_subscription_url_handler
[ "$SUBSCRIPTION_URLS_COLLECTED" = "$EQ_URL" ] && echo 'suburlopt:migrate-equrl-islist:OK' || echo "suburlopt:migrate-equrl-islist:FAIL [$SUBSCRIPTION_URLS_COLLECTED]"
# Clean single element: `uci show` must render exactly one list value per section
# (no leftover scalar option, no duplicate). uci renders a list element with the
# index-bearing `[0]` syntax; assert exactly one line each.
eq_lines="$(uci -q show netshift.query.subscription_url | grep -c "subscription_url")"
[ "$eq_lines" = "1" ] && echo 'suburlopt:migrate-equrl-single:OK' || echo "suburlopt:migrate-equrl-single:FAIL [$eq_lines]"
# Second run: idempotent no-op (already a list -> flag stays 0, no churn).
migrate_legacy_subscription_url_option
if [ "$SUBSCRIPTION_URL_OPTION_MIGRATED" = "0" ]; then
echo 'suburlopt:migrate-idempotent:OK'
else
echo "suburlopt:migrate-idempotent:FAIL [$SUBSCRIPTION_URL_OPTION_MIGRATED]"
fi
idem_val="$(uci -q get netshift.main.subscription_url)"
[ "$idem_val" = "https://example.com/sub" ] && echo 'suburlopt:migrate-idempotent-value:OK' || echo "suburlopt:migrate-idempotent-value:FAIL [$idem_val]"
idem_eq="$(uci -q get netshift.query.subscription_url)"
[ "$idem_eq" = "$EQ_URL" ] && echo 'suburlopt:migrate-idempotent-equrl:OK' || echo "suburlopt:migrate-idempotent-equrl:FAIL [$idem_eq]"
rm -f /etc/config/netshift
echo 'DONE'
SUBOPTEOF
sed -i "s|LIB_DIR_PLACEHOLDER|$lib|g; s|BIN_PATH_PLACEHOLDER|$bin|g" "$drv"
# Protect any real /etc/config/netshift the container may carry: the
# migration path writes a throwaway one under that exact name.
local netshift_cfg_backup=""
if [ -f /etc/config/netshift ]; then
netshift_cfg_backup="/tmp/netshift-cfg-backup-$$"
cp /etc/config/netshift "$netshift_cfg_backup"
fi
# Parse in the CURRENT shell (temp file + `while read < "$out"`, NO pipe) so
# pass/fail update the global counters and a suburlopt:*:FAIL actually gates
# the suite (a pipe would run the while-body in a subshell — non-gating).
local sub_out="/tmp/netshift-suburlopt-out-$$"
sh "$drv" > "$sub_out" 2>/dev/null
# FAIL/SKIP tokens carry a trailing " [diagnostic]" suffix, so match with a
# trailing glob (*:FAIL*) — a bare "*:FAIL)" would miss them and silently
# drop the failure, defeating S1's gating.
while IFS= read -r line; do
case "$line" in
*:FAIL*) fail "$line" ;;
*:SKIP*) skip "$line" ;;
*:OK) pass "$line" ;;
DONE) ;;
*) ;;
esac
done < "$sub_out"
rm -f "$drv" "$sub_out"
if [ -n "$netshift_cfg_backup" ]; then
mv "$netshift_cfg_backup" /etc/config/netshift
else
rm -f /etc/config/netshift
fi
}
# ─────────────────────────────────────────────────────────────────
# Test: global_proxy route rule semantics
# ─────────────────────────────────────────────────────────────────
@ -5348,6 +5782,9 @@ updates_log() { :; }
. "DRV_UPDATER"
NETSHIFT_RELEASE_API_URL="https://api.test/latest"
NETSHIFT_VERSION="$STUBLT_INSTALLED"
# Force the API-fallback path this test targets: the redirect resolver returns
# empty so updates_netshift_latest_tag falls back to the stubbed API body.
updates_github_resolve_redirect() { printf ''; }
updates_http_get_once() { printf '%s' "$STUBLT_BODY"; }
"$STUBLT_FN"
DRVEOF
@ -5422,6 +5859,138 @@ DRVEOF
rm -rf "$work"
}
# ─────────────────────────────────────────────────────────────────
# Test: GitHub redirect-based latest-tag + deterministic asset URLs (task-049)
# ─────────────────────────────────────────────────────────────────
# Sidestepping the api.github.com 60/hour/IP rate limit: the version-check + the
# self-update asset download now resolve github.com/<repo>/releases/latest via a
# redirect (curl -w '%{redirect_url}') → /releases/tag/<tag>, with the API + jq
# path kept only as a graceful fallback. The network boundary is STUBBED here
# (override updates_github_resolve_redirect / updates_http_get_once), so no curl
# shell-out and no real network in CI. Synthetic data only.
test_github_redirect_tag() {
header "GitHub redirect latest-tag + asset URLs (task-049)"
if ! command -v jq > /dev/null 2>&1; then
skip "jq not available"
return
fi
local updater="${NETSHIFT_LIB_DIR}/updater.sh"
if [ ! -r "$updater" ]; then
skip "updater.sh not found in ${NETSHIFT_LIB_DIR}"
return
fi
local work="/tmp/netshift-ghredirect-$$"
rm -rf "$work"
mkdir -p "$work"
# Driver: source helpers.sh + updater.sh, silence logging, pin the redirect
# + API constants, OVERRIDE the redirect resolver ($STUBGR_REDIRECT) and the
# API boundary ($STUBGR_BODY), then run the function/expr named in $STUBGR_FN.
local drv="$work/driver.sh"
cat > "$drv" << 'DRVEOF'
log() { :; }
echolog() { :; }
nolog() { :; }
updates_log() { :; }
. "DRV_HELPERS"
. "DRV_UPDATER"
NETSHIFT_REPO_RELEASES_LATEST_URL="https://github.com/yandexru45/netshift/releases/latest"
NETSHIFT_REPO_RELEASES_DOWNLOAD_BASE="https://github.com/yandexru45/netshift/releases/download"
NETSHIFT_RELEASE_API_URL="https://api.test/latest"
UPDATES_NETSHIFT_PKG_CORE="netshift"
UPDATES_NETSHIFT_PKG_LUCI="luci-app-netshift"
UPDATES_NETSHIFT_PKG_I18N_RU="luci-i18n-netshift-ru"
updates_github_resolve_redirect() { printf '%s' "$STUBGR_REDIRECT"; }
updates_http_get_once() { printf '%s' "$STUBGR_BODY"; }
eval "$STUBGR_FN"
DRVEOF
sed -i "s|DRV_UPDATER|$updater|g;s|DRV_HELPERS|${NETSHIFT_LIB_DIR}/helpers.sh|g" "$drv"
local out="$work/out.txt"
local rc_file="$work/rc.txt"
run_gr() {
ash "$drv" > "$out" 2>/dev/null && printf '0' > "$rc_file" || printf '%s' "$?" > "$rc_file"
}
export STUBGR_FN="updates_netshift_latest_tag"
export STUBGR_REDIRECT=""
export STUBGR_BODY=""
# ── Case 1: clean redirect → tag 0.8.9 (primary path, no API) ────────────
export STUBGR_REDIRECT="https://github.com/yandexru45/netshift/releases/tag/0.8.9"
export STUBGR_BODY=""
run_gr
if [ "$(cat "$out" 2>/dev/null)" = "0.8.9" ] && [ "$(cat "$rc_file" 2>/dev/null)" = "0" ]; then
pass "ghredirect:tag-from-redirect:OK"
else
fail "ghredirect:tag-from-redirect:FAIL" "got=[$(cat "$out" 2>/dev/null)] rc=$(cat "$rc_file" 2>/dev/null)"
fi
# ── Case 2: trailing-slash redirect → parse rejects (slash) → falls back ──
# A trailing slash makes the stripped tag contain "/", which the guard
# rejects; with NO API body it then yields empty + non-zero.
export STUBGR_REDIRECT="https://github.com/yandexru45/netshift/releases/tag/0.8.9/"
export STUBGR_BODY=""
run_gr
if [ -z "$(cat "$out" 2>/dev/null)" ] && [ "$(cat "$rc_file" 2>/dev/null)" != "0" ]; then
pass "ghredirect:tag-trailing-slash-rejected:OK"
else
fail "ghredirect:tag-trailing-slash-rejected:FAIL" "got=[$(cat "$out" 2>/dev/null)] rc=$(cat "$rc_file" 2>/dev/null)"
fi
# ── Case 3: non-matching redirect (login page) → primary empty → API
# FALLBACK returns the release object → still yields the tag. ─────────────
export STUBGR_REDIRECT="https://github.com/login?return_to=%2Fyandexru45%2Fnetshift"
export STUBGR_BODY='{"url":"https://api.github.com/repos/yandexru45/netshift/releases/1","tag_name":"0.8.9"}'
run_gr
if [ "$(cat "$out" 2>/dev/null)" = "0.8.9" ] && [ "$(cat "$rc_file" 2>/dev/null)" = "0" ]; then
pass "ghredirect:nonmatch-falls-back:OK"
else
fail "ghredirect:nonmatch-falls-back:FAIL" "got=[$(cat "$out" 2>/dev/null)] rc=$(cat "$rc_file" 2>/dev/null)"
fi
# ── Case 4: curl-absent (resolver empty) + API rate-limit object → empty +
# non-zero (honest failure, no false tag). ───────────────────────────────
export STUBGR_REDIRECT=""
export STUBGR_BODY='{"message":"API rate limit exceeded for 1.2.3.4"}'
run_gr
if [ -z "$(cat "$out" 2>/dev/null)" ] && [ "$(cat "$rc_file" 2>/dev/null)" != "0" ]; then
pass "ghredirect:ratelimit-empty:OK"
else
fail "ghredirect:ratelimit-empty:FAIL" "got=[$(cat "$out" 2>/dev/null)] rc=$(cat "$rc_file" 2>/dev/null)"
fi
# ── Case 5: asset-URL builder, ipk → deterministic names ─────────────────
export STUBGR_REDIRECT=""
export STUBGR_BODY=""
export STUBGR_FN='c="$(updates_netshift_asset_filename netshift 0.8.9 ipk)"; l="$(updates_netshift_asset_filename luci-app-netshift 0.8.9 ipk)"; i="$(updates_netshift_asset_filename luci-i18n-netshift-ru 0.8.9 ipk)"; printf "%s\n%s\n%s\n" "$c" "$l" "$i"'
run_gr
if [ "$(sed -n 1p "$out" 2>/dev/null)" = "netshift-0.8.9-r1-all.ipk" ] &&
[ "$(sed -n 2p "$out" 2>/dev/null)" = "luci-app-netshift-0.8.9-r1-all.ipk" ] &&
[ "$(sed -n 3p "$out" 2>/dev/null)" = "luci-i18n-netshift-ru-0.8.9.ipk" ]; then
pass "ghredirect:asset-ipk:OK"
else
fail "ghredirect:asset-ipk:FAIL" "got=[$(cat "$out" 2>/dev/null)]"
fi
# ── Case 6: asset-URL builder, apk → deterministic names ─────────────────
export STUBGR_FN='c="$(updates_netshift_asset_filename netshift 0.8.9 apk)"; l="$(updates_netshift_asset_filename luci-app-netshift 0.8.9 apk)"; i="$(updates_netshift_asset_filename luci-i18n-netshift-ru 0.8.9 apk)"; printf "%s\n%s\n%s\n" "$c" "$l" "$i"'
run_gr
if [ "$(sed -n 1p "$out" 2>/dev/null)" = "netshift-0.8.9-r1.apk" ] &&
[ "$(sed -n 2p "$out" 2>/dev/null)" = "luci-app-netshift-0.8.9-r1.apk" ] &&
[ "$(sed -n 3p "$out" 2>/dev/null)" = "luci-i18n-netshift-ru-0.8.9.apk" ]; then
pass "ghredirect:asset-apk:OK"
else
fail "ghredirect:asset-apk:FAIL" "got=[$(cat "$out" 2>/dev/null)]"
fi
unset STUBGR_FN STUBGR_REDIRECT STUBGR_BODY
rm -rf "$work"
}
# ─────────────────────────────────────────────────────────────────
# Test: NetShift self-update (task-017)
# ─────────────────────────────────────────────────────────────────
@ -5959,16 +6528,19 @@ main() {
test_unsupported_skip
test_diagnostics
test_subscription
test_fastest_group
test_insecure_fetch
test_rejected_hash
test_jobstate
test_selfheal
test_dns_via_outbound
test_sub_url_option
test_global_proxy
test_check_update_stable
test_check_update_extended
test_check_update_netshift
test_netshift_latest_tag
test_github_redirect_tag
test_self_update_netshift
test_backup_integrity
;;
@ -5984,16 +6556,19 @@ main() {
unsupported) test_unsupported_skip ;;
diagnostics) test_diagnostics ;;
subscription) test_subscription ;;
fastest) test_fastest_group ;;
insecure) test_insecure_fetch ;;
rejected) test_rejected_hash ;;
jobstate) test_jobstate ;;
selfheal) test_selfheal ;;
dnsdetour) test_dns_via_outbound ;;
suburlopt) test_sub_url_option ;;
globalproxy) test_global_proxy ;;
stablecheck) test_check_update_stable ;;
extcheck) test_check_update_extended ;;
netshiftcheck) test_check_update_netshift ;;
latesttag) test_netshift_latest_tag ;;
ghredirect) test_github_redirect_tag ;;
selfupdate) test_self_update_netshift ;;
backupguard) test_backup_integrity ;;
jq) test_jq_helpers ;;
@ -6001,7 +6576,7 @@ main() {
sb) test_sing_box_config ;;
*)
echo "Unknown test: $target"
echo "Available: all deps syntax config helpers jq cm sb nft nftv6 selmark isolation monfd unsupported diagnostics subscription insecure rejected jobstate selfheal dnsdetour globalproxy stablecheck extcheck netshiftcheck latesttag selfupdate backupguard"
echo "Available: all deps syntax config helpers jq cm sb nft nftv6 selmark isolation monfd unsupported diagnostics subscription fastest insecure rejected jobstate selfheal dnsdetour suburlopt globalproxy stablecheck extcheck netshiftcheck latesttag ghredirect selfupdate backupguard"
exit 1
;;
esac