mirror of
https://github.com/yusufipk/OpenFrame.git
synced 2026-09-11 17:46:06 +00:00
test: close the coverage gaps the first round left
Second pass over the suite, driven by the inventory in the gaps document. Nine agents wrote suites in parallel against private databases, then a tenth read all of it adversarially and five of its findings were fixed. unit + component 2076 -> 2079 (+888 over the round) api 647 -> 1015 e2e 18 -> 29 What was closed: - lib/route-access.ts, the page-level authorization layer, went from zero tests to 48. Every API route was guarded and none of the pages were. - The five media proxy routes now have a real 2xx beside every 403. The blocker was the positive control, solved by stubbing r2Client.send() and leaving lib/r2-media-proxy.ts itself real. - Every remaining server-side lib module: invitations, email verification, the upload tokens, the logger, request origin, the whole R2 and Bunny lifecycle, notifications and admin stats. - Six video-page hooks, and the chunking arithmetic extracted out of lib/client/r2-video-upload.ts as a pure module. - Five end-to-end flows: workspace members, bulk operations, the admin area, player interaction and failure recovery. Three things about the harness itself turned out to be wrong: - Two @/lib/r2 stubs in tests/setup/api.ts had the wrong return shape, so every route reaching finalizeR2VideoUpload silently took the "not a valid video" branch and no test noticed. - The auth matrix asserted only "not 2xx", which two entries satisfied without their guard existing. It now requires 401 or 403, which makes both load-bearing, and all 60 routes pass the stricter form. - Both admin API routes had no positive control anywhere: replacing their guard with an unconditional refusal left the entire suite green. Found by the adversarial review, now covered. Process: - bun run test:mutation runs StrykerJS over the authorization and validation modules. Diagnostic, not a gate, weekly in CI rather than on a push. - playwright.config.ts gains an opt-in webkit project for the player spec. - AGENTS.md now requires a batch of new tests to be reviewed by somebody who did not write them. Only two production files change, both deliberate: lib/auth.ts loses a verbatim copy of its own permission formulas, and lib/client/r2-video-upload.ts calls the extracted arithmetic. No behaviour change in either.
This commit is contained in:
+34
-10
@@ -1,10 +1,11 @@
|
||||
#!/usr/bin/env sh
|
||||
# One entry point for the OpenFrame test suites.
|
||||
#
|
||||
# scripts/test.sh unit vitest unit + component projects
|
||||
# scripts/test.sh api vitest api project (needs the test database)
|
||||
# scripts/test.sh e2e playwright specs (needs the test database)
|
||||
# scripts/test.sh all unit, then api, then e2e
|
||||
# scripts/test.sh unit vitest unit + component projects
|
||||
# scripts/test.sh api vitest api project (needs the test database)
|
||||
# scripts/test.sh e2e playwright specs (needs the test database)
|
||||
# scripts/test.sh all unit, then api, then e2e
|
||||
# scripts/test.sh mutation StrykerJS over the authorization modules (slow)
|
||||
#
|
||||
# Run the api and e2e suites one at a time, never side by side. They share one
|
||||
# database, and the api suite empties every table after each of its tests, so a
|
||||
@@ -17,6 +18,11 @@
|
||||
set -eu
|
||||
|
||||
bun_image='docker.io/oven/bun:alpine'
|
||||
# StrykerJS is the one thing here that cannot run under bun: its instrumenter and
|
||||
# its vitest runner both want a node runtime, and @stryker-mutator/core declares
|
||||
# `engines.node >= 20`. `bun run test:mutation` would resolve the bin and then
|
||||
# execute it under bun, so the mutation mode below uses node explicitly.
|
||||
node_image='docker.io/library/node:22-alpine'
|
||||
# Pinned to the installed @playwright/test version. The image carries the
|
||||
# matching browser build, and Playwright refuses a mismatched pair. Microsoft
|
||||
# publishes the image some time after the npm release, so check the tag exists
|
||||
@@ -47,13 +53,18 @@ fi
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/test.sh <unit|api|e2e|all>
|
||||
Usage: scripts/test.sh <unit|api|e2e|all|mutation>
|
||||
|
||||
unit Unit and component suites. No database, no browser.
|
||||
api API integration suites. Starts the disposable test Postgres first.
|
||||
e2e Playwright end-to-end specs. Starts the test Postgres and MinIO first,
|
||||
then builds and starts the app itself on port 3100.
|
||||
all unit, then api, then e2e.
|
||||
unit Unit and component suites. No database, no browser.
|
||||
api API integration suites. Starts the disposable test Postgres first.
|
||||
e2e Playwright end-to-end specs. Starts the test Postgres and MinIO
|
||||
first, then builds and starts the app itself on port 3100.
|
||||
all unit, then api, then e2e.
|
||||
mutation StrykerJS over the authorization and validation modules listed in
|
||||
stryker.config.json. Minutes, not seconds, and not part of `all`:
|
||||
it answers "which of my tests cannot fail", which is a question you
|
||||
ask after writing a batch of them, not on every run. The report
|
||||
lands in reports/mutation/index.html.
|
||||
|
||||
The containers keep running afterwards so the next run is fast. Stop them
|
||||
with: podman compose -f docker-compose.test.yml --profile e2e down -v
|
||||
@@ -233,6 +244,16 @@ run_unit() {
|
||||
run_in_bun_image '' "$install_step && bun run test"
|
||||
}
|
||||
|
||||
run_mutation() {
|
||||
say 'mutation testing'
|
||||
# No install step: the node image has npm, and letting it touch node_modules
|
||||
# that bun installed is a good way to end up with two package managers
|
||||
# disagreeing. Run `scripts/test.sh unit` once first if the tree is cold.
|
||||
run_cmd podman run --rm ${tty_flag:+"$tty_flag"} \
|
||||
-v "$repo_root:/workspace:z" -w /workspace "$node_image" \
|
||||
node node_modules/@stryker-mutator/core/bin/stryker.js run
|
||||
}
|
||||
|
||||
run_api() {
|
||||
say 'api suites'
|
||||
require_compose_file
|
||||
@@ -294,6 +315,9 @@ case "$1" in
|
||||
run_api
|
||||
run_e2e
|
||||
;;
|
||||
mutation)
|
||||
run_mutation
|
||||
;;
|
||||
*)
|
||||
printf 'scripts/test.sh: unknown mode "%s"\n\n' "$1" >&2
|
||||
usage >&2
|
||||
|
||||
Reference in New Issue
Block a user