mirror of
https://github.com/yusufipk/OpenFrame.git
synced 2026-09-11 09:36:08 +00:00
test: close the coverage gaps the first round left
Second pass over the suite, driven by the inventory in the gaps document. Nine agents wrote suites in parallel against private databases, then a tenth read all of it adversarially and five of its findings were fixed. unit + component 2076 -> 2079 (+888 over the round) api 647 -> 1015 e2e 18 -> 29 What was closed: - lib/route-access.ts, the page-level authorization layer, went from zero tests to 48. Every API route was guarded and none of the pages were. - The five media proxy routes now have a real 2xx beside every 403. The blocker was the positive control, solved by stubbing r2Client.send() and leaving lib/r2-media-proxy.ts itself real. - Every remaining server-side lib module: invitations, email verification, the upload tokens, the logger, request origin, the whole R2 and Bunny lifecycle, notifications and admin stats. - Six video-page hooks, and the chunking arithmetic extracted out of lib/client/r2-video-upload.ts as a pure module. - Five end-to-end flows: workspace members, bulk operations, the admin area, player interaction and failure recovery. Three things about the harness itself turned out to be wrong: - Two @/lib/r2 stubs in tests/setup/api.ts had the wrong return shape, so every route reaching finalizeR2VideoUpload silently took the "not a valid video" branch and no test noticed. - The auth matrix asserted only "not 2xx", which two entries satisfied without their guard existing. It now requires 401 or 403, which makes both load-bearing, and all 60 routes pass the stricter form. - Both admin API routes had no positive control anywhere: replacing their guard with an unconditional refusal left the entire suite green. Found by the adversarial review, now covered. Process: - bun run test:mutation runs StrykerJS over the authorization and validation modules. Diagnostic, not a gate, weekly in CI rather than on a push. - playwright.config.ts gains an opt-in webkit project for the player spec. - AGENTS.md now requires a batch of new tests to be reviewed by somebody who did not write them. Only two production files change, both deliberate: lib/auth.ts loses a verbatim copy of its own permission formulas, and lib/client/r2-video-upload.ts calls the extracted arithmetic. No behaviour change in either.
This commit is contained in:
@@ -13,6 +13,8 @@
|
||||
- Run `bun run verify` (this is `bun run check` plus the unit and component tests).
|
||||
- If you touched an API route, also run `bun run test:api`. It needs the test database:
|
||||
`bun run test:db:up` first.
|
||||
- If you added a batch of tests, hand them to a second reviewer before calling the work
|
||||
done. See "A batch of new tests gets an adversarial review, by somebody else" below.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -56,6 +58,45 @@ Both have been found in this repo, so they are worth naming.
|
||||
function looks up means deleting an entry from that constant also deletes its own test
|
||||
case. Write expected values by hand as literals.
|
||||
|
||||
A third variant is specific to `tests/api/auth-matrix.test.ts`: a route that refuses a
|
||||
malformed request before it reaches its access check produces an entry that passes whether
|
||||
or not the guard exists. The suite catches it by requiring a 401 or a 403 rather than
|
||||
merely a non-2xx, and a 404 counts as suspicious rather than as a refusal, since a fixture
|
||||
id that stops resolving would otherwise pass forever. `NON_AUTHORIZATION_REFUSALS` and
|
||||
`NOT_FOUND_IS_THE_GUARD` in that file explain the whole trap; both are empty, and adding to
|
||||
either is meant to be a visible diff.
|
||||
|
||||
`bun run test:mutation` automates case 1 across the authorization and validation modules
|
||||
listed in `stryker.config.json`. It is slow, so it is not part of `bun run check`, and CI
|
||||
runs it weekly rather than on a push. Reach for it when you have written a batch of tests
|
||||
and want to know which of them are decorative.
|
||||
|
||||
### A batch of new tests gets an adversarial review, by somebody else
|
||||
|
||||
**Rule: whoever wrote a batch of tests does not get to be the one who signs it off.** When
|
||||
a change adds a meaningful number of tests (a new suite, or a set of them), a second
|
||||
reviewer goes over them with one question in mind: _do these tests deliver what they claim
|
||||
to?_ If the work is being done by agents, that reviewer is a separate agent with no stake
|
||||
in the code it is reading.
|
||||
|
||||
This is not a style pass. The reviewer's job is to find:
|
||||
|
||||
- Tests that pass whether or not the production code works. Verify by mutation, do not take
|
||||
the author's word for it, and prefer a mutation the author did not already try.
|
||||
- Assertions weak enough to survive the bug they were written for: `toBeTruthy()` on an
|
||||
object, a status code checked without checking the database row, a `403` with no `2xx`
|
||||
beside it, a `not.toThrow()` standing in for a real expectation.
|
||||
- A test whose subject is the mock rather than the code. If every dependency is stubbed,
|
||||
ask what is left to be wrong.
|
||||
- Coverage that reads as complete but is not: the happy path tested five ways and the
|
||||
rollback, the concurrent call and the failure branch tested not at all.
|
||||
- Names that promise more than the body checks. The name is what the next person trusts.
|
||||
- Setup so elaborate that the test no longer describes a situation the app can reach.
|
||||
|
||||
Two rounds of this have already been run on this suite and both found real problems, so it
|
||||
is worth the cost. Findings go back to the author to fix; the reviewer does not quietly
|
||||
rewrite the tests.
|
||||
|
||||
## Repo-specific conventions
|
||||
|
||||
- Use `auth()` from `@/lib/auth` for server-side session reads.
|
||||
|
||||
Reference in New Issue
Block a user