bun loads a plain `.env` into process.env before anything runs, and
tests/helpers/env.ts read that as a deliberate export, so it beat `.env.test`
outright. `scripts/test.sh api` therefore pointed the api suites at whatever
deployment `.env` describes: `prisma db push --accept-data-loss` for the
schema, then a truncate of every table between tests. The e2e half was worse,
because playwright.config.ts built and started the app with that DATABASE_URL
and those R2 credentials, then wrote fixtures into it. CI never saw any of
this: a runner has no `.env`.
Three changes, in order of what each one catches:
- helpers/dev-env.ts drops the values bun copied out of a development env file,
leaving `.env.test` to fill them. Only values that match the file
character-for-character go, so a real export still wins and the per-suite
databases of a parallel api run keep working.
- helpers/test-database.ts refuses a DATABASE_URL whose database name is not
marked as a test one, at the single point every path into the setup passes
through. This is the backstop, not the fix.
- playwright.config.ts blanks the variables a development env file defines and
the config does not. Dropping them from process.env is not enough there:
`next build` and `next start` run @next/env themselves and read the files
again. That is also why a local e2e run could not build at all (a set
DISABLE_RATE_LIMIT throws in lib/rate-limit.ts under NODE_ENV=production) and
why auth.spec.ts failed on a machine with SMTP configured.
`scripts/test.sh` now creates `.env.test` from the committed example instead of
asking for a one-line copy, so the guard above is something nobody has to meet.
Second pass over the suite, driven by the inventory in the gaps document. Nine
agents wrote suites in parallel against private databases, then a tenth read all
of it adversarially and five of its findings were fixed.
unit + component 2076 -> 2079 (+888 over the round)
api 647 -> 1015
e2e 18 -> 29
What was closed:
- lib/route-access.ts, the page-level authorization layer, went from zero tests
to 48. Every API route was guarded and none of the pages were.
- The five media proxy routes now have a real 2xx beside every 403. The blocker
was the positive control, solved by stubbing r2Client.send() and leaving
lib/r2-media-proxy.ts itself real.
- Every remaining server-side lib module: invitations, email verification, the
upload tokens, the logger, request origin, the whole R2 and Bunny lifecycle,
notifications and admin stats.
- Six video-page hooks, and the chunking arithmetic extracted out of
lib/client/r2-video-upload.ts as a pure module.
- Five end-to-end flows: workspace members, bulk operations, the admin area,
player interaction and failure recovery.
Three things about the harness itself turned out to be wrong:
- Two @/lib/r2 stubs in tests/setup/api.ts had the wrong return shape, so every
route reaching finalizeR2VideoUpload silently took the "not a valid video"
branch and no test noticed.
- The auth matrix asserted only "not 2xx", which two entries satisfied without
their guard existing. It now requires 401 or 403, which makes both
load-bearing, and all 60 routes pass the stricter form.
- Both admin API routes had no positive control anywhere: replacing their guard
with an unconditional refusal left the entire suite green. Found by the
adversarial review, now covered.
Process:
- bun run test:mutation runs StrykerJS over the authorization and validation
modules. Diagnostic, not a gate, weekly in CI rather than on a push.
- playwright.config.ts gains an opt-in webkit project for the player spec.
- AGENTS.md now requires a batch of new tests to be reviewed by somebody who
did not write them.
Only two production files change, both deliberate: lib/auth.ts loses a verbatim
copy of its own permission formulas, and lib/client/r2-video-upload.ts calls the
extracted arithmetic. No behaviour change in either.
The repo had no automated tests. Every change was verified by hand.
Adds four layers, 2023 tests in total, runnable with one command:
- 1191 unit tests over the pure logic in lib/, including the full
computeProjectAccess permission matrix and the billing gate
- 167 component and hook tests in jsdom, covering the hooks that hold
real logic rather than presentational wrappers
- 647 API integration tests against a real Postgres, with only auth()
mocked, including a data-driven sweep asserting that none of the 60
route modules answers 2xx to an unauthenticated caller
- 18 Playwright specs driving a real browser against a real build
Infrastructure: vitest.config.ts with three projects, a disposable
Postgres and MinIO in docker-compose.test.yml, factories and helpers
under tests/, scripts/test.sh as the single entry point, a pre-push
hook running bun run verify, and CI split into check, test and e2e jobs.
The test database is built with prisma db push plus a replay of the
hand-written SQL, because prisma migrate deploy cannot build this schema
from empty: the migration history has no captured baseline. This mirrors
what scripts/docker-db-bootstrap.ts already does in production, and
tests/setup/db-global.ts carries a drift guard so a new migration fails
the run until someone reviews it.
Production code is unchanged apart from one pure-function extraction out
of use-video-player.ts, which was too large to test in jsdom.
Several tests pin behaviour that looks wrong, each marked KNOWN BUG in
place. TESTING.md section 12 records where the plan turned out to be
wrong, and AGENTS.md now states which layer a change needs a test in.
yusufipk