mirror of
https://github.com/yusufipk/OpenFrame.git
synced 2026-09-11 09:36:08 +00:00
Second pass over the suite, driven by the inventory in the gaps document. Nine agents wrote suites in parallel against private databases, then a tenth read all of it adversarially and five of its findings were fixed. unit + component 2076 -> 2079 (+888 over the round) api 647 -> 1015 e2e 18 -> 29 What was closed: - lib/route-access.ts, the page-level authorization layer, went from zero tests to 48. Every API route was guarded and none of the pages were. - The five media proxy routes now have a real 2xx beside every 403. The blocker was the positive control, solved by stubbing r2Client.send() and leaving lib/r2-media-proxy.ts itself real. - Every remaining server-side lib module: invitations, email verification, the upload tokens, the logger, request origin, the whole R2 and Bunny lifecycle, notifications and admin stats. - Six video-page hooks, and the chunking arithmetic extracted out of lib/client/r2-video-upload.ts as a pure module. - Five end-to-end flows: workspace members, bulk operations, the admin area, player interaction and failure recovery. Three things about the harness itself turned out to be wrong: - Two @/lib/r2 stubs in tests/setup/api.ts had the wrong return shape, so every route reaching finalizeR2VideoUpload silently took the "not a valid video" branch and no test noticed. - The auth matrix asserted only "not 2xx", which two entries satisfied without their guard existing. It now requires 401 or 403, which makes both load-bearing, and all 60 routes pass the stricter form. - Both admin API routes had no positive control anywhere: replacing their guard with an unconditional refusal left the entire suite green. Found by the adversarial review, now covered. Process: - bun run test:mutation runs StrykerJS over the authorization and validation modules. Diagnostic, not a gate, weekly in CI rather than on a push. - playwright.config.ts gains an opt-in webkit project for the player spec. - AGENTS.md now requires a batch of new tests to be reviewed by somebody who did not write them. Only two production files change, both deliberate: lib/auth.ts loses a verbatim copy of its own permission formulas, and lib/client/r2-video-upload.ts calls the extracted arithmetic. No behaviour change in either.
118 lines
7.2 KiB
Markdown
118 lines
7.2 KiB
Markdown
# AGENTS.md
|
|
|
|
## Must-follow constraints
|
|
|
|
- Use `bun` only. Do not use `npm` or `pnpm`.
|
|
- Do not start the dev server (`bun run dev`); assume it is already running.
|
|
- If you change `prisma/schema.prisma`, run `bun run db:generate`.
|
|
- In App Router dynamic routes, keep `params` typed as `Promise<...>` and `await params` in handlers/pages.
|
|
|
|
## Validation before finishing
|
|
|
|
- Run `bun run check`.
|
|
- Run `bun run verify` (this is `bun run check` plus the unit and component tests).
|
|
- If you touched an API route, also run `bun run test:api`. It needs the test database:
|
|
`bun run test:db:up` first.
|
|
- If you added a batch of tests, hand them to a second reviewer before calling the work
|
|
done. See "A batch of new tests gets an adversarial review, by somebody else" below.
|
|
|
|
## Testing
|
|
|
|
- The testing stack, layout and conventions live in `TESTING.md`. Read it before adding a
|
|
test.
|
|
- Tests live in a top-level `tests/` tree, never colocated with the code.
|
|
- Import test globals explicitly: `import { describe, it, expect, vi } from 'vitest';`.
|
|
- Never write a BigInt literal (`1n`) in any file. `tsconfig.json` targets ES2017, so `tsc`
|
|
rejects the syntax with TS2737 and `bun run check` fails. Use `BigInt(1)` instead, and
|
|
compare `BigInt(...)` against `BigInt(...)`.
|
|
|
|
### When a change needs a test
|
|
|
|
Match the change to a layer. Most changes need exactly one.
|
|
|
|
| You changed | Write |
|
|
| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
|
|
| A pure function in `lib/` (validation, a limit, a date rule, a URL or filename check, a permission calculation) | a unit test in `tests/unit/lib/` |
|
|
| An API route, or any authorization, quota or billing rule behind one | an integration test in `tests/api/` |
|
|
| A new API route | classify it in `tests/api/auth-matrix.test.ts`, or the suite fails until you do |
|
|
| Real logic in a React hook (optimistic updates, throttling, retries) | a hook test in `tests/component/hooks/` |
|
|
| A user-visible flow across more than one page | an end-to-end spec in `tests/e2e/` |
|
|
| Presentation only (styling, copy, layout, a `components/ui/` wrapper) | nothing |
|
|
|
|
Always write a test for a bug fix, at the layer where the bug lived. The test must fail
|
|
before the fix and pass after it. If it passes before the fix, it is testing the wrong
|
|
thing.
|
|
|
|
For an API route, three cases are the minimum: an unauthenticated caller, a caller who is
|
|
signed in but not authorized, and the happy path. Assert the database row, not only the
|
|
status code: a refused DELETE has to leave the row present.
|
|
|
|
### Two ways a test can be worthless
|
|
|
|
Both have been found in this repo, so they are worth naming.
|
|
|
|
1. **A test that cannot fail.** Before you finish, name the specific mutation of the
|
|
production code your test would catch. If you cannot name one, delete the test. When it
|
|
matters, prove it: break the code on purpose, watch the test go red, then revert.
|
|
2. **A test whose input comes from the code under test.** Iterating the same constant the
|
|
function looks up means deleting an entry from that constant also deletes its own test
|
|
case. Write expected values by hand as literals.
|
|
|
|
A third variant is specific to `tests/api/auth-matrix.test.ts`: a route that refuses a
|
|
malformed request before it reaches its access check produces an entry that passes whether
|
|
or not the guard exists. The suite catches it by requiring a 401 or a 403 rather than
|
|
merely a non-2xx, and a 404 counts as suspicious rather than as a refusal, since a fixture
|
|
id that stops resolving would otherwise pass forever. `NON_AUTHORIZATION_REFUSALS` and
|
|
`NOT_FOUND_IS_THE_GUARD` in that file explain the whole trap; both are empty, and adding to
|
|
either is meant to be a visible diff.
|
|
|
|
`bun run test:mutation` automates case 1 across the authorization and validation modules
|
|
listed in `stryker.config.json`. It is slow, so it is not part of `bun run check`, and CI
|
|
runs it weekly rather than on a push. Reach for it when you have written a batch of tests
|
|
and want to know which of them are decorative.
|
|
|
|
### A batch of new tests gets an adversarial review, by somebody else
|
|
|
|
**Rule: whoever wrote a batch of tests does not get to be the one who signs it off.** When
|
|
a change adds a meaningful number of tests (a new suite, or a set of them), a second
|
|
reviewer goes over them with one question in mind: _do these tests deliver what they claim
|
|
to?_ If the work is being done by agents, that reviewer is a separate agent with no stake
|
|
in the code it is reading.
|
|
|
|
This is not a style pass. The reviewer's job is to find:
|
|
|
|
- Tests that pass whether or not the production code works. Verify by mutation, do not take
|
|
the author's word for it, and prefer a mutation the author did not already try.
|
|
- Assertions weak enough to survive the bug they were written for: `toBeTruthy()` on an
|
|
object, a status code checked without checking the database row, a `403` with no `2xx`
|
|
beside it, a `not.toThrow()` standing in for a real expectation.
|
|
- A test whose subject is the mock rather than the code. If every dependency is stubbed,
|
|
ask what is left to be wrong.
|
|
- Coverage that reads as complete but is not: the happy path tested five ways and the
|
|
rollback, the concurrent call and the failure branch tested not at all.
|
|
- Names that promise more than the body checks. The name is what the next person trusts.
|
|
- Setup so elaborate that the test no longer describes a situation the app can reach.
|
|
|
|
Two rounds of this have already been run on this suite and both found real problems, so it
|
|
is worth the cost. Findings go back to the author to fix; the reviewer does not quietly
|
|
rewrite the tests.
|
|
|
|
## Repo-specific conventions
|
|
|
|
- Use `auth()` from `@/lib/auth` for server-side session reads.
|
|
- Use `checkProjectAccess()` / `checkWorkspaceAccess()` for authorization instead of ad-hoc role checks.
|
|
- For API responses, use `successResponse` / `apiErrors` from `@/lib/api-response`.
|
|
- Keep API and UI imports on `@/` aliases when available.
|
|
- In Prisma raw SQL, use `$executeRaw` for statements that return no rows (e.g. `pg_advisory_xact_lock`). Using `$queryRaw` on void-returning functions causes a Prisma deserialization error (`Failed to deserialize column of type 'void'`).
|
|
|
|
## Important locations
|
|
|
|
- Custom SQL managed by Prisma migrations: `prisma/migrations/*/migration.sql`.
|
|
- Shared API response helpers: `lib/api-response.ts`.
|
|
- Auth + access-control helpers: `lib/auth.ts`.
|
|
|
|
## Change safety rules
|
|
|
|
- Prefer backward-compatible API changes unless explicitly asked to break contracts.
|
|
- For multi-step DB writes, use Prisma transactions.
|