mirror of
https://github.com/yusufipk/OpenFrame.git
synced 2026-09-11 17:46:06 +00:00
6561e0817e8229783188bf291b21246ec926f677
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Yusuf İpek
|
2c4c6101d5 |
Merge pull request #54 from yusufipk/fix/bunny-upload-reservation
fix(uploads): count a Bunny upload from the moment it is admitted |
||
 yusufipek
|
00f1d430b8 |
fix(uploads): stop a storage hold from being dropped by whoever can name it
A reservation id was never a secret and could not have been one. An upload token is base64url(payload) followed by its signature, so a client can read every claim out of its own token, and the two R2 init routes hand their reservation ids to the client outright. The asset route takes a reservation id from the request body and deleted it on the strength of that id and the billed user alone, and every hold an account owns is billed to the same user. So a caller could start a Bunny upload, read the id out of the token they were just given, quote it while attaching a one byte image or even a bare YouTube link, and have the quota handed back while the upload carried on. Repeat and a trial worth three gigabytes uploads as much as it likes for as long as Bunny takes to report a figure of its own. Signing the id rather than handing it over bought nothing, because signing is not hiding. A hold now records what it was opened for and is only ever consumed by that flow, so naming one is no longer enough to drop it. Guests hold against the workspace owner's quota rather than their own and had no way to give it back: the release was gated on being signed in. Declaring a size and walking away cost the guest nothing and cost the owner their whole remaining allowance for two hours. The guest grant now carries the reservation and the declared size, bound to the Bunny video as well as to ours, so cancelling gives the quota back and costs them the upload it stood for. What a guest can hold without cancelling lapses in half an hour rather than two hours. The in-transaction fallback check counted the account's Bunny storage as zero on a Bunny upload, because the figure was only prefetched for R2 providers and that branch was unreachable for Bunny until this PR made it reachable. On an account whose storage is all Bunny that was a check that could not fail. It is prefetched for every provider that can reach the fallback now. |
||
|
yusufipek
|
4ff801738c |
fix(uploads): count a Bunny upload from the moment it is admitted
A Bunny init asked the quota whether it could store zero bytes, which is a question with only one answer. Nothing an upload was about to consume was visible to the next request, so every init inside the same window read the same total and every one of them passed, and an upload that could never fit was only refused after it had been sent. The client now declares the size up front. It is checked against the account's remaining room before Bunny is asked for anything, and held as a reservation the next init has to see. The declaration is a claim rather than proof, so it is signed into the upload token: the same token already binds the video id, which is what makes the reservation safe to release on a caller's say-so, since releasing it costs them the video it belongs to. The declared size is then written onto the version or asset row and the reservation is dropped in the same transaction, because Bunny reports no size at all for a video until it has finished encoding it. On a half hour of footage that is most of an hour during which the upload did not appear on the uploader's own storage page and did not count against the next upload. Per-video accounting now takes the larger of what Bunny reports and what was declared, so the estimate stands in until the real figure arrives and Bunny's wins once it does. Two smaller things came out of the same reading. The asset route's in-transaction fallback compared against the plan limit, so a caller quoting a reservation that no longer existed was measured against 200 GiB even on a trial worth three. And the guest branch reserves without being able to release early, because a guest grant is bound to our video id and the caller's network context rather than to the Bunny video, which would let the reservation be dropped while the upload it stands for carried on. |
||
|
yusufipek
|
bfe3cb28b4 |
feat(admin): let a token read the growth scoreboard without a session
The weekly digest reads /api/admin/growth from a script, which has no browser and therefore no NextAuth session. The alternative was copying a session cookie out of a browser by hand: those are JWTs with a 30-day lifetime, so a scheduled job built on one stops working a month later and reports nothing rather than reporting a failure. The token path is off unless OPENFRAME_ADMIN_API_TOKEN is set, so an instance that never sets it keeps session-only admin access. A value under 32 characters is treated as no token at all: behind this header sit every paying account's name, email and usage, and a short token is a guessable path to all of it. Comparison runs over SHA-256 digests so it stays constant time without leaking the token's length. |
||
|
yusufipek
|
313cd552e6 |
fix(billing): stop an unpaid subscription from unlocking the paid limits
isPaidTier read a future stripeCurrentPeriodEnd as proof of payment, and Stripe stamps a current period on an incomplete subscription all the same. A checkout whose first charge failed therefore carried a period end a month out with nothing paid behind it, and every ceiling the cardless trial puts on an unpaid account (200 GB of storage, unlimited projects, unlimited workspaces) came off with it. Written as a deny list of the two statuses that mean no charge has ever gone through, so a real customer whose renewal failed keeps the full plan for the period they already paid for. hasBillingAccess reads the same column the same way and is deliberately left alone: being wrong there locks a paying customer out, and the SQL in buildBillingAccessWhereInput has to move with it. |
||
|
yusufipk
|
39e81042bb |
feat(billing): let people try the product before handing over a card
The trial now starts inside the product, at email verification, and Stripe grants none at all: checkout creates a subscription that bills immediately. Verifying an address is what buys the seven days, which is also the cheapest abuse control there is. An unexpired trial is treated as an entitlement the account already holds, so a Stripe sync can add access but never retracts a trial that has not run out. That matters most for the abandoned checkout: the resulting incomplete subscription carries no trial_end, and writing it through would have erased the days the account still had and locked it out. Unpaid accounts are bounded by what they can cost us rather than by what they can do: one workspace, one project, 3 GiB of direct uploads. YouTube imports, share links, guests, comments and approvals stay unlimited, because those are the parts worth trying and they cost nothing. isPaidTier() is the new seam; hasBillingAccess() answers a different question now that access no longer implies a card. Signup CTAs, the pricing card, the comparison pages, the terms and the refund policy all said the trial converts to a paid plan by itself. It no longer does, so they say what happens instead. Settings and a banner name both dates that matter: when the trial ends, and the fifteen days after that during which nothing is deleted. /admin/growth compares the two funnels on signup to paid within a fixed 30 day window, not trial to paid. Dropping the card requirement multiplies trials, so the old ratio can fall while more people actually pay, and reading it that way would retire the change for the wrong reason. |
||
|
yusufipk
|
33c845636c |
fix(analytics): sign the acquisition cookies and bound what they can write
Both cookies were read straight into database columns after nothing more than a format check. httpOnly keeps JavaScript out of them and does nothing about curl, so the anonymous id was a string the caller picked: enough to write a first-touch row for a visitor who never existed, to file it under a channel of their choosing, and to claim that id's events at signup, since the backfill matches on the id alone. They are now signed with an HMAC over AUTH_SECRET, through Web Crypto rather than node:crypto because the proxy runs on the edge and the pages that read the cookies back run in Node. The first-touch body moved to base64url on the way: cookie values are percent-encoded and decoded by several layers that do not agree on how many times, and a payload carrying its own percent escapes comes back subtly different and takes the signature with it. Signing stops a caller choosing an id, not collecting one, since dropping the cookie and asking for the landing page again mints another. So the bot and prefetch filters moved to where the rows are written rather than only where the cookies are issued, which also fixes a returning visitor's prefetch of /register recording a signup start, and a per-client hourly ceiling now sits in front of the write. The ceiling is skipped when TRUSTED_PROXY_MODE is unset, where every caller resolves to 127.0.0.1 and the bucket would empty on real traffic long before it emptied on a flood. Four smaller things around it: - /api/events checked the flag and the origin after paying for a rate-limit write, so a host who never turned analytics on was still writing a row per anonymous POST. Both checks are free and now come first, and the limiter answers 204 rather than 429: a beacon has nobody to tell, and a flooder should not be handed the reset time. - /api/onboarding/source was keyed by IP on an authenticated route. Without TRUSTED_PROXY_MODE that is five answers an hour for the whole deployment, and with it a shared office address locks out everyone after one colleague answered. Keyed by account, like /api/onboarding/complete beside it. - The cookies took their Secure flag from request.nextUrl.protocol, which behind a TLS-terminating reverse proxy is the container-internal http address. It comes off the configured public origin now. - sanitizeLandingPath took anything that started with a slash, including from the cookie, so a hand-written one could put newlines and markup into a column an admin table may render one day. Also: the paid-account query had no LIMIT and returned every active account's name and email, the growth route answered 403 where it meant 401, and the schema claimed no free text is stored when self_reported_note holds 200 characters of it. |
||
|
yusufipk
|
7ca5abd041 |
feat(analytics): record where paying customers actually came from
Adds first-party acquisition attribution and a sixteen-event funnel, written to this deployment's own database and read back on /admin/growth. Nothing is sent anywhere else, and the whole subsystem is off unless OPENFRAME_ENABLE_ANALYTICS is set, so a self-hosted instance carries the tables empty and pays nothing. The proxy gives a visitor an anonymous id and stores what brought them in two first-party cookies; signup copies that onto the account and claims the events the visitor produced before they had one, which is what joins the two halves of the funnel. Recording happens where each step actually happens rather than in the browser: an ad blocker cannot undercount landing views, and blocking rates differ by channel, so an undercounted denominator would have made GitHub traffic look like it converts better than it does. Every event carries a dedupe key on a UNIQUE column, so "recorded exactly once" is a property of the schema rather than of fifteen call sites. Subscription events are derived by comparing the row being overwritten with the row being written inside the existing Stripe sync, which makes them order-independent and replay-safe. The scoreboard reports step-to-step conversion with the denominator beside it, and splits by source over a rolling 28-day window rather than a week: at this volume a weekly per-source cell holds single digits, and a percentage computed from three visits reads exactly as confidently as one computed from three hundred. "How did you hear about us?" is asked on the first onboarding screen, not on the registration form. The number being measured is the signup conversion rate, and a question added to that form would move it. |
||
|
yusufipk
|
93e85683e9 |
fix(downloads): stop repeating an extension the name already carries
A voice comment's display name is the generated file name, extension included, so appending the extension again downloaded it as <uuid>.webm.webm. Both download paths, the single asset route and the project zip, now append only when the name does not already end in it. |
||
|
yusufipk
|
e7008c571a |
fix(voice): record the real length and write it into the file
The recording clock counted setInterval ticks, which a background tab throttles away: a recording that kept going looked frozen at 13 seconds and was saved with that length. It now reads the wall clock instead. MediaRecorder also writes WebM with no usable duration. Chrome omits the element entirely, Firefox reserves a Duration of 0.0 it never fills in, so players had no length to show and played past the end of the seek bar. lib/webm-duration.ts stamps the recorded length into Segment > Info when the recording stops, in place where the browser reserved room for it. |
||
|
yusufipk
|
a3036f1a52 |
fix(billing): select expired owners whose billing dates are null
The expired-owner filter expressed "no billing access" as
NOT: buildBillingAccessWhereInput(now). Prisma renders that as
NOT (status IN ('ACTIVE','TRIALING') OR "trialEndsAt" > $1 OR
"stripeCurrentPeriodEnd" > $2), and a SQL comparison against NULL is
unknown rather than false, so for a row with both dates empty the OR is
NULL and NOT NULL is NULL: the row is never returned.
Both dates empty is exactly what a canceled subscriber looks like, since
markSubscriptionCanceledByCustomerId clears trialEndsAt and Stripe no
longer reports current_period_end on the subscription. The scheduled
cleanup therefore matched nobody at all while reporting success, and
media of owners fifteen days past their grace period stayed in Bunny and
R2 indefinitely.
Each branch now names NULL explicitly. Disabling Stripe also selects
nobody instead of falling through to NOT {}, which Prisma drops
entirely: that left a filter keyed on the grace period alone, so a
self-hosted deployment running the cleanup would delete workspaces of
users it never charged.
The unit tests could not catch this, because as an object the old filter
reads correctly and no SQL is produced. The new coverage lives in
tests/api and runs against Postgres.
|
||
|
yusufipk
|
b51e690062 |
fix: close the findings the test suite surfaced
The suite that landed in #43/#44 was written against existing behaviour, so a number of tests pinned bugs rather than asserting correct behaviour. This fixes the production code and moves each of those tests onto the fixed behaviour in the same change. Security: - project-download: derive the archive entry extension from the last path segment and restrict it to a short alphanumeric run, so an extensionless allowlisted url can no longer contribute a path separator; validate the r2 branch against the strict proxy-path pattern instead of a `startsWith`, which let `/api/upload/video/clip.mp4/../../etc/passwd` through verbatim. - rate-limit: hash a key or action wider than its column instead of skipping the query. Both the guard and the failing INSERT used to answer "allowed", so the limit stopped applying entirely. Warn at startup when TRUSTED_PROXY_MODE is unset in production. - video uploads: the file name decides the content type; a client-declared video mime no longer makes `payload.exe` acceptable. - email templates: escape in the helpers rather than relying on every caller, with an explicit `rawEmailHtml()` opt-out for the one call site that builds markup. `escapeHtml` now covers the single quote. - CSP: allow loopback object storage outside production only. - route-access: reach the billing redirect only for the workspace owner. Keying it off the owner's billing status alone made the redirect target an oracle for whose subscription had lapsed, and sent members to a page they cannot act on. - search: carry the same billing condition every other read path carries. - logger: check `err.name` as well as `err.constructor.name`, so a re-thrown, deserialised or minified Prisma error is still redacted. - upload tokens: resolve the signing secret outside the try, so a server booted without one fails loudly instead of reporting every grant as a forgery. - invitations: never downgrade an existing membership, and report a scoped invitation that points at nothing as not_found rather than accepted. - auth: resolve the workspace role for every signed-in caller, so checkProjectAccess and computeProjectAccess stop disagreeing about the owner who also owns the workspace. The `intent` option is gone with it. - r2-media-proxy: validate the object key inside the proxy so the guard travels with the function; delete the unused, unanchored `mediaUrlToR2Key`. - r2: sign the content type into presigned PUT grants. Correctness: - frame rate snapping picks the nearest standard, not the first within tolerance, so 24, 30 and 60 fps are reachable at all. - a version upload registers its Bunny cleanup as soon as bunny-init answers, so a failed tus upload no longer leaves a billed video behind. - deleting videos clears storage before the rows, so a refused DELETE leaves a retryable row rather than an orphaned object. - an expired upload session can be cancelled, which is what releases its quota. - `voice/` joins the delete allowlist, so a voice note can be removed by the module that wrote it. - a failed CORS write propagates instead of being mistaken for an empty config and replacing the bucket's rules. - filtering projects by workspace no longer hides projects the unfiltered call returns. - upload retries skip aborts and permanent 4xx; progress no longer divides by zero. - reply edits no longer clear the comment's tag; optimistic resolve rolls back to the state it replaced; the delete snapshot is captured once. - assorted UI fixes: duplicate React keys, double-click guards reading stale closures, the tag list fetched twice per load, a failed member list rendering as an empty one, a stale "Initializing upload..." beside a failure, and a registration banner pointing at an email that never arrives. Consistency and access: - the two download routes answer 404 for an id belonging to another tenant, as the comment export route already did. A caller who does belong still gets 403. - accessible names for the share-link password field, the guest name gates, the version dialog inputs and the comment-tag controls. Repository health: - the runner image installs production dependencies only. - a setup file for the unit project restores stubbed env centrally. - native tsconfig path resolution replaces vite-tsconfig-paths. - `uploadBytesWithProgress` exists once. - admin stats bill Bunny storage to the workspace owner like every other quota, gate on the configured flag, wire up the single-flight guard and count the statuses that belonged to no bucket. - `r2Client.destroy()` releases the presign client too. - `prepare` tolerates a production install, where husky is absent. |
||
|
yusufipk
|
6136817f75 |
fix(test): keep the suites off a developer's real database
bun loads a plain `.env` into process.env before anything runs, and tests/helpers/env.ts read that as a deliberate export, so it beat `.env.test` outright. `scripts/test.sh api` therefore pointed the api suites at whatever deployment `.env` describes: `prisma db push --accept-data-loss` for the schema, then a truncate of every table between tests. The e2e half was worse, because playwright.config.ts built and started the app with that DATABASE_URL and those R2 credentials, then wrote fixtures into it. CI never saw any of this: a runner has no `.env`. Three changes, in order of what each one catches: - helpers/dev-env.ts drops the values bun copied out of a development env file, leaving `.env.test` to fill them. Only values that match the file character-for-character go, so a real export still wins and the per-suite databases of a parallel api run keep working. - helpers/test-database.ts refuses a DATABASE_URL whose database name is not marked as a test one, at the single point every path into the setup passes through. This is the backstop, not the fix. - playwright.config.ts blanks the variables a development env file defines and the config does not. Dropping them from process.env is not enough there: `next build` and `next start` run @next/env themselves and read the files again. That is also why a local e2e run could not build at all (a set DISABLE_RATE_LIMIT throws in lib/rate-limit.ts under NODE_ENV=production) and why auth.spec.ts failed on a machine with SMTP configured. `scripts/test.sh` now creates `.env.test` from the committed example instead of asking for a one-line copy, so the guard above is something nobody has to meet. |
||
|
yusufipk
|
0187db5dc7 |
test: close the coverage gaps the first round left
Second pass over the suite, driven by the inventory in the gaps document. Nine agents wrote suites in parallel against private databases, then a tenth read all of it adversarially and five of its findings were fixed. unit + component 2076 -> 2079 (+888 over the round) api 647 -> 1015 e2e 18 -> 29 What was closed: - lib/route-access.ts, the page-level authorization layer, went from zero tests to 48. Every API route was guarded and none of the pages were. - The five media proxy routes now have a real 2xx beside every 403. The blocker was the positive control, solved by stubbing r2Client.send() and leaving lib/r2-media-proxy.ts itself real. - Every remaining server-side lib module: invitations, email verification, the upload tokens, the logger, request origin, the whole R2 and Bunny lifecycle, notifications and admin stats. - Six video-page hooks, and the chunking arithmetic extracted out of lib/client/r2-video-upload.ts as a pure module. - Five end-to-end flows: workspace members, bulk operations, the admin area, player interaction and failure recovery. Three things about the harness itself turned out to be wrong: - Two @/lib/r2 stubs in tests/setup/api.ts had the wrong return shape, so every route reaching finalizeR2VideoUpload silently took the "not a valid video" branch and no test noticed. - The auth matrix asserted only "not 2xx", which two entries satisfied without their guard existing. It now requires 401 or 403, which makes both load-bearing, and all 60 routes pass the stricter form. - Both admin API routes had no positive control anywhere: replacing their guard with an unconditional refusal left the entire suite green. Found by the adversarial review, now covered. Process: - bun run test:mutation runs StrykerJS over the authorization and validation modules. Diagnostic, not a gate, weekly in CI rather than on a push. - playwright.config.ts gains an opt-in webkit project for the player spec. - AGENTS.md now requires a batch of new tests to be reviewed by somebody who did not write them. Only two production files change, both deliberate: lib/auth.ts loses a verbatim copy of its own permission formulas, and lib/client/r2-video-upload.ts calls the extracted arithmetic. No behaviour change in either. |
||
|
yusufipk
|
1d099c68f2 |
test: add unit, API, component and end-to-end test suites
The repo had no automated tests. Every change was verified by hand. Adds four layers, 2023 tests in total, runnable with one command: - 1191 unit tests over the pure logic in lib/, including the full computeProjectAccess permission matrix and the billing gate - 167 component and hook tests in jsdom, covering the hooks that hold real logic rather than presentational wrappers - 647 API integration tests against a real Postgres, with only auth() mocked, including a data-driven sweep asserting that none of the 60 route modules answers 2xx to an unauthenticated caller - 18 Playwright specs driving a real browser against a real build Infrastructure: vitest.config.ts with three projects, a disposable Postgres and MinIO in docker-compose.test.yml, factories and helpers under tests/, scripts/test.sh as the single entry point, a pre-push hook running bun run verify, and CI split into check, test and e2e jobs. The test database is built with prisma db push plus a replay of the hand-written SQL, because prisma migrate deploy cannot build this schema from empty: the migration history has no captured baseline. This mirrors what scripts/docker-db-bootstrap.ts already does in production, and tests/setup/db-global.ts carries a drift guard so a new migration fails the run until someone reviews it. Production code is unchanged apart from one pure-function extraction out of use-video-player.ts, which was too large to test in jsdom. Several tests pin behaviour that looks wrong, each marked KNOWN BUG in place. TESTING.md section 12 records where the plan turned out to be wrong, and AGENTS.md now states which layer a change needs a test in. |