feat: Postgres in place: a database per stack, migrations, backup and Drizzle Gateway #506
No reviewers
Labels
No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
spec
wayfinder:grilling
wayfinder:map
wayfinder:prototype
wayfinder:research
wayfinder:task
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
amaurycoudr/chess-project!506
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "claude/implement-spec-354-102e80"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Every stack (production and each preview) runs its own private Postgres 18, migrated by a one-shot step before the app is replaced. Production is backed up nightly; a Drizzle Gateway on the VPS browses every stack through an SSH tunnel. Empty schema, no table: nothing a Player sees changes.
Evidence
main: checks, front-test, e2e (built API on apostgres:18.4service), build, review (preview with its Postgres): https://gitlab.core-project.space/amaurycoudr/chess-project/-/pipelines/40.github/workflows/edits are dropped (GitHub Actions are off); their GitLab equivalents are the e2e service, the production job's password check andbackup-production. Our ADR became 0038, sincemaintook 0036 and 0037. API test start-up got a 60 s hook budget after PGlite starts timed out under CI load (pipeline 12).Earlier evidence, from GitHub Actions:
test:slow35/35 locally on the dev and built targets against a real migrated Postgres.pnpm test; an unreachable db is refused namingDATABASE_URL.pnpm devside by side used two databases on the one container; the migrate step against a missing db namespnpm db; a stopped server leaves 0 connections. CI's browser-suite job gets a Postgres service container.deploy/lib.sh, not the preview overlay, since a Compose secret cannot hold a literal.reviewjob has since deployed the Gateway-aware stack green): Gateway on the stack's network afterup.sh; thegatewayrole reads a row, cannot create a table (permission denied for schema public) and logs in with its password (a wrong one fails the check); ensuring the role twice changes nothing; Gateway stopped →up.shexits 0 with a warning; removal with the Gateway attached → verify-gone passes; port on127.0.0.1:4983only, "check it matches" reports a match. PGlite tests cover read/write, refusal of create/alter/drop, and idempotence..agefile is uploaded (if-no-files-found: error, 30 days).ghcr.io/drizzle-team/gateway:1.6.2@sha256:db9c50c7…): no licence file in the image (labelNOASSERTION, inherited from its base) and no licence key. Drizzle's Studio page calls it "a free self-hosted Drizzle Studio", formerly paid, with no stated usage restriction; it is closed source. Approved by the owner for this use.down.sh+verify-gone.shwith the close, which is proven).Merge Danger
Door: one-way, narrowly.
The first production deploy creates production's database volume and fixes its password from
DATABASE_PASSWORDat that moment. Reverting the code is agit revert; the volume and the stored password stay until removed by hand. The schema is empty, so no data is at stake yet.Blast Radius: production
The production job now refuses to run without the masked
DATABASE_PASSWORDandGATEWAY_DATABASE_PASSWORDCI/CD variables, and the API refuses to start without a reachable database. A missing secret fails the deploy, and the old app keeps serving. The nightly backup fails until the owner's age public key is committed. The backup schedule is created by hand. Every contributor needspnpm db(Docker) beforepnpm dev;pnpm testneeds no Docker.Proposed rule changes (not edited here)
apps/api/CODING_STANDARDS.md, layout sentence: "...src/main.tsboots andsrc/migrate.tsmigrates,src/app.tswithsrc/app.module.tsassembles the app,src/config/reads the environment,src/common/holds what every feature shares, among it the framework-free database modulecommon/database.ts(connect, migrate, ensure the Gateway role) with the schema beside it,drizzle/holds the committed migrations, andsrc/room/is the one feature module..."apps/api/CODING_STANDARDS.md, rule 6, second half: "...the wire through the real app in process withtest/rpc.fixtures.ts, whose fetch hands each request to the Express instance, no server or port, and whose app runs on an in-process database fromtest/database.fixtures.ts, one per test file."Owner checks
deploy/owner-setup.shstage 1:DATABASE_PASSWORDas a masked, protected GitLab variable (the old GitHub secret is gone; production never used it).deploy/backup/backup-recipient.txton this branch.GATEWAY_DATABASE_PASSWORDas a masked, protected GitLab variable.chess-db-1,chess-pr-<n>-db-1), usergateway, databasechess.main,41 2 * * *UTC, variableBACKUP=1.mainwithBACKUP=1, green, recorded on #354.Closes #354
Closes #367
Closes #368
Closes #369
Closes #370
Closes #371
added 59 commits
mainafd8ffa4- docs(repo): say how the Drizzle Gateway reaches every stack987348ad- refactor(api): give withPasswordFile one object argument2bd18282- fix(api): let the migrate entry point run without PORT9376b328- refactor(api): name the Gateway role once in ensureGatewayRole34c0aee7- refactor(api): read the migrations folder from the drizzle config in the database test4fdb5fac- refactor(ci): move the backup and Gateway files under the paths ADR 0032 governs5d91078a- refactor(ci): point the deploy comments at ADR 0032 rather than paraphrase it7f9d186f- refactor(repo): read the dev Postgres port from compose.dev.yaml alone78cf2928- refactor(ci): share the deploy checks' helpers through lib.sh and split verify.sh by concernc5979e38- fix(ci): check the Gateway role's password over the stack network, where it is requiredCompare with previous version
added 5 commits
mainc7ced72f- Merge remote-tracking branch 'origin/main' into feat/354-postgres-gitlabb805c84c- ci: drop the GitHub workflow edits, which GitLab no longer runs44b917f1- ci: run the database's CI on GitLab: e2e Postgres, production passwords, backupCompare with previous version
marked this merge request as ready
added 43 commits
397d67e4- refactor(api): name the Gateway role once in ensureGatewayRole50bf270e- refactor(api): read the migrations folder from the drizzle config in the database testfc91d58f- refactor(ci): move the backup and Gateway files under the paths ADR 0032 governs3c6480d6- refactor(ci): point the deploy comments at ADR 0032 rather than paraphrase itc49cb221- refactor(repo): read the dev Postgres port from compose.dev.yaml alone8dc9f9bc- refactor(ci): share the deploy checks' helpers through lib.sh and split verify.sh by concern1b2e9fb8- fix(ci): check the Gateway role's password over the stack network, where it is requireddc36b322- Merge remote-tracking branch 'origin/main' into feat/354-postgres-gitlab6538ba77- ci: drop the GitHub workflow edits, which GitLab no longer runs69809f69- ci: run the database's CI on GitLab: e2e Postgres, production passwords, backupCompare with previous version
mentioned in merge request !510
mentioned in merge request !511
added 1 commit
91e37585- test(api): start each file's in-process Postgres in a hook with a budget of its ownCompare with previous version
changed the description
added 1 commit
04c481c9- chore(ci): commit the owner's age public key for the nightly backupCompare with previous version
marked the checklist item Stage 2: backup key pair; commit deploy/backup/backup-recipient.txt on this branch. as completed
marked the checklist item Stage 4: Gateway applied on the VPS; add connections in its UI by container name
chess\-db\-1, chess\-pr\-\<n\>\-db\-1, user gateway, database chess. as completedmarked the checklist item deploy/owner-setup.sh stage 1: DATABASE_PASSWORD as a masked, protected GitLab variable (the old GitHub secret is gone; production never used it). as completed
marked the checklist item Stage 3: GATEWAY_DATABASE_PASSWORD as a masked, protected GitLab variable. as completed
marked the checklist item Create a pipeline schedule on main, 41 2 * * * UTC, variable BACKUP=1. as completed
added 51 commits
main9dc7bd33- refactor(api): read the migrations folder from the drizzle config in the database test7be273fb- refactor(ci): move the backup and Gateway files under the paths ADR 0032 governs6ccd6822- refactor(ci): point the deploy comments at ADR 0032 rather than paraphrase itca387364- refactor(repo): read the dev Postgres port from compose.dev.yaml alonec861d8a7- refactor(ci): share the deploy checks' helpers through lib.sh and split verify.sh by concern8eb8aaac- fix(ci): check the Gateway role's password over the stack network, where it is required7a26f698- ci: drop the GitHub workflow edits, which GitLab no longer runsb70c205c- ci: run the database's CI on GitLab: e2e Postgres, production passwords, backup3ed9b8bc- test(api): start each file's in-process Postgres in a hook with a budget of its ownfc5f6597- chore(ci): commit the owner's age public key for the nightly backupCompare with previous version
changed the description