goyomi/docs/phase2-database.md

Phase 2 — Database Layer

Persistent storage using PostgreSQL via pgx. Migrations run on startup. sqlc generates type-safe query code.

Reference schema modeled after Suwayomi-Server:

• /Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/model/table/
• Migrations: /Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/server/database/migration/

---

2.1 Schema Migration — internal/db/migrations/000001_init.up.sql

• sources table
  • id BIGINT PRIMARY KEY
  • name VARCHAR(128) NOT NULL
  • lang VARCHAR(32) NOT NULL
  • is_nsfw BOOLEAN NOT NULL DEFAULT FALSE
• manga table
  • id SERIAL PRIMARY KEY
  • source_id BIGINT NOT NULL REFERENCES sources(id)
  • url VARCHAR(2048) NOT NULL
  • title VARCHAR(512) NOT NULL
  • initialized BOOLEAN NOT NULL DEFAULT FALSE
  • artist TEXT, author TEXT, description TEXT, genre TEXT
  • status INTEGER NOT NULL DEFAULT 0
  • thumbnail_url VARCHAR(2048)
  • thumbnail_last_fetched BIGINT NOT NULL DEFAULT 0
  • in_library BOOLEAN NOT NULL DEFAULT FALSE
  • in_library_at BIGINT NOT NULL DEFAULT 0
  • real_url VARCHAR(2048)
  • last_fetched_at BIGINT NOT NULL DEFAULT 0
  • chapters_last_fetched_at BIGINT NOT NULL DEFAULT 0
  • update_strategy VARCHAR(64) NOT NULL DEFAULT 'ALWAYS_UPDATE'
  • UNIQUE (source_id, url)
• chapters table
  • id SERIAL PRIMARY KEY
  • manga_id INTEGER NOT NULL REFERENCES manga(id) ON DELETE CASCADE
  • url VARCHAR(2048) NOT NULL
  • name VARCHAR(512) NOT NULL
  • date_upload BIGINT NOT NULL DEFAULT 0
  • chapter_number REAL NOT NULL DEFAULT -1
  • scanlator VARCHAR(256)
  • source_order INTEGER NOT NULL
  • is_read BOOLEAN NOT NULL DEFAULT FALSE
  • is_bookmarked BOOLEAN NOT NULL DEFAULT FALSE
  • last_page_read INTEGER NOT NULL DEFAULT 0
  • last_read_at BIGINT NOT NULL DEFAULT 0
  • fetched_at BIGINT NOT NULL DEFAULT 0
  • real_url VARCHAR(2048)
  • is_downloaded BOOLEAN NOT NULL DEFAULT FALSE
  • page_count INTEGER NOT NULL DEFAULT -1
  • UNIQUE (manga_id, url)
• pages table
  • id SERIAL PRIMARY KEY
  • chapter_id INTEGER NOT NULL REFERENCES chapters(id) ON DELETE CASCADE
  • "index" INTEGER NOT NULL
  • url VARCHAR(2048) NOT NULL
  • image_url TEXT
  • UNIQUE (chapter_id, "index") — added for upsert support
• source_meta table
  • source_id BIGINT NOT NULL REFERENCES sources(id)
  • key VARCHAR(256) NOT NULL
  • value TEXT NOT NULL
  • PRIMARY KEY (source_id, key)
• Indexes
  • CREATE INDEX ON manga (source_id)
  • CREATE INDEX ON manga (last_fetched_at)
  • CREATE INDEX ON chapters (manga_id)
  • CREATE INDEX ON pages (chapter_id)

---

2.2 DB Initialization — internal/db/db.go

• pgxpool.Pool init from DATABASE_URL env var
  • Configurable MaxConns via DB_MAX_CONNS (default 10)
  • Configurable MinConns via DB_MIN_CONNS (default 2)
  • Connection health check on startup (pool.Ping)
• Migration runner using golang-migrate/migrate
  • Source: iofs (embed migration SQL files with //go:embed)
  • Driver: pgx5 (DSN scheme rewritten from postgres:// → pgx5://)
  • Run migrate.Up() on startup; log version before/after
  • Non-fatal on "no change" (migrate.ErrNoChange)
• Queries struct wrapping sqlc-generated query clients
• Close() method to drain pool on shutdown

---

2.3 SQL Queries — internal/db/queries/

Generated by sqlc generate (config: sqlc.yaml, sql_package: pgx/v5).

manga.sql

• UpsertManga
• GetMangaBySourceURL
• GetMangaByID
• ListMangaBySource
• UpdateMangaDetails
• UpdateMangaFetchedAt
• UpdateChaptersFetchedAt

chapter.sql

• UpsertChapter
• GetChapterByID
• ListChaptersByManga
• UpdateChapterFetchedAt

page.sql

• UpsertPage
• ListPagesByChapter
• UpdatePageImageURL

source.sql

• UpsertSource
• ListSources
• GetSourceByID
• GetSourceMeta
• SetSourceMeta

---

2.4 sqlc Configuration — sqlc.yaml

• version: "2"
• engine: postgresql
• schema path: internal/db/migrations/
• queries path: internal/db/queries/*.sql
• output package: internal/db/queries
• sql_package: "pgx/v5" — required for pgxpool compatibility
• emit_json_tags: true
• emit_db_tags: true
• emit_pointers_for_null_types: true

---

2.5 Data Flow & Cache Logic

TTL env vars implemented in internal/config/config.go. Cache check logic lives in the API handler (Phase 5) using these values.

• MANGA_LIST_TTL_SECONDS env var (default 600)
• MANGA_DETAIL_TTL_SECONDS env var (default 3600)
• CHAPTER_LIST_TTL_SECONDS env var (default 600)
• ?refresh=true query param bypass — implemented in Phase 5 API handler
• Per-flow cache check + upsert logic — implemented in Phase 5 API handler

---

Checklist: Phase 2 Done When

• golang-migrate runs 000001_init.up.sql on a fresh DB without error (manual test with running postgres)
• sqlc generate completes without errors; generated files compile
• UpsertManga + GetMangaBySourceURL round-trip test passes (requires running postgres)
• UpsertChapter correctly sets source_order from slice position (requires running postgres)
• TTL cache logic returns DB rows on second call (Phase 5)
• ?refresh=true bypasses TTL (Phase 5)
• All tables visible in psql after API calls (Phase 5)