# Phase 2 — Database Layer

Persistent storage using PostgreSQL via pgx. Migrations run on startup. sqlc generates type-safe query code.

Reference schema modeled after Suwayomi-Server:
- `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/model/table/`
- Migrations: `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/server/database/migration/`

---

## 2.1 Schema Migration — `internal/db/migrations/001_init.sql`

- [ ] `sources` table
  - [ ] `id BIGINT PRIMARY KEY` — generated via `GenerateSourceID(name, lang)` same as Tachiyomi
  - [ ] `name VARCHAR(128) NOT NULL`
  - [ ] `lang VARCHAR(32) NOT NULL`
  - [ ] `is_nsfw BOOLEAN NOT NULL DEFAULT FALSE`
- [ ] `manga` table
  - [ ] `id SERIAL PRIMARY KEY`
  - [ ] `source_id BIGINT NOT NULL REFERENCES sources(id)`
  - [ ] `url VARCHAR(2048) NOT NULL`
  - [ ] `title VARCHAR(512) NOT NULL`
  - [ ] `initialized BOOLEAN NOT NULL DEFAULT FALSE`
  - [ ] `artist TEXT`, `author TEXT`, `description TEXT`, `genre TEXT`
  - [ ] `status INTEGER NOT NULL DEFAULT 0`
  - [ ] `thumbnail_url VARCHAR(2048)`
  - [ ] `thumbnail_last_fetched BIGINT NOT NULL DEFAULT 0`
  - [ ] `in_library BOOLEAN NOT NULL DEFAULT FALSE`
  - [ ] `in_library_at BIGINT NOT NULL DEFAULT 0`
  - [ ] `real_url VARCHAR(2048)`
  - [ ] `last_fetched_at BIGINT NOT NULL DEFAULT 0`
  - [ ] `chapters_last_fetched_at BIGINT NOT NULL DEFAULT 0`
  - [ ] `update_strategy VARCHAR(64) NOT NULL DEFAULT 'ALWAYS_UPDATE'`
  - [ ] `UNIQUE (source_id, url)`
- [ ] `chapters` table
  - [ ] `id SERIAL PRIMARY KEY`
  - [ ] `manga_id INTEGER NOT NULL REFERENCES manga(id) ON DELETE CASCADE`
  - [ ] `url VARCHAR(2048) NOT NULL`
  - [ ] `name VARCHAR(512) NOT NULL`
  - [ ] `date_upload BIGINT NOT NULL DEFAULT 0`
  - [ ] `chapter_number REAL NOT NULL DEFAULT -1`
  - [ ] `scanlator VARCHAR(256)`
  - [ ] `source_order INTEGER NOT NULL`
  - [ ] `is_read BOOLEAN NOT NULL DEFAULT FALSE`
  - [ ] `is_bookmarked BOOLEAN NOT NULL DEFAULT FALSE`
  - [ ] `last_page_read INTEGER NOT NULL DEFAULT 0`
  - [ ] `last_read_at BIGINT NOT NULL DEFAULT 0`
  - [ ] `fetched_at BIGINT NOT NULL DEFAULT 0`
  - [ ] `real_url VARCHAR(2048)`
  - [ ] `is_downloaded BOOLEAN NOT NULL DEFAULT FALSE`
  - [ ] `page_count INTEGER NOT NULL DEFAULT -1`
  - [ ] `UNIQUE (manga_id, url)`
- [ ] `pages` table
  - [ ] `id SERIAL PRIMARY KEY`
  - [ ] `chapter_id INTEGER NOT NULL REFERENCES chapters(id) ON DELETE CASCADE`
  - [ ] `"index" INTEGER NOT NULL`
  - [ ] `url VARCHAR(2048) NOT NULL`
  - [ ] `image_url TEXT`
- [ ] `source_meta` table
  - [ ] `source_id BIGINT NOT NULL REFERENCES sources(id)`
  - [ ] `key VARCHAR(256) NOT NULL`
  - [ ] `value TEXT NOT NULL`
  - [ ] `PRIMARY KEY (source_id, key)`
- [ ] Indexes
  - [ ] `CREATE INDEX ON manga (source_id)`
  - [ ] `CREATE INDEX ON manga (last_fetched_at)`
  - [ ] `CREATE INDEX ON chapters (manga_id)`
  - [ ] `CREATE INDEX ON pages (chapter_id)`

---

## 2.2 DB Initialization — `internal/db/db.go`

- [ ] `pgxpool.Pool` init from `DATABASE_URL` env var
  - [ ] Configurable `MaxConns` (default 10)
  - [ ] Configurable `MinConns` (default 2)
  - [ ] Connection health check on startup
- [ ] Migration runner using `golang-migrate/migrate`
  - [ ] Source: `iofs` (embed migration SQL files with `//go:embed`)
  - [ ] Driver: `pgx5`
  - [ ] Run `migrate.Up()` on startup; log version before/after
  - [ ] Non-fatal on "no change" (already at latest)
- [ ] `Queries` struct wrapping sqlc-generated query clients
- [ ] `Close()` method to drain pool on shutdown

---

## 2.3 SQL Queries — `internal/db/queries/`

Write `.sql` files, then run `sqlc generate` to produce type-safe Go.

### `manga.sql`

- [ ] `UpsertManga` — INSERT ... ON CONFLICT (source_id, url) DO UPDATE all fields; returns row
- [ ] `GetMangaBySourceURL` — SELECT by source_id + url
- [ ] `GetMangaByID` — SELECT by primary key
- [ ] `ListMangaBySource` — SELECT by source_id ORDER BY last_fetched_at DESC
- [ ] `UpdateMangaDetails` — UPDATE artist/author/description/genre/status/thumbnail/initialized for a manga id
- [ ] `UpdateMangaFetchedAt` — UPDATE last_fetched_at = $now WHERE id = $id
- [ ] `UpdateChaptersFetchedAt` — UPDATE chapters_last_fetched_at WHERE id = $id

### `chapter.sql`

- [ ] `UpsertChapter` — INSERT ... ON CONFLICT (manga_id, url) DO UPDATE name/date/chapter_number/scanlator/source_order
- [ ] `GetChapterByID`
- [ ] `ListChaptersByManga` — ORDER BY source_order
- [ ] `UpdateChapterFetchedAt`

### `page.sql`

- [ ] `UpsertPage` — INSERT ... ON CONFLICT (chapter_id, index) DO UPDATE url/image_url
- [ ] `ListPagesByChapter` — ORDER BY index
- [ ] `UpdatePageImageURL` — UPDATE image_url WHERE id = $id

### `source.sql`

- [ ] `UpsertSource` — INSERT ... ON CONFLICT (id) DO UPDATE name/lang/is_nsfw
- [ ] `ListSources`
- [ ] `GetSourceByID`
- [ ] `GetSourceMeta` — SELECT value WHERE source_id = $id AND key = $key
- [ ] `SetSourceMeta` — INSERT ... ON CONFLICT DO UPDATE value

---

## 2.4 sqlc Configuration — `sqlc.yaml`

- [ ] `version: "2"`
- [ ] engine: `postgresql`
- [ ] schema path pointing to `internal/db/migrations/`
- [ ] queries path pointing to `internal/db/queries/*.sql`
- [ ] output package: `internal/db/queries`
- [ ] `emit_json_tags: true`
- [ ] `emit_db_tags: true`

---

## 2.5 Data Flow & Cache Logic

Four upsert flows — each called from the API handler before returning data.

### Manga List (`GetPopularManga` / `GetLatestUpdates` / `GetSearchManga`)

- [ ] Check `manga.last_fetched_at`: if within TTL (default 10 min) and `?refresh=false`, return DB rows
- [ ] Otherwise call source, then `UpsertManga` for each returned SManga
- [ ] Update `last_fetched_at` on upserted rows
- [ ] `UpsertSource` for the source record (idempotent on every list call)

### Manga Detail (`GetMangaDetails`)

- [ ] Check `manga.initialized`: if true and within TTL (default 1h), return DB row
- [ ] Otherwise call source, then `UpdateMangaDetails` + set `initialized=true`

### Chapter List (`GetChapterList`)

- [ ] Check `manga.chapters_last_fetched_at`: if within TTL (default 10 min), return DB rows
- [ ] Otherwise call source, then `UpsertChapter` for each SChapter (preserving `source_order` = slice index)
- [ ] Update `chapters_last_fetched_at` on the manga row

### Page List (`GetPageList`)

- [ ] Check if pages exist for chapter in DB
- [ ] Otherwise call source, then `UpsertPage` for each Page
- [ ] If source needs a second call to resolve image URLs (via `GetImageURL`), call it and `UpdatePageImageURL`
- [ ] Store resolved `image_url` — do NOT cache for sources with signed/expiring URLs (globalcomix, etc.)

### TTL Configuration

- [ ] `MANGA_LIST_TTL_SECONDS` env var (default 600)
- [ ] `MANGA_DETAIL_TTL_SECONDS` env var (default 3600)
- [ ] `CHAPTER_LIST_TTL_SECONDS` env var (default 600)
- [ ] `?refresh=true` query param bypasses TTL on any route

---

## Checklist: Phase 2 Done When

- [ ] `golang-migrate` runs `001_init.sql` on a fresh DB without error
- [ ] `sqlc generate` completes without errors; generated files compile
- [ ] `UpsertManga` + `GetMangaBySourceURL` round-trip test passes
- [ ] `UpsertChapter` correctly sets `source_order` from slice position
- [ ] TTL cache logic returns DB rows on second call (verify no source HTTP call via log)
- [ ] `?refresh=true` bypasses TTL and re-fetches from source
- [ ] All tables visible in `psql` after API calls