feat: initial Phase 1 implementation — core framework + Docker
- Data types (SManga, SChapter, Page, MangasPage, all Filter variants) - Source interfaces (Source, CatalogueSource) with MD5-based ID generation matching Tachiyomi/Suwayomi - HTTP client with per-host rate limiting, cookie jar, and 429 retry - FlareSolverr v1 client (FLARESOLVERR_URL env) - Generic GraphQL POST helper - goquery HTML parser wrappers - Source registry (panics on duplicate ID) - Multi-stage Dockerfile (golang:1.26-alpine + distroless) and compose.yml (postgres, flaresolverr, app)
This commit is contained in:
achmad
@@ -0,0 +1,180 @@
|
||||
# Phase 5 — HTTP API (REST)
|
||||
|
||||
Plain REST/JSON API over HTTP using `github.com/go-chi/chi/v5`. No GraphQL on our end.
|
||||
All handlers live in `api/handler.go`. Server bootstrap in `cmd/server/main.go`.
|
||||
|
||||
Reference:
|
||||
- Suwayomi API patterns: `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/api/`
|
||||
|
||||
---
|
||||
|
||||
## 5.1 Server Bootstrap — `cmd/server/main.go`
|
||||
|
||||
- [ ] Parse config from env vars:
|
||||
- [ ] `DATABASE_URL` (required)
|
||||
- [ ] `PORT` (default `8080`)
|
||||
- [ ] `FLARESOLVERR_URL` (optional; disables CF-dependent sources if empty)
|
||||
- [ ] TTL vars: `MANGA_LIST_TTL_SECONDS`, `MANGA_DETAIL_TTL_SECONDS`, `CHAPTER_LIST_TTL_SECONDS`
|
||||
- [ ] Initialize `pgxpool.Pool` via `db.New()`
|
||||
- [ ] Run DB migrations via `db.Migrate()`
|
||||
- [ ] Blank-import all source packages so their `init()` registers them:
|
||||
```go
|
||||
import (
|
||||
_ "tachiyomi-go/sources/all/mangadex"
|
||||
_ "tachiyomi-go/sources/all/kemono"
|
||||
_ "tachiyomi-go/sources/en/allanime"
|
||||
// ... all sources
|
||||
)
|
||||
```
|
||||
- [ ] `UpsertSource` for every registered source on startup (sync source table with compiled sources)
|
||||
- [ ] Mount router and `http.ListenAndServe`
|
||||
- [ ] Graceful shutdown on SIGINT/SIGTERM (drain pool, close server)
|
||||
|
||||
---
|
||||
|
||||
## 5.2 Router — `api/handler.go`
|
||||
|
||||
- [ ] `NewRouter(db *pgxpool.Pool) http.Handler` using chi
|
||||
- [ ] Mount all routes under `/api`
|
||||
- [ ] Middleware:
|
||||
- [ ] `chi.Logger` — request logging
|
||||
- [ ] `chi.Recoverer` — panic recovery → 500 JSON error
|
||||
- [ ] `requestID` middleware
|
||||
- [ ] CORS headers (configurable origin, default `*`)
|
||||
|
||||
---
|
||||
|
||||
## 5.3 Source Routes
|
||||
|
||||
### `GET /api/sources`
|
||||
- [ ] Call `registry.All()`
|
||||
- [ ] Return `[]{id, name, lang, supportsLatest, isNsfw}` sorted by ID
|
||||
- [ ] No DB call required (registry is in-memory)
|
||||
|
||||
### `GET /api/sources/{id}/popular?page=1`
|
||||
- [ ] Parse `id` as int64, `page` as int (default 1), `refresh` as bool
|
||||
- [ ] Validate source exists in registry
|
||||
- [ ] Check manga list TTL (see Phase 2 cache logic)
|
||||
- [ ] If cache miss or `?refresh=true`: call `source.GetPopularManga(page)`, upsert results
|
||||
- [ ] Return `{mangas: [...], hasNextPage: bool}`
|
||||
|
||||
### `GET /api/sources/{id}/latest?page=1`
|
||||
- [ ] Same as popular but calls `source.GetLatestUpdates(page)`
|
||||
- [ ] Return 405 if `source.SupportsLatest()` is false
|
||||
|
||||
### `GET /api/sources/{id}/search?q=&page=1`
|
||||
- [ ] Parse `q` (query string), `page`, `refresh`
|
||||
- [ ] Pass `filters` from query params (see Filter parsing below)
|
||||
- [ ] Call `source.GetSearchManga(page, q, filters)`
|
||||
- [ ] Upsert results, return `{mangas: [...], hasNextPage: bool}`
|
||||
|
||||
### `GET /api/sources/{id}/filters`
|
||||
- [ ] Call `source.GetFilterList()`
|
||||
- [ ] Return filter descriptors as JSON: `[{type, name, values}]`
|
||||
- [ ] No DB involvement
|
||||
|
||||
### Filter Param Parsing
|
||||
- [ ] Parse filter values from query params: `filter_{name}={value}`
|
||||
- [ ] `SelectFilter`: single string value
|
||||
- [ ] `CheckboxFilter`: "true"/"false" → bool
|
||||
- [ ] `TriStateFilter`: "0"/"1"/"2" → int
|
||||
- [ ] `TextFilter`: raw string
|
||||
- [ ] `SortFilter`: `{name}:{asc|desc}`
|
||||
|
||||
---
|
||||
|
||||
## 5.4 Manga Routes (Source-aware)
|
||||
|
||||
### `GET /api/sources/{id}/manga?url={encodedURL}`
|
||||
- [ ] URL-decode `url` param
|
||||
- [ ] Look up manga in DB by `(source_id, url)` — `GetMangaBySourceURL`
|
||||
- [ ] If not found or `initialized=false` or TTL expired or `?refresh=true`:
|
||||
- [ ] Call `source.GetMangaDetails(SManga{URL: url})`
|
||||
- [ ] `UpsertManga` + `UpdateMangaDetails` + set `initialized=true`
|
||||
- [ ] Return full manga JSON (all fields)
|
||||
|
||||
### `GET /api/sources/{id}/manga?url={encodedURL}&resource=chapters`
|
||||
OR `GET /api/sources/{id}/chapters?url={encodedURL}`
|
||||
- [ ] Look up manga_id from DB
|
||||
- [ ] Check `chapters_last_fetched_at` TTL
|
||||
- [ ] If miss: call `source.GetChapterList(manga)`, `UpsertChapter` for each
|
||||
- [ ] Return `[{url, name, chapterNumber, dateUpload, scanlator, sourceOrder}]`
|
||||
|
||||
### `GET /api/sources/{id}/pages?chapterUrl={encodedURL}`
|
||||
- [ ] Look up chapter by URL
|
||||
- [ ] Check if pages exist in DB for chapter
|
||||
- [ ] If miss: call `source.GetPageList(chapter)`, then optionally `source.GetImageURL(page)` for each
|
||||
- [ ] `UpsertPage` for each; store `image_url` (except sources with expiring URLs)
|
||||
- [ ] Return `[{index, url, imageUrl}]`
|
||||
|
||||
---
|
||||
|
||||
## 5.5 DB-Backed Routes (no source call)
|
||||
|
||||
### `GET /api/manga/{id}`
|
||||
- [ ] `GetMangaByID` from DB
|
||||
- [ ] Return manga JSON or 404
|
||||
|
||||
### `GET /api/manga/{id}/chapters`
|
||||
- [ ] `ListChaptersByManga` from DB
|
||||
- [ ] Return chapter array ordered by `source_order`
|
||||
|
||||
### `GET /api/chapters/{id}`
|
||||
- [ ] `GetChapterByID` from DB
|
||||
- [ ] Return chapter JSON or 404
|
||||
|
||||
### `GET /api/chapters/{id}/pages`
|
||||
- [ ] `ListPagesByChapter` from DB
|
||||
- [ ] Return page array ordered by `index`
|
||||
|
||||
---
|
||||
|
||||
## 5.6 Image Proxy — `GET /api/image?url={encodedURL}&source_id={id}`
|
||||
|
||||
- [ ] URL-decode `url` param
|
||||
- [ ] Validate `source_id` exists in registry
|
||||
- [ ] Fetch image using source's configured HTTP client (respects source-specific headers/cookies)
|
||||
- [ ] Set `Referer` if required (e.g. Pixiv)
|
||||
- [ ] Use FlareSolverr-cleared cookies if source needs CF bypass
|
||||
- [ ] Stream response bytes directly to client
|
||||
- [ ] Forward `Content-Type` header from upstream
|
||||
- [ ] Set `Cache-Control: public, max-age=86400` for non-expiring images
|
||||
- [ ] **Do not** set long cache for signed/expiring URLs (globalcomix, etc.)
|
||||
- [ ] Return 502 on upstream error
|
||||
|
||||
---
|
||||
|
||||
## 5.7 Error Handling
|
||||
|
||||
- [ ] All errors: `{"error": "human-readable message"}` with correct HTTP status
|
||||
- [ ] 400: invalid params (bad ID, missing required param, bad page number)
|
||||
- [ ] 404: source not found, manga not found, chapter not found
|
||||
- [ ] 405: method not supported (e.g. `GetLatestUpdates` on source with `SupportsLatest=false`)
|
||||
- [ ] 500: DB error, unexpected source error
|
||||
- [ ] 502: upstream source returned error (connection refused, non-200, parse failure)
|
||||
- [ ] 503: FlareSolverr unavailable (when source requires it but `FLARESOLVERR_URL` is unset)
|
||||
- [ ] Structured error logging: include `source_id`, `url`, `error` in log fields
|
||||
|
||||
---
|
||||
|
||||
## 5.8 `?refresh=true` Bypass
|
||||
|
||||
- [ ] Accepted on all source-facing routes: `/popular`, `/latest`, `/search`, `/manga`, `/chapters`, `/pages`
|
||||
- [ ] Bypasses TTL — always calls source regardless of `last_fetched_at`
|
||||
- [ ] Updates DB with fresh data after fetch
|
||||
- [ ] NOT accepted on DB-only routes (`/api/manga/{id}`, `/api/chapters/{id}`, etc.)
|
||||
|
||||
---
|
||||
|
||||
## Checklist: Phase 5 Done When
|
||||
|
||||
- [ ] `GET /api/sources` returns all registered sources
|
||||
- [ ] `GET /api/sources/{heancmsID}/popular?page=1` returns ≥1 manga; rows in `manga` table confirmed via `psql`
|
||||
- [ ] Second call to same URL returns from DB (no source HTTP call — confirm via log)
|
||||
- [ ] `?refresh=true` forces re-fetch and updates `last_fetched_at` in DB
|
||||
- [ ] `GET /api/sources/{madaraID}/pages?chapterUrl=...` resolves image URLs via FlareSolverr
|
||||
- [ ] `GET /api/image?url=...&source_id=...` proxies image bytes with correct `Content-Type`
|
||||
- [ ] 404 response for unknown source ID: `{"error": "source not found"}`
|
||||
- [ ] 405 response when `GetLatestUpdates` called on source with `SupportsLatest=false`
|
||||
- [ ] `go build ./cmd/server/` succeeds
|
||||
- [ ] Server starts, logs migration version, logs registered source count
|
||||
Reference in New Issue
Block a user