feat: initial Phase 1 implementation — core framework + Docker
- Data types (SManga, SChapter, Page, MangasPage, all Filter variants) - Source interfaces (Source, CatalogueSource) with MD5-based ID generation matching Tachiyomi/Suwayomi - HTTP client with per-host rate limiting, cookie jar, and 429 retry - FlareSolverr v1 client (FLARESOLVERR_URL env) - Generic GraphQL POST helper - goquery HTML parser wrappers - Source registry (panics on duplicate ID) - Multi-stage Dockerfile (golang:1.26-alpine + distroless) and compose.yml (postgres, flaresolverr, app)
This commit is contained in:
achmad
@@ -0,0 +1,170 @@
|
||||
# Phase 1 — Core Framework
|
||||
|
||||
All foundational types, interfaces, and helpers that every source implementation depends on.
|
||||
Nothing in `sources/` can be written until this phase is complete.
|
||||
|
||||
Reference:
|
||||
- Tachiyomi source contract: `/Users/achmad/Documents/Belajar/Android/extensions-source/core/src/` (HttpSource, CatalogueSource)
|
||||
- Kotlin originals: `/Users/achmad/Documents/Belajar/Android/extensions-source/lib-multisrc/madara/src/eu/kanade/tachiyomi/multisrc/madara/Madara.kt`
|
||||
- Suwayomi source interface: `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/impl/`
|
||||
|
||||
---
|
||||
|
||||
## 1.1 Data Types — `internal/source/types.go`
|
||||
|
||||
- [x] `SManga` struct
|
||||
- [x] `URL`, `Title`, `Artist`, `Author`, `Description` string fields
|
||||
- [x] `Genre` string (comma-separated, matches Tachiyomi convention)
|
||||
- [x] `Status` int — 0=unknown, 1=ongoing, 2=completed, 3=licensed, 5=hiatus, 6=cancelled
|
||||
- [x] `ThumbnailURL` string
|
||||
- [x] `Initialized` bool (false until `GetMangaDetails` has been called)
|
||||
- [x] `SChapter` struct
|
||||
- [x] `URL`, `Name`, `Scanlator` string fields
|
||||
- [x] `DateUpload` int64 — unix milliseconds (matches Tachiyomi)
|
||||
- [x] `ChapterNumber` float32
|
||||
- [x] `Page` struct
|
||||
- [x] `Index` int
|
||||
- [x] `URL` string — page HTML URL or chapter URL
|
||||
- [x] `ImageURL` string — direct image URL (may be empty until `GetImageURL` resolves it)
|
||||
- [x] `MangasPage` struct
|
||||
- [x] `Mangas []SManga`
|
||||
- [x] `HasNextPage bool`
|
||||
- [x] Filter types
|
||||
- [x] `Filter` interface with `Name()` and `Value()` methods
|
||||
- [x] `TextFilter` — free-text input
|
||||
- [x] `CheckboxFilter` — boolean
|
||||
- [x] `TriStateFilter` — ignore / include / exclude (0/1/2)
|
||||
- [x] `SelectFilter` — dropdown with named options (name + values []string)
|
||||
- [x] `SortFilter` — sortable list with ascending/descending state
|
||||
- [x] `GroupFilter` — container of sub-filters
|
||||
- [x] Status constants (`StatusUnknown`, `StatusOngoing`, etc.)
|
||||
|
||||
---
|
||||
|
||||
## 1.2 Source Interfaces — `internal/source/interfaces.go`
|
||||
|
||||
- [x] `Source` interface (base)
|
||||
- [x] `ID() int64`
|
||||
- [x] `Name() string`
|
||||
- [x] `Lang() string`
|
||||
- [x] `CatalogueSource` interface (embeds Source)
|
||||
- [x] `SupportsLatest() bool`
|
||||
- [x] `GetPopularManga(page int) (MangasPage, error)`
|
||||
- [x] `GetLatestUpdates(page int) (MangasPage, error)`
|
||||
- [x] `GetSearchManga(page int, query string, filters []Filter) (MangasPage, error)`
|
||||
- [x] `GetMangaDetails(manga SManga) (SManga, error)`
|
||||
- [x] `GetChapterList(manga SManga) ([]SChapter, error)`
|
||||
- [x] `GetPageList(chapter SChapter) ([]Page, error)`
|
||||
- [x] `GetImageURL(page Page) (string, error)` — no-op default for sources that embed image URLs directly in pages
|
||||
- [x] `GetFilterList() []Filter`
|
||||
- [x] ID generation helper
|
||||
- [x] `GenerateSourceID(name, lang string) int64` — MD5 of `"${name.lowercase()}/$lang/1"`, first 8 bytes big-endian, sign bit cleared. Matches Suwayomi `HttpSource.generateId`. (The original plan incorrectly described Java `String.hashCode()`.)
|
||||
- [x] Unit test against computed IDs (MangaDex/en, MangaDex/all, HeanCms/en)
|
||||
|
||||
---
|
||||
|
||||
## 1.3 HTTP Client — `internal/httpclient/client.go`
|
||||
|
||||
- [x] `Client` struct wrapping `*http.Client`
|
||||
- [x] Persistent `cookiejar.Jar` (one per source instance)
|
||||
- [x] Per-host `rate.Limiter` map (keyed by `Host`) with `sync.Mutex`
|
||||
- [x] Configurable `RateLimit` (requests/sec) and `Burst` per source
|
||||
- [x] Configurable `Timeout` (default 30s)
|
||||
- [x] Default `User-Agent` header (Chrome/Android UA, matches Tachiyomi)
|
||||
- [x] `NewClient(opts ...Option) *Client` constructor with functional options
|
||||
- [x] `Do(req *http.Request) (*http.Response, error)` — applies rate limit before executing
|
||||
- [x] `Get(ctx context.Context, url string, headers map[string]string) (*http.Response, error)` helper
|
||||
- [x] `Post(ctx context.Context, url string, body io.Reader, contentType string, headers map[string]string) (*http.Response, error)` helper
|
||||
- [x] 429 retry logic
|
||||
- [x] Read `Retry-After` header (seconds or HTTP-date)
|
||||
- [x] Sleep and retry up to N times (configurable, default 3)
|
||||
- [x] Optional `Referer` injection (configurable per client)
|
||||
|
||||
---
|
||||
|
||||
## 1.4 FlareSolverr — `internal/httpclient/flaresolverr.go`
|
||||
|
||||
- [x] `FlareSolverrClient` struct
|
||||
- [x] `endpoint` string (FlareSolverr v1 base URL, e.g. `http://localhost:8191`)
|
||||
- [x] Underlying `*http.Client` for talking to FlareSolverr itself
|
||||
- [x] `Get(ctx context.Context, url string, cookies []Cookie) (html string, cookies []Cookie, err error)`
|
||||
- [x] POST `{"cmd":"request.get","url":"...","maxTimeout":60000}` to `/v1`
|
||||
- [x] Extract `solution.response` (rendered HTML body)
|
||||
- [x] Extract `solution.cookies` → convert to `[]*http.Cookie`
|
||||
- [x] Cookie reuse strategy — cookies returned to caller; caller injects into source's cookie jar
|
||||
- [x] `FlareSolverrResponse` JSON struct (full response shape)
|
||||
- [x] Config: `FLARESOLVERR_URL` env var (disable if empty — `NewFlareSolverrClient` returns error)
|
||||
|
||||
---
|
||||
|
||||
## 1.5 GraphQL Helper — `internal/httpclient/graphql.go`
|
||||
|
||||
Used only to talk to upstream sources that expose GraphQL (mangahub, senkuro, allanime, luscious, stashapp).
|
||||
Our own API is plain REST.
|
||||
|
||||
- [x] `GraphQLRequest` struct — `Query string`, `Variables any`
|
||||
- [x] `Post[T any](ctx context.Context, client *http.Client, url string, req GraphQLRequest, headers map[string]string) (T, error)`
|
||||
- [x] Marshal request to JSON
|
||||
- [x] POST with `Content-Type: application/json`
|
||||
- [x] Unmarshal `data` field of response into T
|
||||
- [x] Surface `errors[]` as a Go error if present
|
||||
- [x] `graphQLResponse[T any]` envelope struct
|
||||
|
||||
---
|
||||
|
||||
## 1.6 Header Builders — `internal/httpclient/headers.go`
|
||||
|
||||
- [x] `AndroidUA() string` — Chrome/Android user-agent string (matches Tachiyomi default)
|
||||
- [x] `DesktopUA() string` — desktop Chrome user-agent (for sources requiring desktop)
|
||||
- [x] `JSONHeaders() map[string]string` — `Content-Type: application/json`, `Accept: application/json`
|
||||
- [x] `FormHeaders() map[string]string` — `Content-Type: application/x-www-form-urlencoded`
|
||||
- [x] `WithRefererHeader(headers map[string]string, referer string) map[string]string`
|
||||
- [x] `WithOrigin(headers map[string]string, origin string) map[string]string`
|
||||
|
||||
---
|
||||
|
||||
## 1.7 HTML Parser — `internal/parser/html.go`
|
||||
|
||||
Thin wrappers over `github.com/PuerkitoBio/goquery` (Go equivalent of JSoup used in Tachiyomi extensions).
|
||||
|
||||
- [x] `Parse(html string) (*goquery.Document, error)` — parse raw HTML string
|
||||
- [x] `ParseResponse(resp *http.Response) (*goquery.Document, error)` — parse from HTTP response body
|
||||
- [x] `Select(doc *goquery.Document, css string) *goquery.Selection`
|
||||
- [x] `SelectFrom(sel *goquery.Selection, css string) *goquery.Selection`
|
||||
- [x] `Attr(sel *goquery.Selection, name string) string` — returns empty string if not found
|
||||
- [x] `AbsURL(sel *goquery.Selection, attr string, baseURL string) string` — resolves relative URLs
|
||||
- [x] `OwnText(sel *goquery.Selection) string` — text of element excluding child elements
|
||||
- [x] `TextTrim(sel *goquery.Selection) string` — `.Text()` with strings.TrimSpace
|
||||
- [x] `First(sel *goquery.Selection) *goquery.Selection`
|
||||
- [x] `Each(sel *goquery.Selection, fn func(i int, s *goquery.Selection))` — convenience wrapper
|
||||
|
||||
---
|
||||
|
||||
## 1.8 Registry — `internal/registry/registry.go`
|
||||
|
||||
- [x] Package-level `map[int64]source.CatalogueSource` protected by `sync.RWMutex`
|
||||
- [x] `Register(s source.CatalogueSource)` — panics on duplicate ID (caught at startup)
|
||||
- [x] `Get(id int64) (source.CatalogueSource, bool)`
|
||||
- [x] `All() []source.CatalogueSource` — returns sorted slice (by ID) for deterministic listing
|
||||
- [x] Each source package must call `registry.Register(NewXxx())` in its `init()` function
|
||||
- [ ] All source packages blank-imported in `cmd/server/main.go` (done when sources exist)
|
||||
|
||||
---
|
||||
|
||||
## 1.9 Docker — `Dockerfile` + `compose.yml`
|
||||
|
||||
- [x] `Dockerfile` — multi-stage build (`golang:1.26-alpine` builder + `distroless/static-debian12` runtime)
|
||||
- [x] `compose.yml` — services: `app`, `postgres`, `flaresolverr`
|
||||
- [x] `postgres` — official image, healthcheck, named volume
|
||||
- [x] `flaresolverr` — ghcr.io/flaresolverr/flaresolverr, depends on nothing
|
||||
- [x] `app` — builds from `Dockerfile`, depends on postgres healthcheck, env vars wired
|
||||
|
||||
---
|
||||
|
||||
## Checklist: Phase 1 Done When
|
||||
|
||||
- [x] `go build ./internal/...` succeeds with no errors
|
||||
- [x] `GenerateSourceID` matches Tachiyomi IDs for at least 3 known sources
|
||||
- [ ] `FlareSolverrClient.Get` returns rendered HTML for a Cloudflare-protected URL (manual test — requires running FlareSolverr)
|
||||
- [ ] `GraphQLPost` works against a public GraphQL endpoint (manual test)
|
||||
- [x] Registry panics on duplicate source ID (unit test)
|
||||
@@ -0,0 +1,183 @@
|
||||
# Phase 2 — Database Layer
|
||||
|
||||
Persistent storage using PostgreSQL via pgx. Migrations run on startup. sqlc generates type-safe query code.
|
||||
|
||||
Reference schema modeled after Suwayomi-Server:
|
||||
- `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/model/table/`
|
||||
- Migrations: `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/server/database/migration/`
|
||||
|
||||
---
|
||||
|
||||
## 2.1 Schema Migration — `internal/db/migrations/001_init.sql`
|
||||
|
||||
- [ ] `sources` table
|
||||
- [ ] `id BIGINT PRIMARY KEY` — generated via `GenerateSourceID(name, lang)` same as Tachiyomi
|
||||
- [ ] `name VARCHAR(128) NOT NULL`
|
||||
- [ ] `lang VARCHAR(32) NOT NULL`
|
||||
- [ ] `is_nsfw BOOLEAN NOT NULL DEFAULT FALSE`
|
||||
- [ ] `manga` table
|
||||
- [ ] `id SERIAL PRIMARY KEY`
|
||||
- [ ] `source_id BIGINT NOT NULL REFERENCES sources(id)`
|
||||
- [ ] `url VARCHAR(2048) NOT NULL`
|
||||
- [ ] `title VARCHAR(512) NOT NULL`
|
||||
- [ ] `initialized BOOLEAN NOT NULL DEFAULT FALSE`
|
||||
- [ ] `artist TEXT`, `author TEXT`, `description TEXT`, `genre TEXT`
|
||||
- [ ] `status INTEGER NOT NULL DEFAULT 0`
|
||||
- [ ] `thumbnail_url VARCHAR(2048)`
|
||||
- [ ] `thumbnail_last_fetched BIGINT NOT NULL DEFAULT 0`
|
||||
- [ ] `in_library BOOLEAN NOT NULL DEFAULT FALSE`
|
||||
- [ ] `in_library_at BIGINT NOT NULL DEFAULT 0`
|
||||
- [ ] `real_url VARCHAR(2048)`
|
||||
- [ ] `last_fetched_at BIGINT NOT NULL DEFAULT 0`
|
||||
- [ ] `chapters_last_fetched_at BIGINT NOT NULL DEFAULT 0`
|
||||
- [ ] `update_strategy VARCHAR(64) NOT NULL DEFAULT 'ALWAYS_UPDATE'`
|
||||
- [ ] `UNIQUE (source_id, url)`
|
||||
- [ ] `chapters` table
|
||||
- [ ] `id SERIAL PRIMARY KEY`
|
||||
- [ ] `manga_id INTEGER NOT NULL REFERENCES manga(id) ON DELETE CASCADE`
|
||||
- [ ] `url VARCHAR(2048) NOT NULL`
|
||||
- [ ] `name VARCHAR(512) NOT NULL`
|
||||
- [ ] `date_upload BIGINT NOT NULL DEFAULT 0`
|
||||
- [ ] `chapter_number REAL NOT NULL DEFAULT -1`
|
||||
- [ ] `scanlator VARCHAR(256)`
|
||||
- [ ] `source_order INTEGER NOT NULL`
|
||||
- [ ] `is_read BOOLEAN NOT NULL DEFAULT FALSE`
|
||||
- [ ] `is_bookmarked BOOLEAN NOT NULL DEFAULT FALSE`
|
||||
- [ ] `last_page_read INTEGER NOT NULL DEFAULT 0`
|
||||
- [ ] `last_read_at BIGINT NOT NULL DEFAULT 0`
|
||||
- [ ] `fetched_at BIGINT NOT NULL DEFAULT 0`
|
||||
- [ ] `real_url VARCHAR(2048)`
|
||||
- [ ] `is_downloaded BOOLEAN NOT NULL DEFAULT FALSE`
|
||||
- [ ] `page_count INTEGER NOT NULL DEFAULT -1`
|
||||
- [ ] `UNIQUE (manga_id, url)`
|
||||
- [ ] `pages` table
|
||||
- [ ] `id SERIAL PRIMARY KEY`
|
||||
- [ ] `chapter_id INTEGER NOT NULL REFERENCES chapters(id) ON DELETE CASCADE`
|
||||
- [ ] `"index" INTEGER NOT NULL`
|
||||
- [ ] `url VARCHAR(2048) NOT NULL`
|
||||
- [ ] `image_url TEXT`
|
||||
- [ ] `source_meta` table
|
||||
- [ ] `source_id BIGINT NOT NULL REFERENCES sources(id)`
|
||||
- [ ] `key VARCHAR(256) NOT NULL`
|
||||
- [ ] `value TEXT NOT NULL`
|
||||
- [ ] `PRIMARY KEY (source_id, key)`
|
||||
- [ ] Indexes
|
||||
- [ ] `CREATE INDEX ON manga (source_id)`
|
||||
- [ ] `CREATE INDEX ON manga (last_fetched_at)`
|
||||
- [ ] `CREATE INDEX ON chapters (manga_id)`
|
||||
- [ ] `CREATE INDEX ON pages (chapter_id)`
|
||||
|
||||
---
|
||||
|
||||
## 2.2 DB Initialization — `internal/db/db.go`
|
||||
|
||||
- [ ] `pgxpool.Pool` init from `DATABASE_URL` env var
|
||||
- [ ] Configurable `MaxConns` (default 10)
|
||||
- [ ] Configurable `MinConns` (default 2)
|
||||
- [ ] Connection health check on startup
|
||||
- [ ] Migration runner using `golang-migrate/migrate`
|
||||
- [ ] Source: `iofs` (embed migration SQL files with `//go:embed`)
|
||||
- [ ] Driver: `pgx5`
|
||||
- [ ] Run `migrate.Up()` on startup; log version before/after
|
||||
- [ ] Non-fatal on "no change" (already at latest)
|
||||
- [ ] `Queries` struct wrapping sqlc-generated query clients
|
||||
- [ ] `Close()` method to drain pool on shutdown
|
||||
|
||||
---
|
||||
|
||||
## 2.3 SQL Queries — `internal/db/queries/`
|
||||
|
||||
Write `.sql` files, then run `sqlc generate` to produce type-safe Go.
|
||||
|
||||
### `manga.sql`
|
||||
|
||||
- [ ] `UpsertManga` — INSERT ... ON CONFLICT (source_id, url) DO UPDATE all fields; returns row
|
||||
- [ ] `GetMangaBySourceURL` — SELECT by source_id + url
|
||||
- [ ] `GetMangaByID` — SELECT by primary key
|
||||
- [ ] `ListMangaBySource` — SELECT by source_id ORDER BY last_fetched_at DESC
|
||||
- [ ] `UpdateMangaDetails` — UPDATE artist/author/description/genre/status/thumbnail/initialized for a manga id
|
||||
- [ ] `UpdateMangaFetchedAt` — UPDATE last_fetched_at = $now WHERE id = $id
|
||||
- [ ] `UpdateChaptersFetchedAt` — UPDATE chapters_last_fetched_at WHERE id = $id
|
||||
|
||||
### `chapter.sql`
|
||||
|
||||
- [ ] `UpsertChapter` — INSERT ... ON CONFLICT (manga_id, url) DO UPDATE name/date/chapter_number/scanlator/source_order
|
||||
- [ ] `GetChapterByID`
|
||||
- [ ] `ListChaptersByManga` — ORDER BY source_order
|
||||
- [ ] `UpdateChapterFetchedAt`
|
||||
|
||||
### `page.sql`
|
||||
|
||||
- [ ] `UpsertPage` — INSERT ... ON CONFLICT (chapter_id, index) DO UPDATE url/image_url
|
||||
- [ ] `ListPagesByChapter` — ORDER BY index
|
||||
- [ ] `UpdatePageImageURL` — UPDATE image_url WHERE id = $id
|
||||
|
||||
### `source.sql`
|
||||
|
||||
- [ ] `UpsertSource` — INSERT ... ON CONFLICT (id) DO UPDATE name/lang/is_nsfw
|
||||
- [ ] `ListSources`
|
||||
- [ ] `GetSourceByID`
|
||||
- [ ] `GetSourceMeta` — SELECT value WHERE source_id = $id AND key = $key
|
||||
- [ ] `SetSourceMeta` — INSERT ... ON CONFLICT DO UPDATE value
|
||||
|
||||
---
|
||||
|
||||
## 2.4 sqlc Configuration — `sqlc.yaml`
|
||||
|
||||
- [ ] `version: "2"`
|
||||
- [ ] engine: `postgresql`
|
||||
- [ ] schema path pointing to `internal/db/migrations/`
|
||||
- [ ] queries path pointing to `internal/db/queries/*.sql`
|
||||
- [ ] output package: `internal/db/queries`
|
||||
- [ ] `emit_json_tags: true`
|
||||
- [ ] `emit_db_tags: true`
|
||||
|
||||
---
|
||||
|
||||
## 2.5 Data Flow & Cache Logic
|
||||
|
||||
Four upsert flows — each called from the API handler before returning data.
|
||||
|
||||
### Manga List (`GetPopularManga` / `GetLatestUpdates` / `GetSearchManga`)
|
||||
|
||||
- [ ] Check `manga.last_fetched_at`: if within TTL (default 10 min) and `?refresh=false`, return DB rows
|
||||
- [ ] Otherwise call source, then `UpsertManga` for each returned SManga
|
||||
- [ ] Update `last_fetched_at` on upserted rows
|
||||
- [ ] `UpsertSource` for the source record (idempotent on every list call)
|
||||
|
||||
### Manga Detail (`GetMangaDetails`)
|
||||
|
||||
- [ ] Check `manga.initialized`: if true and within TTL (default 1h), return DB row
|
||||
- [ ] Otherwise call source, then `UpdateMangaDetails` + set `initialized=true`
|
||||
|
||||
### Chapter List (`GetChapterList`)
|
||||
|
||||
- [ ] Check `manga.chapters_last_fetched_at`: if within TTL (default 10 min), return DB rows
|
||||
- [ ] Otherwise call source, then `UpsertChapter` for each SChapter (preserving `source_order` = slice index)
|
||||
- [ ] Update `chapters_last_fetched_at` on the manga row
|
||||
|
||||
### Page List (`GetPageList`)
|
||||
|
||||
- [ ] Check if pages exist for chapter in DB
|
||||
- [ ] Otherwise call source, then `UpsertPage` for each Page
|
||||
- [ ] If source needs a second call to resolve image URLs (via `GetImageURL`), call it and `UpdatePageImageURL`
|
||||
- [ ] Store resolved `image_url` — do NOT cache for sources with signed/expiring URLs (globalcomix, etc.)
|
||||
|
||||
### TTL Configuration
|
||||
|
||||
- [ ] `MANGA_LIST_TTL_SECONDS` env var (default 600)
|
||||
- [ ] `MANGA_DETAIL_TTL_SECONDS` env var (default 3600)
|
||||
- [ ] `CHAPTER_LIST_TTL_SECONDS` env var (default 600)
|
||||
- [ ] `?refresh=true` query param bypasses TTL on any route
|
||||
|
||||
---
|
||||
|
||||
## Checklist: Phase 2 Done When
|
||||
|
||||
- [ ] `golang-migrate` runs `001_init.sql` on a fresh DB without error
|
||||
- [ ] `sqlc generate` completes without errors; generated files compile
|
||||
- [ ] `UpsertManga` + `GetMangaBySourceURL` round-trip test passes
|
||||
- [ ] `UpsertChapter` correctly sets `source_order` from slice position
|
||||
- [ ] TTL cache logic returns DB rows on second call (verify no source HTTP call via log)
|
||||
- [ ] `?refresh=true` bypasses TTL and re-fetches from source
|
||||
- [ ] All tables visible in `psql` after API calls
|
||||
@@ -0,0 +1,261 @@
|
||||
# Phase 3 — Base Source Implementations
|
||||
|
||||
Complete port checklist for all 68 bases in `/Users/achmad/Documents/Belajar/Android/extensions-source/lib-multisrc/`.
|
||||
Each base goes into `sources/base/{name}/`. Check a box when the base compiles and
|
||||
at least one derived source passes a smoke test.
|
||||
|
||||
Detailed implementation notes for complex bases are in the **Notes** section at the bottom.
|
||||
|
||||
---
|
||||
|
||||
## All Bases — 68 total
|
||||
|
||||
- [ ] `base/bakkin` ⚠️ see notes
|
||||
- [ ] `base/colorlibanime`
|
||||
- [ ] `base/comicaso`
|
||||
- [ ] `base/comiciviewer`
|
||||
- [ ] `base/eromuse`
|
||||
- [ ] `base/ezmanhwa`
|
||||
- [ ] `base/fansubscat`
|
||||
- [ ] `base/fmreader` ⚠️ see notes
|
||||
- [ ] `base/foolslide` ⚠️ see notes
|
||||
- [ ] `base/fuzzydoodle`
|
||||
- [ ] `base/galleryadults`
|
||||
- [ ] `base/gattsu`
|
||||
- [ ] `base/gigaviewer` ⚠️ see notes
|
||||
- [ ] `base/gmanga` ⚠️ see notes
|
||||
- [ ] `base/goda`
|
||||
- [ ] `base/gravureblogger`
|
||||
- [ ] `base/greenshit`
|
||||
- [ ] `base/grouple` ⚠️ see notes
|
||||
- [ ] `base/guya` ⚠️ see notes
|
||||
- [ ] `base/heancms` ⚠️ see notes
|
||||
- [ ] `base/hentaihand` ⚠️ see notes
|
||||
- [ ] `base/hotcomics`
|
||||
- [ ] `base/iken` ⚠️ see notes
|
||||
- [ ] `base/initmanga`
|
||||
- [ ] `base/kemono` ⚠️ see notes
|
||||
- [ ] `base/keyoapp`
|
||||
- [ ] `base/lectormoe` ⚠️ see notes
|
||||
- [ ] `base/libgroup` ⚠️ see notes
|
||||
- [ ] `base/liliana` ⚠️ see notes
|
||||
- [ ] `base/madara` ⚠️ see notes
|
||||
- [ ] `base/madtheme` ⚠️ see notes
|
||||
- [ ] `base/manga18`
|
||||
- [ ] `base/mangabox`
|
||||
- [ ] `base/mangacatalog`
|
||||
- [ ] `base/mangadventure` ⚠️ see notes
|
||||
- [ ] `base/mangahub` ⚠️ see notes
|
||||
- [ ] `base/mangareader` ⚠️ see notes
|
||||
- [ ] `base/mangataro`
|
||||
- [ ] `base/mangathemesia` ⚠️ see notes
|
||||
- [ ] `base/mangawork`
|
||||
- [ ] `base/mangaworld` ⚠️ see notes
|
||||
- [ ] `base/mangotheme` ⚠️ see notes
|
||||
- [ ] `base/manhwaz`
|
||||
- [ ] `base/masonry`
|
||||
- [ ] `base/mccms`
|
||||
- [ ] `base/mmlook` ⚠️ see notes
|
||||
- [ ] `base/mmrcms` ⚠️ see notes
|
||||
- [ ] `base/monochrome`
|
||||
- [ ] `base/multichan`
|
||||
- [ ] `base/natsuid`
|
||||
- [ ] `base/oceanwp`
|
||||
- [ ] `base/paprika`
|
||||
- [ ] `base/peachscan`
|
||||
- [ ] `base/pizzareader` ⚠️ see notes
|
||||
- [ ] `base/raijinscans`
|
||||
- [ ] `base/scanr`
|
||||
- [ ] `base/scanreader` ⚠️ see notes
|
||||
- [ ] `base/senkuro` ⚠️ see notes
|
||||
- [ ] `base/sinmh`
|
||||
- [ ] `base/spicytheme`
|
||||
- [ ] `base/stalkercms`
|
||||
- [ ] `base/uzaymanga`
|
||||
- [ ] `base/vercomics`
|
||||
- [ ] `base/wpcomics` ⚠️ see notes
|
||||
- [ ] `base/yuyu`
|
||||
- [ ] `base/zeistmanga`
|
||||
- [ ] `base/zmanga` ⚠️ see notes
|
||||
|
||||
---
|
||||
|
||||
## Notes — Complex Bases
|
||||
|
||||
### `base/heancms` ⚠️
|
||||
- Config struct: `BaseURL`, `APIURL`, `Lang`, `DateFormat`
|
||||
- `GetPopularManga` — `GET {api}/series?page={n}&order=views`
|
||||
- `GetLatestUpdates` — `GET {api}/series?page={n}&order=latest`
|
||||
- `GetSearchManga` — `GET {api}/series?page={n}&query={q}`
|
||||
- `GetMangaDetails` — `GET {api}/series/{slug}`
|
||||
- `GetChapterList` — `GET {api}/chapter/query?series_slug={s}&limit=9999`
|
||||
- `GetPageList` — `GET {api}/chapter/{slug}` → extract image array
|
||||
- Parse `has_next_page` field from list responses
|
||||
|
||||
### `base/hentaihand` ⚠️
|
||||
- Config struct: `BaseURL`, `Lang`
|
||||
- `GetPopularManga` — `GET {base}/api/comics?page={n}&order_by=popularity`
|
||||
- `GetLatestUpdates` — `GET {base}/api/comics?page={n}&order_by=date`
|
||||
- `GetPageList` — `GET {base}/api/comics/{slug}/chapters/{n}` → images array
|
||||
|
||||
### `base/pizzareader` ⚠️
|
||||
- Config struct: `APIURL`, `Lang`
|
||||
- `GetPageList` — `GET {api}/comics/{comic}/{chapter}` → page URLs
|
||||
- FlareSolverr mode (CF=Yes): inject cookies from clearance
|
||||
|
||||
### `base/gmanga` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/api/releases?page={n}`
|
||||
- `GetSearchManga` — `POST {base}/api/mangas/search` with JSON body `{q, page}`
|
||||
- `GetPageList` — extract chapter page data (may require resolving CDN token)
|
||||
- Handle Arabic/RTL title fields
|
||||
|
||||
### `base/mangadventure` ⚠️
|
||||
- Django manga reader REST API
|
||||
- `GET {base}/api/v2/series/` — list; `GET {base}/api/v2/series/{slug}/` — detail
|
||||
- `GET {base}/api/v2/chapters/?series={slug}` — chapters; `GET {base}/api/v2/chapters/{vol}/{num}/` — pages
|
||||
|
||||
### `base/madara` ⚠️
|
||||
- Config struct: `BaseURL`, `Lang`, `DateFormat`, overridable CSS selectors
|
||||
- `GetPopularManga` — `POST {base}/wp-admin/admin-ajax.php` with `action=madara_load_more`, `vars[paged]={n}`
|
||||
- `GetLatestUpdates` — same AJAX with `vars[meta_key]=_latest_update`
|
||||
- `GetSearchManga` — `GET {base}/?s={q}&post_type=wp-manga`
|
||||
- `GetMangaDetails` — `GET {url}` → parse `.post-title`, `.author-content`, `.genres-content`, `.manga-summary`, `.post-image img`
|
||||
- `GetChapterList` — `POST {url}/ajax/chapters/` → parse chapter list HTML
|
||||
- `GetPageList` — `GET {chapter_url}` → parse `div.reading-content img`
|
||||
- FlareSolverr required; configurable selectors struct (child sources override individual selectors)
|
||||
- Date parsing: relative ("X days ago") + absolute ("MMMM dd, yyyy") formats
|
||||
|
||||
### `base/mangathemesia` ⚠️
|
||||
- Config struct: `BaseURL`, `Lang`, `MangaDir` (e.g. "manga"/"manhwa"), overridable selectors
|
||||
- `GetPopularManga` — `GET {base}/{dir}/?page={n}&order=popular`
|
||||
- `GetChapterList` — parse `#chapterlist li` elements
|
||||
- `GetPageList` — extract `ts_reader.run({...})` JS JSON blob, parse `sources[].images`
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/madtheme` ⚠️
|
||||
- All list types via `GET {base}/search?page={n}&sort=...`
|
||||
- `GetPageList` — parse JSON blob in `<script>` tag containing image array
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/wpcomics` ⚠️
|
||||
- Config struct: `BaseURL`, `Lang`, `PopularPath` (default "tim-kiem"), `DateFormat`
|
||||
- `GetPopularManga` — `GET {base}/{popularPath}?page={n}`
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/fmreader` ⚠️
|
||||
- Config struct: `BaseURL`, `Lang`, `RequestPath`, overridable selectors
|
||||
- `GetPopularManga` — `GET {base}/{requestPath}?page={n}&sort=views`
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/mmrcms` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/filterList?page={n}&sortBy=views&asc=false`
|
||||
- `GetSearchManga` — `POST {base}/advSearchFilter` with form data
|
||||
- JSON response; no FlareSolverr required
|
||||
|
||||
### `base/mangareader` ⚠️
|
||||
- Config struct: `BaseURL`, `Lang`, `TypeParam` (comic/manga/manhwa)
|
||||
- `GetPopularManga` — `GET {base}/?page={n}&type={t}&status=all&order=popular`
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/zmanga` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/advanced-search/page/{n}/?order=popular`
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/mangaworld` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/archive?sort=most_read&page={n}`
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/grouple` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/list?sortType=rate&offset={50*(n-1)}`
|
||||
- No FlareSolverr
|
||||
|
||||
### `base/foolslide` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/directory/{n}/`
|
||||
- `GetChapterList` — JSON API: `GET {base}/api/reader/chapters?comic={slug}`
|
||||
- `GetPageList` — JSON API: `GET {base}/api/reader/images?chapter={id}`
|
||||
- No FlareSolverr
|
||||
|
||||
### `base/liliana` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/ranking/week/{n}`
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/scanreader` ⚠️
|
||||
- `GetPopularManga` — `GET {base}/bibliotheque/page/{n}/?sort=views`
|
||||
- No FlareSolverr
|
||||
|
||||
### `base/gigaviewer` ⚠️
|
||||
- `GET {base}/series` returns all manga at once; no pagination (`HasNextPage` always false)
|
||||
- `GetLatestUpdates` = same request as popular (no separate endpoint)
|
||||
- FlareSolverr required; client-side filtering replicated in Go
|
||||
|
||||
### `base/mangahub` ⚠️ (GraphQL)
|
||||
- Cookie acquisition: `GET` to any chapter URL to set `mhub_access` cookie, then reuse for GraphQL
|
||||
- All operations via GraphQL POST to `{api}/graphql`
|
||||
- `GetPopularManga` — `searchManga(x:POPULAR, genre:"all", page:N)`
|
||||
- `GetPageList` — `chapter(mangaId:N, number:N)` → extract pages
|
||||
- Inject `mhub_access` cookie header on all GraphQL requests
|
||||
|
||||
### `base/senkuro` ⚠️ (GraphQL)
|
||||
- Config struct: `APIURL` (configurable per source preferences)
|
||||
- Standard GraphQL list/detail/chapter/page queries
|
||||
- No CF requirement
|
||||
|
||||
### `base/mangotheme` ⚠️ (encrypted)
|
||||
- JSON list + detail (standard REST)
|
||||
- Page URL decryption: extract encryption key from embedded JS, then XOR/AES decrypt each page URL
|
||||
- Use `crypto/aes` or manual XOR depending on algorithm found in extension source
|
||||
|
||||
### `base/mmlook` ⚠️ (encrypted + CF)
|
||||
- Same page URL decryption as mangotheme
|
||||
- FlareSolverr required
|
||||
|
||||
### `base/guya` ⚠️
|
||||
- `GET {base}/api/get_all_series/` returns all manga as a map; no pagination
|
||||
- `HasNextPage` always false
|
||||
- Scanlation group filter applied client-side
|
||||
|
||||
### `base/bakkin` ⚠️
|
||||
- No list/search; all manga from a single JSON URL; enumerate from object keys
|
||||
- `GetSearchManga` does client-side title filtering only
|
||||
|
||||
### `base/iken` ⚠️
|
||||
- CF-protected; FlareSolverr required for cookie acquisition
|
||||
- JSON REST after clearance
|
||||
|
||||
### `base/lectormoe` ⚠️
|
||||
- CF-protected; Token auth required
|
||||
- Obtain token via FlareSolverr session
|
||||
|
||||
### `base/libgroup` ⚠️
|
||||
- FlareSolverr required to acquire WebView auth token
|
||||
- `GET {api}/api/latest-updates`, `GET {api}/api/auth/me`
|
||||
- Store acquired token in `source_meta` table for reuse
|
||||
|
||||
### `base/kemono` ⚠️
|
||||
- `GET {base}/api/v1/creators` — all creators (= manga list)
|
||||
- `GET {base}/api/v1/{service}/{creator}/posts?o={offset}` — paginate in 50-post increments
|
||||
- File URLs: prefix relative paths with `{base}/data`
|
||||
- FlareSolverr required
|
||||
|
||||
---
|
||||
|
||||
## Shared Helpers (implement once in `sources/base/util/`)
|
||||
|
||||
- [ ] `parseRelativeDate(s string) int64` — "2 days ago" → unix ms
|
||||
- [ ] `parseAbsoluteDate(s, format string) int64` — "January 01, 2024" → unix ms
|
||||
- [ ] `slugFromURL(url string) string` — trailing path segment
|
||||
- [ ] `cleanText(s string) string` — HTML entity decode + whitespace normalize
|
||||
- [ ] `statusFromString(s string) int` — "ongoing"/"completed"/etc. → int constant
|
||||
- [ ] `extractNextDataJSON(html string) ([]byte, error)` — pull `__NEXT_DATA__` JSON from NextJS pages
|
||||
|
||||
---
|
||||
|
||||
## Checklist: Phase 3 Done When
|
||||
|
||||
- [ ] All 68 bases compile: `go build ./sources/base/...`
|
||||
- [ ] `base/heancms` — `GetPopularManga` returns ≥1 manga from a live site
|
||||
- [ ] `base/madara` — `GetChapterList` returns chapters via AJAX endpoint
|
||||
- [ ] `base/mangathemesia` — `GetPageList` extracts images from `ts_reader.run()` JS blob
|
||||
- [ ] `base/mangahub` — GraphQL popular list works with cookie acquisition
|
||||
- [ ] `base/mangotheme` — decrypted page URL returns HTTP 200 image
|
||||
- [ ] FlareSolverr path — a CF-protected base returns data when FlareSolverr is running
|
||||
@@ -0,0 +1,665 @@
|
||||
# Phase 4 — Standalone Sources
|
||||
|
||||
Complete port checklist. Check a box when the source passes a basic smoke test
|
||||
(popular/latest list returns ≥1 result, or detail+pages resolve for a known URL).
|
||||
|
||||
Reference:
|
||||
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/all/`
|
||||
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/en/`
|
||||
|
||||
Detailed implementation notes for complex sources are in the **Notes** section at the bottom.
|
||||
|
||||
---
|
||||
|
||||
## `sources/all/` — 125 sources
|
||||
|
||||
- [ ] `all/ahottie`
|
||||
- [ ] `all/akuma`
|
||||
- [ ] `all/allporncomicsco`
|
||||
- [ ] `all/asmhentai`
|
||||
- [ ] `all/baobua`
|
||||
- [ ] `all/beauty3600000`
|
||||
- [ ] `all/buondua`
|
||||
- [ ] `all/comicfury`
|
||||
- [ ] `all/comicgrowl`
|
||||
- [ ] `all/comicklive`
|
||||
- [ ] `all/comicskingdom`
|
||||
- [ ] `all/comicsvalley`
|
||||
- [ ] `all/comikey`
|
||||
- [ ] `all/commitstrip`
|
||||
- [ ] `all/coomer`
|
||||
- [ ] `all/coronaex`
|
||||
- [ ] `all/cosplaytele`
|
||||
- [ ] `all/cubari`
|
||||
- [ ] `all/danbooru` ⚠️ see notes
|
||||
- [ ] `all/deviantart`
|
||||
- [ ] `all/dragonballmultiverse`
|
||||
- [ ] `all/e621` ⚠️ see notes
|
||||
- [ ] `all/elitebabes`
|
||||
- [ ] `all/everiaclub`
|
||||
- [ ] `all/everiaclubcom`
|
||||
- [ ] `all/femjoyhunter`
|
||||
- [ ] `all/foamgirl`
|
||||
- [ ] `all/foolslidecustomizable`
|
||||
- [ ] `all/fourkhd`
|
||||
- [ ] `all/ftvhunter`
|
||||
- [ ] `all/fuwayomi`
|
||||
- [ ] `all/globalcomix` ⚠️ see notes
|
||||
- [ ] `all/grabberzone`
|
||||
- [ ] `all/hdoujin`
|
||||
- [ ] `all/hennojin`
|
||||
- [ ] `all/hentai3`
|
||||
- [ ] `all/hentaicosplay`
|
||||
- [ ] `all/hentaienvy`
|
||||
- [ ] `all/hentaiera`
|
||||
- [ ] `all/hentaifox`
|
||||
- [ ] `all/hentaihand`
|
||||
- [ ] `all/hentairox`
|
||||
- [ ] `all/hentaizap`
|
||||
- [ ] `all/hniscantrad`
|
||||
- [ ] `all/holonometria`
|
||||
- [ ] `all/honeytoon`
|
||||
- [ ] `all/imhentai`
|
||||
- [ ] `all/izneo`
|
||||
- [ ] `all/jjcos`
|
||||
- [ ] `all/joymiihub`
|
||||
- [ ] `all/junmeitu`
|
||||
- [ ] `all/kemono` ⚠️ see notes
|
||||
- [ ] `all/kiutaku`
|
||||
- [ ] `all/kodokustudio`
|
||||
- [ ] `all/koharu`
|
||||
- [ ] `all/komga` ⚠️ see notes
|
||||
- [ ] `all/lanraragi`
|
||||
- [ ] `all/leagueoflegends`
|
||||
- [ ] `all/lunaranime`
|
||||
- [ ] `all/luscious` ⚠️ see notes
|
||||
- [ ] `all/magicaltranslators`
|
||||
- [ ] `all/manga18me`
|
||||
- [ ] `all/mangaball`
|
||||
- [ ] `all/mangacrazy`
|
||||
- [ ] `all/mangadex` ⚠️ see notes
|
||||
- [ ] `all/mangadraft`
|
||||
- [ ] `all/mangafire`
|
||||
- [ ] `all/mangaforfree`
|
||||
- [ ] `all/mangaplus` ⚠️ see notes
|
||||
- [ ] `all/mangapluscreators`
|
||||
- [ ] `all/mangataro`
|
||||
- [ ] `all/mangatoon`
|
||||
- [ ] `all/mangaup`
|
||||
- [ ] `all/mango`
|
||||
- [ ] `all/manhuarm`
|
||||
- [ ] `all/manhwa18cc`
|
||||
- [ ] `all/manhwa18net`
|
||||
- [ ] `all/manhwa18uncensored`
|
||||
- [ ] `all/manhwaclubnet`
|
||||
- [ ] `all/manhwadashraw`
|
||||
- [ ] `all/mayotune`
|
||||
- [ ] `all/metarthunter`
|
||||
- [ ] `all/miauscan`
|
||||
- [ ] `all/misskon`
|
||||
- [ ] `all/mitaku`
|
||||
- [ ] `all/myreadingmanga`
|
||||
- [ ] `all/namicomi`
|
||||
- [ ] `all/nhentaicom` ⚠️ see notes
|
||||
- [ ] `all/nhentaixxx`
|
||||
- [ ] `all/niadd`
|
||||
- [ ] `all/ninemanga`
|
||||
- [ ] `all/novelcool`
|
||||
- [ ] `all/ososedki`
|
||||
- [ ] `all/pandachaika`
|
||||
- [ ] `all/peppercarrot`
|
||||
- [ ] `all/photos18`
|
||||
- [ ] `all/pixiv` ⚠️ see notes
|
||||
- [ ] `all/playmatehunter`
|
||||
- [ ] `all/pornpics`
|
||||
- [ ] `all/projectsuki`
|
||||
- [ ] `all/qtoon`
|
||||
- [ ] `all/rokuhentai`
|
||||
- [ ] `all/sakuramanhwa`
|
||||
- [ ] `all/sandraandwoo`
|
||||
- [ ] `all/seraphicdeviltry`
|
||||
- [ ] `all/simplycosplay`
|
||||
- [ ] `all/simplyhentai`
|
||||
- [ ] `all/stashapp` ⚠️ see notes
|
||||
- [ ] `all/taddyink`
|
||||
- [ ] `all/tappytoon`
|
||||
- [ ] `all/thelibraryofohara`
|
||||
- [ ] `all/thunderscans`
|
||||
- [ ] `all/toomics`
|
||||
- [ ] `all/twicomi`
|
||||
- [ ] `all/uncensoredmanhwa`
|
||||
- [ ] `all/vinnieVeritas`
|
||||
- [ ] `all/webtoons` ⚠️ see notes
|
||||
- [ ] `all/xarthunter`
|
||||
- [ ] `all/xasiatalbums`
|
||||
- [ ] `all/xgmn`
|
||||
- [ ] `all/xinmeitulu`
|
||||
- [ ] `all/xiutaku`
|
||||
- [ ] `all/xkcd`
|
||||
- [ ] `all/yabai`
|
||||
- [ ] `all/yaoimangaonline`
|
||||
- [ ] `all/yellownote`
|
||||
- [ ] `all/yskcomics`
|
||||
|
||||
---
|
||||
|
||||
## `sources/en/` — 430 sources
|
||||
|
||||
- [ ] `en/akaicomic`
|
||||
- [ ] `en/alandal`
|
||||
- [ ] `en/allanime` ⚠️ see notes
|
||||
- [ ] `en/allporncomic`
|
||||
- [ ] `en/allporncomicio`
|
||||
- [ ] `en/anisascans`
|
||||
- [ ] `en/apcomics`
|
||||
- [ ] `en/aquamanga`
|
||||
- [ ] `en/arcrelight`
|
||||
- [ ] `en/arenascans`
|
||||
- [ ] `en/armageddon`
|
||||
- [ ] `en/artlapsa`
|
||||
- [ ] `en/arvencomics`
|
||||
- [ ] `en/arvenscans`
|
||||
- [ ] `en/aryascans`
|
||||
- [ ] `en/asiatoon`
|
||||
- [ ] `en/asmotoon`
|
||||
- [ ] `en/assortedscans`
|
||||
- [ ] `en/asurascans` ⚠️ see notes
|
||||
- [ ] `en/athreascans`
|
||||
- [ ] `en/atsumaru`
|
||||
- [ ] `en/aurora`
|
||||
- [ ] `en/azcomic`
|
||||
- [ ] `en/azuki`
|
||||
- [ ] `en/baektoons`
|
||||
- [ ] `en/bakkin` ⚠️ see notes
|
||||
- [ ] `en/bakkinselfhosted`
|
||||
- [ ] `en/batcave`
|
||||
- [ ] `en/battleinfivesecondsaftermeeting`
|
||||
- [ ] `en/beehentai`
|
||||
- [ ] `en/bookwalker` ⚠️ see notes
|
||||
- [ ] `en/boratscans`
|
||||
- [ ] `en/boxmanhwa`
|
||||
- [ ] `en/broccolisoup`
|
||||
- [ ] `en/bunmanga`
|
||||
- [ ] `en/buttsmithy`
|
||||
- [ ] `en/clonemanga`
|
||||
- [ ] `en/clowncorps`
|
||||
- [ ] `en/cmanhua`
|
||||
- [ ] `en/cocomic`
|
||||
- [ ] `en/coffeemanga`
|
||||
- [ ] `en/collectedcurios`
|
||||
- [ ] `en/comicasura`
|
||||
- [ ] `en/comiccx`
|
||||
- [ ] `en/comichubfree`
|
||||
- [ ] `en/comickfan`
|
||||
- [ ] `en/comickiba`
|
||||
- [ ] `en/comicland`
|
||||
- [ ] `en/comicsland`
|
||||
- [ ] `en/comix`
|
||||
- [ ] `en/crowscans`
|
||||
- [ ] `en/cucumbermanga`
|
||||
- [ ] `en/culturedworks`
|
||||
- [ ] `en/cutiecomics`
|
||||
- [ ] `en/dankefurslesen`
|
||||
- [ ] `en/darklegacycomics`
|
||||
- [ ] `en/darkscans`
|
||||
- [ ] `en/darkscience`
|
||||
- [ ] `en/darthsdroids`
|
||||
- [ ] `en/deathtollscans`
|
||||
- [ ] `en/decadencescans`
|
||||
- [ ] `en/dflowscans`
|
||||
- [ ] `en/digitalcomicmuseum`
|
||||
- [ ] `en/divascans`
|
||||
- [ ] `en/doujinio`
|
||||
- [ ] `en/doujins`
|
||||
- [ ] `en/dragontea`
|
||||
- [ ] `en/drakescans`
|
||||
- [ ] `en/dynasty`
|
||||
- [ ] `en/eggporncomics`
|
||||
- [ ] `en/egscomics`
|
||||
- [ ] `en/eighteenporncomic`
|
||||
- [ ] `en/eightmuses`
|
||||
- [ ] `en/elanschool`
|
||||
- [ ] `en/elftoon`
|
||||
- [ ] `en/epicmanga`
|
||||
- [ ] `en/erisscans`
|
||||
- [ ] `en/ero18x`
|
||||
- [ ] `en/erofus`
|
||||
- [ ] `en/erosscans`
|
||||
- [ ] `en/evascans`
|
||||
- [ ] `en/evilflowers`
|
||||
- [ ] `en/existentialcomics`
|
||||
- [ ] `en/explosm`
|
||||
- [ ] `en/ezmanga`
|
||||
- [ ] `en/fablescans`
|
||||
- [ ] `en/fairyscans`
|
||||
- [ ] `en/firescans`
|
||||
- [ ] `en/flamecomics`
|
||||
- [ ] `en/frierenonline`
|
||||
- [ ] `en/gakamangas`
|
||||
- [ ] `en/galaxydegenscans`
|
||||
- [ ] `en/galaxymanga`
|
||||
- [ ] `en/gedecomix`
|
||||
- [ ] `en/gingertoon`
|
||||
- [ ] `en/goda`
|
||||
- [ ] `en/gourmetscans`
|
||||
- [ ] `en/greedscans`
|
||||
- [ ] `en/grimscans`
|
||||
- [ ] `en/grrlpower`
|
||||
- [ ] `en/gunnerkriggcourt`
|
||||
- [ ] `en/guya` ⚠️ see notes
|
||||
- [ ] `en/gwtb`
|
||||
- [ ] `en/hachirumi`
|
||||
- [ ] `en/hadesscans`
|
||||
- [ ] `en/harimanga`
|
||||
- [ ] `en/hentai3zcc`
|
||||
- [ ] `en/hentai4free`
|
||||
- [ ] `en/hentaidex`
|
||||
- [ ] `en/hentaihere`
|
||||
- [ ] `en/hentaikun`
|
||||
- [ ] `en/hentainexus`
|
||||
- [ ] `en/hentairead`
|
||||
- [ ] `en/hentaireadio`
|
||||
- [ ] `en/hentaisco`
|
||||
- [ ] `en/hentaixcomic`
|
||||
- [ ] `en/hentaixdickgirl`
|
||||
- [ ] `en/hentaixyuri`
|
||||
- [ ] `en/hentara`
|
||||
- [ ] `en/heytoon`
|
||||
- [ ] `en/hijalascans`
|
||||
- [ ] `en/hiperdex`
|
||||
- [ ] `en/hiveworks`
|
||||
- [ ] `en/hm2d`
|
||||
- [ ] `en/honkaiimpact`
|
||||
- [ ] `en/hotcomics`
|
||||
- [ ] `en/hyakuro`
|
||||
- [ ] `en/infernalvoidscans`
|
||||
- [ ] `en/infinityscans`
|
||||
- [ ] `en/irovedout`
|
||||
- [ ] `en/isekaiscantop`
|
||||
- [ ] `en/jinmangas`
|
||||
- [ ] `en/jnovel`
|
||||
- [ ] `en/kagane`
|
||||
- [ ] `en/kaizenscan`
|
||||
- [ ] `en/kaliscancom`
|
||||
- [ ] `en/kaliscanio`
|
||||
- [ ] `en/kaliscanme`
|
||||
- [ ] `en/kappabeast`
|
||||
- [ ] `en/kaynscans`
|
||||
- [ ] `en/keenspot`
|
||||
- [ ] `en/kenscans`
|
||||
- [ ] `en/kewnscans`
|
||||
- [ ] `en/killsixbilliondemons`
|
||||
- [ ] `en/kingcomix`
|
||||
- [ ] `en/kingofshojo`
|
||||
- [ ] `en/kissmangain`
|
||||
- [ ] `en/kmanga`
|
||||
- [ ] `en/kodansha`
|
||||
- [ ] `en/ksgroupscans`
|
||||
- [ ] `en/kunmanga`
|
||||
- [ ] `en/kuramanga`
|
||||
- [ ] `en/lagoonscans`
|
||||
- [ ] `en/leslievictims`
|
||||
- [ ] `en/lhtranslation`
|
||||
- [ ] `en/likemanga`
|
||||
- [ ] `en/likemangain`
|
||||
- [ ] `en/lilymanga`
|
||||
- [ ] `en/linkmanga`
|
||||
- [ ] `en/loadingartist`
|
||||
- [ ] `en/luascans`
|
||||
- [ ] `en/luminaretranslations`
|
||||
- [ ] `en/lunatoons`
|
||||
- [ ] `en/madaradex`
|
||||
- [ ] `en/madarascans`
|
||||
- [ ] `en/madokami`
|
||||
- [ ] `en/magusmanga`
|
||||
- [ ] `en/mahouirexnohentaikarte`
|
||||
- [ ] `en/manga18club`
|
||||
- [ ] `en/manga18free`
|
||||
- [ ] `en/manga18fx`
|
||||
- [ ] `en/manga18x`
|
||||
- [ ] `en/mangabat`
|
||||
- [ ] `en/mangablaze`
|
||||
- [ ] `en/mangabolt`
|
||||
- [ ] `en/mangabtt`
|
||||
- [ ] `en/mangabuddy`
|
||||
- [ ] `en/mangabuddyme`
|
||||
- [ ] `en/mangaclash`
|
||||
- [ ] `en/mangacloud`
|
||||
- [ ] `en/mangacute`
|
||||
- [ ] `en/mangadass`
|
||||
- [ ] `en/mangade`
|
||||
- [ ] `en/mangademon`
|
||||
- [ ] `en/mangadia`
|
||||
- [ ] `en/mangadistrict`
|
||||
- [ ] `en/mangadotnet`
|
||||
- [ ] `en/mangadrama`
|
||||
- [ ] `en/mangafab`
|
||||
- [ ] `en/mangaforest`
|
||||
- [ ] `en/mangaforfreecom`
|
||||
- [ ] `en/mangafox`
|
||||
- [ ] `en/mangafoxfun`
|
||||
- [ ] `en/mangafreak`
|
||||
- [ ] `en/mangafree`
|
||||
- [ ] `en/mangagg`
|
||||
- [ ] `en/mangago`
|
||||
- [ ] `en/mangagofun`
|
||||
- [ ] `en/mangahe`
|
||||
- [ ] `en/mangahen`
|
||||
- [ ] `en/mangahentai`
|
||||
- [ ] `en/mangahere`
|
||||
- [ ] `en/mangahereonl`
|
||||
- [ ] `en/mangahubio`
|
||||
- [ ] `en/mangaka`
|
||||
- [ ] `en/mangakakalot`
|
||||
- [ ] `en/mangakakalotfun`
|
||||
- [ ] `en/mangakatana`
|
||||
- [ ] `en/mangakiss`
|
||||
- [ ] `en/mangamaniacs`
|
||||
- [ ] `en/mangamo`
|
||||
- [ ] `en/mangamob`
|
||||
- [ ] `en/mangamonk`
|
||||
- [ ] `en/manganel`
|
||||
- [ ] `en/manganelo`
|
||||
- [ ] `en/manganow`
|
||||
- [ ] `en/mangaonlinefun`
|
||||
- [ ] `en/mangaowlio`
|
||||
- [ ] `en/mangapandaonl`
|
||||
- [ ] `en/mangapill`
|
||||
- [ ] `en/mangapuma`
|
||||
- [ ] `en/mangarawclub`
|
||||
- [ ] `en/mangaread`
|
||||
- [ ] `en/mangareadercc`
|
||||
- [ ] `en/mangareadersite`
|
||||
- [ ] `en/mangareadorg`
|
||||
- [ ] `en/mangasaga`
|
||||
- [ ] `en/mangasect`
|
||||
- [ ] `en/mangaspin`
|
||||
- [ ] `en/mangasushi`
|
||||
- [ ] `en/mangatellers`
|
||||
- [ ] `en/mangatoday`
|
||||
- [ ] `en/mangatown`
|
||||
- [ ] `en/mangatrend`
|
||||
- [ ] `en/mangatx`
|
||||
- [ ] `en/mangaxyz`
|
||||
- [ ] `en/manhuafast`
|
||||
- [ ] `en/manhuafastnet`
|
||||
- [ ] `en/manhuahot`
|
||||
- [ ] `en/manhuanext`
|
||||
- [ ] `en/manhuanow`
|
||||
- [ ] `en/manhuaplus`
|
||||
- [ ] `en/manhuaplusorg`
|
||||
- [ ] `en/manhuarush`
|
||||
- [ ] `en/manhuascanus`
|
||||
- [ ] `en/manhuasite`
|
||||
- [ ] `en/manhuatop`
|
||||
- [ ] `en/manhuaus`
|
||||
- [ ] `en/manhuazonghe`
|
||||
- [ ] `en/manhwa18`
|
||||
- [ ] `en/manhwa18org`
|
||||
- [ ] `en/manhwa68`
|
||||
- [ ] `en/manhwabuddy`
|
||||
- [ ] `en/manhwaclan`
|
||||
- [ ] `en/manhwacomics`
|
||||
- [ ] `en/manhwaden`
|
||||
- [ ] `en/manhwaget`
|
||||
- [ ] `en/manhwahub`
|
||||
- [ ] `en/manhwajoy`
|
||||
- [ ] `en/manhwalike`
|
||||
- [ ] `en/manhwalover`
|
||||
- [ ] `en/manhwamanhua`
|
||||
- [ ] `en/manhwaread`
|
||||
- [ ] `en/manhwareads`
|
||||
- [ ] `en/manhwatoon`
|
||||
- [ ] `en/manhwatop`
|
||||
- [ ] `en/manhwax`
|
||||
- [ ] `en/manhwaxxl`
|
||||
- [ ] `en/manhwaz`
|
||||
- [ ] `en/manhwazone`
|
||||
- [ ] `en/manta`
|
||||
- [ ] `en/megatokyo`
|
||||
- [ ] `en/mehgazone`
|
||||
- [ ] `en/meitoon`
|
||||
- [ ] `en/mgjinx`
|
||||
- [ ] `en/mgreadio`
|
||||
- [ ] `en/milftoon`
|
||||
- [ ] `en/mistscans`
|
||||
- [ ] `en/mlbblore`
|
||||
- [ ] `en/monochromecustom`
|
||||
- [ ] `en/monochromescans`
|
||||
- [ ] `en/multporn`
|
||||
- [ ] `en/murimscan`
|
||||
- [ ] `en/myhentaicomics`
|
||||
- [ ] `en/myhentaigallery`
|
||||
- [ ] `en/necroscans`
|
||||
- [ ] `en/newmanhwa`
|
||||
- [ ] `en/nexcomic`
|
||||
- [ ] `en/nikatoons`
|
||||
- [ ] `en/nineanime`
|
||||
- [ ] `en/ninehentai`
|
||||
- [ ] `en/ninekon`
|
||||
- [ ] `en/novel24h`
|
||||
- [ ] `en/novelcrow`
|
||||
- [ ] `en/noxenscans`
|
||||
- [ ] `en/nuxscans`
|
||||
- [ ] `en/nyanukafe`
|
||||
- [ ] `en/nyrascans`
|
||||
- [ ] `en/nyxscans`
|
||||
- [ ] `en/octopusmanga`
|
||||
- [ ] `en/oglaf`
|
||||
- [ ] `en/ohjoysextoy`
|
||||
- [ ] `en/omegascans`
|
||||
- [ ] `en/onemangaco`
|
||||
- [ ] `en/onemangainfo`
|
||||
- [ ] `en/onepunchmanonline`
|
||||
- [ ] `en/onlythebesthentai`
|
||||
- [ ] `en/oots`
|
||||
- [ ] `en/oppaistream`
|
||||
- [ ] `en/orchisasia`
|
||||
- [ ] `en/orionscans`
|
||||
- [ ] `en/paradisescans`
|
||||
- [ ] `en/paragonscans`
|
||||
- [ ] `en/paritehaber`
|
||||
- [ ] `en/patchfriday`
|
||||
- [ ] `en/pawmanga`
|
||||
- [ ] `en/petrotechsociety`
|
||||
- [ ] `en/philiascans`
|
||||
- [ ] `en/plutoscans`
|
||||
- [ ] `en/pmscans`
|
||||
- [ ] `en/porncomix`
|
||||
- [ ] `en/qiscans`
|
||||
- [ ] `en/questionablecontent`
|
||||
- [ ] `en/ragescans`
|
||||
- [ ] `en/randowiz`
|
||||
- [ ] `en/ravenscans`
|
||||
- [ ] `en/razure`
|
||||
- [ ] `en/rdscans`
|
||||
- [ ] `en/readallcomicscom`
|
||||
- [ ] `en/readattackontitanshingekinokyojinmanga`
|
||||
- [ ] `en/readberserkmanga`
|
||||
- [ ] `en/readblackclovermangaonline`
|
||||
- [ ] `en/readbokunoheroacademiamyheroacademiamanga`
|
||||
- [ ] `en/readchainsawmanmangaonline`
|
||||
- [ ] `en/readcomiconline`
|
||||
- [ ] `en/readcomicsonline`
|
||||
- [ ] `en/readfairytailedenszeromangaonline`
|
||||
- [ ] `en/readhaikyuumangaonline`
|
||||
- [ ] `en/readjujutsukaisenmangaonline`
|
||||
- [ ] `en/readkingdommangaonline`
|
||||
- [ ] `en/readnanatsunotaizai7deadlysinsmangaonline`
|
||||
- [ ] `en/readnarutoborutosamurai8mangaonline`
|
||||
- [ ] `en/readonepiecemangaonline`
|
||||
- [ ] `en/readonepunchmanmangaonlinetwo`
|
||||
- [ ] `en/readsololevelingmangamanhwaonline`
|
||||
- [ ] `en/readtokyoghoulretokyoghoulmangaonline`
|
||||
- [ ] `en/readvagabondmanga`
|
||||
- [ ] `en/reallifecomics`
|
||||
- [ ] `en/reimanga`
|
||||
- [ ] `en/renascans`
|
||||
- [ ] `en/resetscans`
|
||||
- [ ] `en/restscans`
|
||||
- [ ] `en/retsu`
|
||||
- [ ] `en/revivalscans`
|
||||
- [ ] `en/rinkocomics`
|
||||
- [ ] `en/ritharscans`
|
||||
- [ ] `en/rizzcomic`
|
||||
- [ ] `en/rizzcomicunoriginal`
|
||||
- [ ] `en/rokaricomics`
|
||||
- [ ] `en/roliascan`
|
||||
- [ ] `en/rosesquadscans`
|
||||
- [ ] `en/ryumanga`
|
||||
- [ ] `en/s2manga`
|
||||
- [ ] `en/sanascans`
|
||||
- [ ] `en/saturdaymorningbreakfastcomics`
|
||||
- [ ] `en/schlockmercenary`
|
||||
- [ ] `en/setsuscans`
|
||||
- [ ] `en/shibamanga`
|
||||
- [ ] `en/shojoscans`
|
||||
- [ ] `en/sirenscans`
|
||||
- [ ] `en/skymanga`
|
||||
- [ ] `en/sleepytranslations`
|
||||
- [ ] `en/solarandsundry`
|
||||
- [ ] `en/spmanhwa`
|
||||
- [ ] `en/spyfakku`
|
||||
- [ ] `en/stonescape`
|
||||
- [ ] `en/sunshinebutterflyscans`
|
||||
- [ ] `en/supermega`
|
||||
- [ ] `en/suryascans`
|
||||
- [ ] `en/swordscomic`
|
||||
- [ ] `en/tapastic`
|
||||
- [ ] `en/tcbscans`
|
||||
- [ ] `en/tcbscansunoriginal`
|
||||
- [ ] `en/teamshadowi`
|
||||
- [ ] `en/templescan`
|
||||
- [ ] `en/theblank`
|
||||
- [ ] `en/theduckwebcomics`
|
||||
- [ ] `en/thepropertyofhate`
|
||||
- [ ] `en/timelesstoons`
|
||||
- [ ] `en/todaymanga`
|
||||
- [ ] `en/toon18`
|
||||
- [ ] `en/toongod`
|
||||
- [ ] `en/toonily`
|
||||
- [ ] `en/toonilyme`
|
||||
- [ ] `en/toonitube`
|
||||
- [ ] `en/toonizy`
|
||||
- [ ] `en/topmanhua`
|
||||
- [ ] `en/topmanhuafan`
|
||||
- [ ] `en/topmanhuanet`
|
||||
- [ ] `en/tritiniascans`
|
||||
- [ ] `en/utoon`
|
||||
- [ ] `en/valirscans`
|
||||
- [ ] `en/vanillascans`
|
||||
- [ ] `en/vgperson`
|
||||
- [ ] `en/vizshonenjump`
|
||||
- [ ] `en/voyceme`
|
||||
- [ ] `en/vyvymanga`
|
||||
- [ ] `en/vyvymangaorg`
|
||||
- [ ] `en/warforrayuba`
|
||||
- [ ] `en/wearehunger`
|
||||
- [ ] `en/webcomics`
|
||||
- [ ] `en/webdexscans`
|
||||
- [ ] `en/webnovel`
|
||||
- [ ] `en/webtoonscan`
|
||||
- [ ] `en/webtoonxyz`
|
||||
- [ ] `en/weebcentral`
|
||||
- [ ] `en/whalemanga`
|
||||
- [ ] `en/witchscans`
|
||||
- [ ] `en/woopread`
|
||||
- [ ] `en/writerscans`
|
||||
- [ ] `en/wuxiaworld`
|
||||
- [ ] `en/xlecx`
|
||||
- [ ] `en/xomanga`
|
||||
- [ ] `en/xoxocomics`
|
||||
- [ ] `en/yakshacomics`
|
||||
- [ ] `en/yaoihot`
|
||||
- [ ] `en/yaoihub`
|
||||
- [ ] `en/yaoiscan`
|
||||
- [ ] `en/yaoitoon`
|
||||
- [ ] `en/yorai`
|
||||
- [ ] `en/zazamanga`
|
||||
- [ ] `en/zinchanmanga`
|
||||
- [ ] `en/zinchanmangacom`
|
||||
- [ ] `en/zinmanga`
|
||||
- [ ] `en/zinmanganet`
|
||||
|
||||
---
|
||||
|
||||
## Notes — Complex Sources
|
||||
|
||||
### `all/mangadex` ⚠️
|
||||
- Rate limit: 5 req/s (`golang.org/x/time/rate`)
|
||||
- `GetPopularManga` — `GET /manga?order[rating]=desc&includes[]=cover_art&limit=20&offset={(n-1)*20}`
|
||||
- `GetLatestUpdates` — `GET /manga?order[updatedAt]=desc&includes[]=cover_art`
|
||||
- `GetSearchManga` — tag filters (`includedTags[]`, `excludedTags[]`), demographic, content rating, status, sort
|
||||
- `GetMangaDetails` — `GET /manga/{id}?includes[]=cover_art&includes[]=author&includes[]=artist`; cover URL from relationships
|
||||
- `GetChapterList` — `GET /manga/{id}/feed?translatedLanguage[]=en&limit=500`; paginate until all chapters fetched
|
||||
- `GetPageList` — `GET /at-home/server/{chapterId}` → `{baseUrl}/{quality}/{hash}/{filename}`; quality = `data` or `data-saver`
|
||||
- `GetFilterList` — `GET /manga/tag`; cache tag list
|
||||
- Language configurable via `MANGADEX_LANG` env var (default `en`)
|
||||
|
||||
### `all/nhentaicom` ⚠️
|
||||
- FlareSolverr required
|
||||
- `GetPageList` — extract JSON blob from `document.getElementById(...)` in `<script>` tag; reconstruct URLs as `{imageServer}/galleries/{mediaId}/{pageNum}.{ext}`
|
||||
|
||||
### `all/kemono` ⚠️
|
||||
- FlareSolverr required
|
||||
- Service + creator = manga; posts = chapters; attachment files = pages
|
||||
- File URLs: prefix relative paths with `{base}/data`
|
||||
- `GetChapterList` — paginate in 50-post increments via `?o={offset}`
|
||||
|
||||
### `all/komga` ⚠️
|
||||
- Self-hosted; `KOMGA_BASE_URL`, `KOMGA_USERNAME`, `KOMGA_PASSWORD` env vars; Basic Auth
|
||||
- Series = manga; Books = chapters; pages via `GET /api/v1/books/{bookId}/pages/{n}/thumbnail`
|
||||
|
||||
### `all/e621` ⚠️
|
||||
- Optional Basic Auth (`E621_USERNAME`, `E621_API_KEY`); required for Gold content
|
||||
- Pools = manga; posts = pages; `GET /pools.json`, `GET /posts.json?tags=pool:{id}`
|
||||
|
||||
### `all/danbooru` ⚠️
|
||||
- Optional Basic Auth; free accounts limited to 2 tags per search
|
||||
- Pools = manga; pool post_ids = pages; image URL from `file_url` field
|
||||
|
||||
### `all/pixiv` ⚠️
|
||||
- Auth: `PIXIV_PHPSESSID` env var → Cookie header
|
||||
- `GetPageList` — `GET /ajax/illust/{illustId}/pages`; extract `urls.original`
|
||||
- **Must set `Referer: https://www.pixiv.net/`** on all image requests or get 403
|
||||
|
||||
### `all/luscious` ⚠️
|
||||
- GraphQL POST to `https://members.luscious.net/graphql/playground`
|
||||
- Albums = manga; `SearchAlbumPictures` paginated for page list
|
||||
|
||||
### `all/mangaplus` ⚠️
|
||||
- JSON endpoint: `GET https://jumpg-webapi.tokyo-cdn.com/api/title_detail?title_id={id}`
|
||||
- `GetPageList` — XOR page URL decryption using per-page `encryptionKey` (hex string); implement in `GetImageURL`
|
||||
- Required header: `Origin: https://mangaplus.shueisha.co.jp`
|
||||
|
||||
### `all/stashapp` ⚠️
|
||||
- Self-hosted; `STASHAPP_BASE_URL`, `STASHAPP_API_KEY` env vars
|
||||
- GraphQL POST to `{base}/graphql`; `ApiKey` header
|
||||
|
||||
### `all/globalcomix` ⚠️
|
||||
- Signed CDN URLs with short expiry — **do NOT cache `image_url` in DB**; always fetch fresh page URLs
|
||||
|
||||
### `all/webtoons` ⚠️
|
||||
- HMAC-SHA256 `Sec-Webtoon-Client-Data` header: `HMAC-SHA256(fixedKey, "{url}_{timestamp}")` → base64; fixed key in extension source
|
||||
- Web API: `https://global.apis.naver.com/webtoonSvc/v1/...`
|
||||
|
||||
### `en/allanime` ⚠️
|
||||
- GraphQL POST to `https://api.allanime.day/api`
|
||||
- Episode-based chapters; `episodeString` used as chapter number
|
||||
- Multiple CDN quality tiers in page response; pick highest
|
||||
|
||||
### `en/asurascans` ⚠️
|
||||
- FlareSolverr required
|
||||
- Page selector varies by layout version; check extension source for current selector
|
||||
|
||||
### `en/bakkin` ⚠️
|
||||
- No list/search; all manga from a single JSON URL; enumerate from object keys
|
||||
- `GetSearchManga` does client-side title filtering only
|
||||
|
||||
### `en/bookwalker` ⚠️
|
||||
- DRM-protected; metadata only
|
||||
- `GetPageList` returns empty slice; pages not accessible without purchase
|
||||
|
||||
### `en/guya` ⚠️
|
||||
- `GET {base}/api/get_all_series/` returns all manga at once; no pagination
|
||||
- Scanlation group filter applied client-side in response parsing
|
||||
@@ -0,0 +1,180 @@
|
||||
# Phase 5 — HTTP API (REST)
|
||||
|
||||
Plain REST/JSON API over HTTP using `github.com/go-chi/chi/v5`. No GraphQL on our end.
|
||||
All handlers live in `api/handler.go`. Server bootstrap in `cmd/server/main.go`.
|
||||
|
||||
Reference:
|
||||
- Suwayomi API patterns: `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/api/`
|
||||
|
||||
---
|
||||
|
||||
## 5.1 Server Bootstrap — `cmd/server/main.go`
|
||||
|
||||
- [ ] Parse config from env vars:
|
||||
- [ ] `DATABASE_URL` (required)
|
||||
- [ ] `PORT` (default `8080`)
|
||||
- [ ] `FLARESOLVERR_URL` (optional; disables CF-dependent sources if empty)
|
||||
- [ ] TTL vars: `MANGA_LIST_TTL_SECONDS`, `MANGA_DETAIL_TTL_SECONDS`, `CHAPTER_LIST_TTL_SECONDS`
|
||||
- [ ] Initialize `pgxpool.Pool` via `db.New()`
|
||||
- [ ] Run DB migrations via `db.Migrate()`
|
||||
- [ ] Blank-import all source packages so their `init()` registers them:
|
||||
```go
|
||||
import (
|
||||
_ "tachiyomi-go/sources/all/mangadex"
|
||||
_ "tachiyomi-go/sources/all/kemono"
|
||||
_ "tachiyomi-go/sources/en/allanime"
|
||||
// ... all sources
|
||||
)
|
||||
```
|
||||
- [ ] `UpsertSource` for every registered source on startup (sync source table with compiled sources)
|
||||
- [ ] Mount router and `http.ListenAndServe`
|
||||
- [ ] Graceful shutdown on SIGINT/SIGTERM (drain pool, close server)
|
||||
|
||||
---
|
||||
|
||||
## 5.2 Router — `api/handler.go`
|
||||
|
||||
- [ ] `NewRouter(db *pgxpool.Pool) http.Handler` using chi
|
||||
- [ ] Mount all routes under `/api`
|
||||
- [ ] Middleware:
|
||||
- [ ] `chi.Logger` — request logging
|
||||
- [ ] `chi.Recoverer` — panic recovery → 500 JSON error
|
||||
- [ ] `requestID` middleware
|
||||
- [ ] CORS headers (configurable origin, default `*`)
|
||||
|
||||
---
|
||||
|
||||
## 5.3 Source Routes
|
||||
|
||||
### `GET /api/sources`
|
||||
- [ ] Call `registry.All()`
|
||||
- [ ] Return `[]{id, name, lang, supportsLatest, isNsfw}` sorted by ID
|
||||
- [ ] No DB call required (registry is in-memory)
|
||||
|
||||
### `GET /api/sources/{id}/popular?page=1`
|
||||
- [ ] Parse `id` as int64, `page` as int (default 1), `refresh` as bool
|
||||
- [ ] Validate source exists in registry
|
||||
- [ ] Check manga list TTL (see Phase 2 cache logic)
|
||||
- [ ] If cache miss or `?refresh=true`: call `source.GetPopularManga(page)`, upsert results
|
||||
- [ ] Return `{mangas: [...], hasNextPage: bool}`
|
||||
|
||||
### `GET /api/sources/{id}/latest?page=1`
|
||||
- [ ] Same as popular but calls `source.GetLatestUpdates(page)`
|
||||
- [ ] Return 405 if `source.SupportsLatest()` is false
|
||||
|
||||
### `GET /api/sources/{id}/search?q=&page=1`
|
||||
- [ ] Parse `q` (query string), `page`, `refresh`
|
||||
- [ ] Pass `filters` from query params (see Filter parsing below)
|
||||
- [ ] Call `source.GetSearchManga(page, q, filters)`
|
||||
- [ ] Upsert results, return `{mangas: [...], hasNextPage: bool}`
|
||||
|
||||
### `GET /api/sources/{id}/filters`
|
||||
- [ ] Call `source.GetFilterList()`
|
||||
- [ ] Return filter descriptors as JSON: `[{type, name, values}]`
|
||||
- [ ] No DB involvement
|
||||
|
||||
### Filter Param Parsing
|
||||
- [ ] Parse filter values from query params: `filter_{name}={value}`
|
||||
- [ ] `SelectFilter`: single string value
|
||||
- [ ] `CheckboxFilter`: "true"/"false" → bool
|
||||
- [ ] `TriStateFilter`: "0"/"1"/"2" → int
|
||||
- [ ] `TextFilter`: raw string
|
||||
- [ ] `SortFilter`: `{name}:{asc|desc}`
|
||||
|
||||
---
|
||||
|
||||
## 5.4 Manga Routes (Source-aware)
|
||||
|
||||
### `GET /api/sources/{id}/manga?url={encodedURL}`
|
||||
- [ ] URL-decode `url` param
|
||||
- [ ] Look up manga in DB by `(source_id, url)` — `GetMangaBySourceURL`
|
||||
- [ ] If not found or `initialized=false` or TTL expired or `?refresh=true`:
|
||||
- [ ] Call `source.GetMangaDetails(SManga{URL: url})`
|
||||
- [ ] `UpsertManga` + `UpdateMangaDetails` + set `initialized=true`
|
||||
- [ ] Return full manga JSON (all fields)
|
||||
|
||||
### `GET /api/sources/{id}/manga?url={encodedURL}&resource=chapters`
|
||||
OR `GET /api/sources/{id}/chapters?url={encodedURL}`
|
||||
- [ ] Look up manga_id from DB
|
||||
- [ ] Check `chapters_last_fetched_at` TTL
|
||||
- [ ] If miss: call `source.GetChapterList(manga)`, `UpsertChapter` for each
|
||||
- [ ] Return `[{url, name, chapterNumber, dateUpload, scanlator, sourceOrder}]`
|
||||
|
||||
### `GET /api/sources/{id}/pages?chapterUrl={encodedURL}`
|
||||
- [ ] Look up chapter by URL
|
||||
- [ ] Check if pages exist in DB for chapter
|
||||
- [ ] If miss: call `source.GetPageList(chapter)`, then optionally `source.GetImageURL(page)` for each
|
||||
- [ ] `UpsertPage` for each; store `image_url` (except sources with expiring URLs)
|
||||
- [ ] Return `[{index, url, imageUrl}]`
|
||||
|
||||
---
|
||||
|
||||
## 5.5 DB-Backed Routes (no source call)
|
||||
|
||||
### `GET /api/manga/{id}`
|
||||
- [ ] `GetMangaByID` from DB
|
||||
- [ ] Return manga JSON or 404
|
||||
|
||||
### `GET /api/manga/{id}/chapters`
|
||||
- [ ] `ListChaptersByManga` from DB
|
||||
- [ ] Return chapter array ordered by `source_order`
|
||||
|
||||
### `GET /api/chapters/{id}`
|
||||
- [ ] `GetChapterByID` from DB
|
||||
- [ ] Return chapter JSON or 404
|
||||
|
||||
### `GET /api/chapters/{id}/pages`
|
||||
- [ ] `ListPagesByChapter` from DB
|
||||
- [ ] Return page array ordered by `index`
|
||||
|
||||
---
|
||||
|
||||
## 5.6 Image Proxy — `GET /api/image?url={encodedURL}&source_id={id}`
|
||||
|
||||
- [ ] URL-decode `url` param
|
||||
- [ ] Validate `source_id` exists in registry
|
||||
- [ ] Fetch image using source's configured HTTP client (respects source-specific headers/cookies)
|
||||
- [ ] Set `Referer` if required (e.g. Pixiv)
|
||||
- [ ] Use FlareSolverr-cleared cookies if source needs CF bypass
|
||||
- [ ] Stream response bytes directly to client
|
||||
- [ ] Forward `Content-Type` header from upstream
|
||||
- [ ] Set `Cache-Control: public, max-age=86400` for non-expiring images
|
||||
- [ ] **Do not** set long cache for signed/expiring URLs (globalcomix, etc.)
|
||||
- [ ] Return 502 on upstream error
|
||||
|
||||
---
|
||||
|
||||
## 5.7 Error Handling
|
||||
|
||||
- [ ] All errors: `{"error": "human-readable message"}` with correct HTTP status
|
||||
- [ ] 400: invalid params (bad ID, missing required param, bad page number)
|
||||
- [ ] 404: source not found, manga not found, chapter not found
|
||||
- [ ] 405: method not supported (e.g. `GetLatestUpdates` on source with `SupportsLatest=false`)
|
||||
- [ ] 500: DB error, unexpected source error
|
||||
- [ ] 502: upstream source returned error (connection refused, non-200, parse failure)
|
||||
- [ ] 503: FlareSolverr unavailable (when source requires it but `FLARESOLVERR_URL` is unset)
|
||||
- [ ] Structured error logging: include `source_id`, `url`, `error` in log fields
|
||||
|
||||
---
|
||||
|
||||
## 5.8 `?refresh=true` Bypass
|
||||
|
||||
- [ ] Accepted on all source-facing routes: `/popular`, `/latest`, `/search`, `/manga`, `/chapters`, `/pages`
|
||||
- [ ] Bypasses TTL — always calls source regardless of `last_fetched_at`
|
||||
- [ ] Updates DB with fresh data after fetch
|
||||
- [ ] NOT accepted on DB-only routes (`/api/manga/{id}`, `/api/chapters/{id}`, etc.)
|
||||
|
||||
---
|
||||
|
||||
## Checklist: Phase 5 Done When
|
||||
|
||||
- [ ] `GET /api/sources` returns all registered sources
|
||||
- [ ] `GET /api/sources/{heancmsID}/popular?page=1` returns ≥1 manga; rows in `manga` table confirmed via `psql`
|
||||
- [ ] Second call to same URL returns from DB (no source HTTP call — confirm via log)
|
||||
- [ ] `?refresh=true` forces re-fetch and updates `last_fetched_at` in DB
|
||||
- [ ] `GET /api/sources/{madaraID}/pages?chapterUrl=...` resolves image URLs via FlareSolverr
|
||||
- [ ] `GET /api/image?url=...&source_id=...` proxies image bytes with correct `Content-Type`
|
||||
- [ ] 404 response for unknown source ID: `{"error": "source not found"}`
|
||||
- [ ] 405 response when `GetLatestUpdates` called on source with `SupportsLatest=false`
|
||||
- [ ] `go build ./cmd/server/` succeeds
|
||||
- [ ] Server starts, logs migration version, logs registered source count
|
||||
Reference in New Issue
Block a user