# Phase 1 — Core Framework All foundational types, interfaces, and helpers that every source implementation depends on. Nothing in `sources/` can be written until this phase is complete. Reference: - Tachiyomi source contract: `/Users/achmad/Documents/Belajar/Android/extensions-source/core/src/` (HttpSource, CatalogueSource) - Kotlin originals: `/Users/achmad/Documents/Belajar/Android/extensions-source/lib-multisrc/madara/src/eu/kanade/tachiyomi/multisrc/madara/Madara.kt` - Suwayomi source interface: `/Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/impl/` --- ## 1.1 Data Types — `internal/source/types.go` - [x] `SManga` struct - [x] `URL`, `Title`, `Artist`, `Author`, `Description` string fields - [x] `Genre` string (comma-separated, matches Tachiyomi convention) - [x] `Status` int — 0=unknown, 1=ongoing, 2=completed, 3=licensed, 5=hiatus, 6=cancelled - [x] `ThumbnailURL` string - [x] `Initialized` bool (false until `GetMangaDetails` has been called) - [x] `SChapter` struct - [x] `URL`, `Name`, `Scanlator` string fields - [x] `DateUpload` int64 — unix milliseconds (matches Tachiyomi) - [x] `ChapterNumber` float32 - [x] `Page` struct - [x] `Index` int - [x] `URL` string — page HTML URL or chapter URL - [x] `ImageURL` string — direct image URL (may be empty until `GetImageURL` resolves it) - [x] `MangasPage` struct - [x] `Mangas []SManga` - [x] `HasNextPage bool` - [x] Filter types - [x] `Filter` interface with `Name()` and `Value()` methods - [x] `TextFilter` — free-text input - [x] `CheckboxFilter` — boolean - [x] `TriStateFilter` — ignore / include / exclude (0/1/2) - [x] `SelectFilter` — dropdown with named options (name + values []string) - [x] `SortFilter` — sortable list with ascending/descending state - [x] `GroupFilter` — container of sub-filters - [x] Status constants (`StatusUnknown`, `StatusOngoing`, etc.) --- ## 1.2 Source Interfaces — `internal/source/interfaces.go` - [x] `Source` interface (base) - [x] `ID() int64` - [x] `Name() string` - [x] `Lang() string` - [x] `CatalogueSource` interface (embeds Source) - [x] `SupportsLatest() bool` - [x] `GetPopularManga(page int) (MangasPage, error)` - [x] `GetLatestUpdates(page int) (MangasPage, error)` - [x] `GetSearchManga(page int, query string, filters []Filter) (MangasPage, error)` - [x] `GetMangaDetails(manga SManga) (SManga, error)` - [x] `GetChapterList(manga SManga) ([]SChapter, error)` - [x] `GetPageList(chapter SChapter) ([]Page, error)` - [x] `GetImageURL(page Page) (string, error)` — no-op default for sources that embed image URLs directly in pages - [x] `GetFilterList() []Filter` - [x] ID generation helper - [x] `GenerateSourceID(name, lang string) int64` — MD5 of `"${name.lowercase()}/$lang/1"`, first 8 bytes big-endian, sign bit cleared. Matches Suwayomi `HttpSource.generateId`. (The original plan incorrectly described Java `String.hashCode()`.) - [x] Unit test against computed IDs (MangaDex/en, MangaDex/all, HeanCms/en) --- ## 1.3 HTTP Client — `internal/httpclient/client.go` - [x] `Client` struct wrapping `*http.Client` - [x] Persistent `cookiejar.Jar` (one per source instance) - [x] Per-host `rate.Limiter` map (keyed by `Host`) with `sync.Mutex` - [x] Configurable `RateLimit` (requests/sec) and `Burst` per source - [x] Configurable `Timeout` (default 30s) - [x] Default `User-Agent` header (Chrome/Android UA, matches Tachiyomi) - [x] `NewClient(opts ...Option) *Client` constructor with functional options - [x] `Do(req *http.Request) (*http.Response, error)` — applies rate limit before executing - [x] `Get(ctx context.Context, url string, headers map[string]string) (*http.Response, error)` helper - [x] `Post(ctx context.Context, url string, body io.Reader, contentType string, headers map[string]string) (*http.Response, error)` helper - [x] 429 retry logic - [x] Read `Retry-After` header (seconds or HTTP-date) - [x] Sleep and retry up to N times (configurable, default 3) - [x] Optional `Referer` injection (configurable per client) --- ## 1.4 FlareSolverr — `internal/httpclient/flaresolverr.go` - [x] `FlareSolverrClient` struct - [x] `endpoint` string (FlareSolverr v1 base URL, e.g. `http://localhost:8191`) - [x] Underlying `*http.Client` for talking to FlareSolverr itself - [x] `Get(ctx context.Context, url string, cookies []Cookie) (html string, cookies []Cookie, err error)` - [x] POST `{"cmd":"request.get","url":"...","maxTimeout":60000}` to `/v1` - [x] Extract `solution.response` (rendered HTML body) - [x] Extract `solution.cookies` → convert to `[]*http.Cookie` - [x] Cookie reuse strategy — cookies returned to caller; caller injects into source's cookie jar - [x] `FlareSolverrResponse` JSON struct (full response shape) - [x] Config: `FLARESOLVERR_URL` env var (disable if empty — `NewFlareSolverrClient` returns error) --- ## 1.5 GraphQL Helper — `internal/httpclient/graphql.go` Used only to talk to upstream sources that expose GraphQL (mangahub, senkuro, allanime, luscious, stashapp). Our own API is plain REST. - [x] `GraphQLRequest` struct — `Query string`, `Variables any` - [x] `Post[T any](ctx context.Context, client *http.Client, url string, req GraphQLRequest, headers map[string]string) (T, error)` - [x] Marshal request to JSON - [x] POST with `Content-Type: application/json` - [x] Unmarshal `data` field of response into T - [x] Surface `errors[]` as a Go error if present - [x] `graphQLResponse[T any]` envelope struct --- ## 1.6 Header Builders — `internal/httpclient/headers.go` - [x] `AndroidUA() string` — Chrome/Android user-agent string (matches Tachiyomi default) - [x] `DesktopUA() string` — desktop Chrome user-agent (for sources requiring desktop) - [x] `JSONHeaders() map[string]string` — `Content-Type: application/json`, `Accept: application/json` - [x] `FormHeaders() map[string]string` — `Content-Type: application/x-www-form-urlencoded` - [x] `WithRefererHeader(headers map[string]string, referer string) map[string]string` - [x] `WithOrigin(headers map[string]string, origin string) map[string]string` --- ## 1.7 HTML Parser — `internal/parser/html.go` Thin wrappers over `github.com/PuerkitoBio/goquery` (Go equivalent of JSoup used in Tachiyomi extensions). - [x] `Parse(html string) (*goquery.Document, error)` — parse raw HTML string - [x] `ParseResponse(resp *http.Response) (*goquery.Document, error)` — parse from HTTP response body - [x] `Select(doc *goquery.Document, css string) *goquery.Selection` - [x] `SelectFrom(sel *goquery.Selection, css string) *goquery.Selection` - [x] `Attr(sel *goquery.Selection, name string) string` — returns empty string if not found - [x] `AbsURL(sel *goquery.Selection, attr string, baseURL string) string` — resolves relative URLs - [x] `OwnText(sel *goquery.Selection) string` — text of element excluding child elements - [x] `TextTrim(sel *goquery.Selection) string` — `.Text()` with strings.TrimSpace - [x] `First(sel *goquery.Selection) *goquery.Selection` - [x] `Each(sel *goquery.Selection, fn func(i int, s *goquery.Selection))` — convenience wrapper --- ## 1.8 Registry — `internal/registry/registry.go` - [x] Package-level `map[int64]source.CatalogueSource` protected by `sync.RWMutex` - [x] `Register(s source.CatalogueSource)` — panics on duplicate ID (caught at startup) - [x] `Get(id int64) (source.CatalogueSource, bool)` - [x] `All() []source.CatalogueSource` — returns sorted slice (by ID) for deterministic listing - [x] Each source package must call `registry.Register(NewXxx())` in its `init()` function - [ ] All source packages blank-imported in `cmd/server/main.go` (done when sources exist) --- ## 1.9 Docker — `Dockerfile` + `compose.yml` - [x] `Dockerfile` — multi-stage build (`golang:1.26-alpine` builder + `distroless/static-debian12` runtime) - [x] `compose.yml` — services: `app`, `postgres`, `flaresolverr` - [x] `postgres` — official image, healthcheck, named volume - [x] `flaresolverr` — ghcr.io/flaresolverr/flaresolverr, depends on nothing - [x] `app` — builds from `Dockerfile`, depends on postgres healthcheck, env vars wired --- ## Checklist: Phase 1 Done When - [x] `go build ./internal/...` succeeds with no errors - [x] `GenerateSourceID` matches Tachiyomi IDs for at least 3 known sources - [ ] `FlareSolverrClient.Get` returns rendered HTML for a Cloudflare-protected URL (manual test — requires running FlareSolverr) - [ ] `GraphQLPost` works against a public GraphQL endpoint (manual test) - [x] Registry panics on duplicate source ID (unit test)