goyomi/docs/phase1-core-framework.md

Phase 1 — Core Framework

All foundational types, interfaces, and helpers that every source implementation depends on. Nothing in sources/ can be written until this phase is complete.

Reference:

• Tachiyomi source contract: /Users/achmad/Documents/Belajar/Android/extensions-source/core/src/ (HttpSource, CatalogueSource)
• Kotlin originals: /Users/achmad/Documents/Belajar/Android/extensions-source/lib-multisrc/madara/src/eu/kanade/tachiyomi/multisrc/madara/Madara.kt
• Suwayomi source interface: /Users/achmad/Documents/Belajar/Web/Suwayomi-Server/server/src/main/kotlin/suwayomi/tachidesk/manga/impl/

---

1.1 Data Types — internal/source/types.go

• SManga struct
  • URL, Title, Artist, Author, Description string fields
  • Genre string (comma-separated, matches Tachiyomi convention)
  • Status int — 0=unknown, 1=ongoing, 2=completed, 3=licensed, 5=hiatus, 6=cancelled
  • ThumbnailURL string
  • Initialized bool (false until GetMangaDetails has been called)
• SChapter struct
  • URL, Name, Scanlator string fields
  • DateUpload int64 — unix milliseconds (matches Tachiyomi)
  • ChapterNumber float32
• Page struct
  • Index int
  • URL string — page HTML URL or chapter URL
  • ImageURL string — direct image URL (may be empty until GetImageURL resolves it)
• MangasPage struct
  • Mangas []SManga
  • HasNextPage bool
• Filter types
  • Filter interface with Name() and Value() methods
  • TextFilter — free-text input
  • CheckboxFilter — boolean
  • TriStateFilter — ignore / include / exclude (0/1/2)
  • SelectFilter — dropdown with named options (name + values []string)
  • SortFilter — sortable list with ascending/descending state
  • GroupFilter — container of sub-filters
• Status constants (StatusUnknown, StatusOngoing, etc.)

---

1.2 Source Interfaces — internal/source/interfaces.go

• Source interface (base)
  • ID() int64
  • Name() string
  • Lang() string
• CatalogueSource interface (embeds Source)
  • SupportsLatest() bool
  • GetPopularManga(page int) (MangasPage, error)
  • GetLatestUpdates(page int) (MangasPage, error)
  • GetSearchManga(page int, query string, filters []Filter) (MangasPage, error)
  • GetMangaDetails(manga SManga) (SManga, error)
  • GetChapterList(manga SManga) ([]SChapter, error)
  • GetPageList(chapter SChapter) ([]Page, error)
  • GetImageURL(page Page) (string, error) — no-op default for sources that embed image URLs directly in pages
  • GetFilterList() []Filter
• ID generation helper
  • GenerateSourceID(name, lang string) int64 — MD5 of "${name.lowercase()}/$lang/1", first 8 bytes big-endian, sign bit cleared. Matches Suwayomi HttpSource.generateId. (The original plan incorrectly described Java String.hashCode().)
  • Unit test against computed IDs (MangaDex/en, MangaDex/all, HeanCms/en)

---

1.3 HTTP Client — internal/httpclient/client.go

• Client struct wrapping *http.Client
  • Persistent cookiejar.Jar (one per source instance)
  • Per-host rate.Limiter map (keyed by Host) with sync.Mutex
  • Configurable RateLimit (requests/sec) and Burst per source
  • Configurable Timeout (default 30s)
  • Default User-Agent header (Chrome/Android UA, matches Tachiyomi)
• NewClient(opts ...Option) *Client constructor with functional options
• Do(req *http.Request) (*http.Response, error) — applies rate limit before executing
• Get(ctx context.Context, url string, headers map[string]string) (*http.Response, error) helper
• Post(ctx context.Context, url string, body io.Reader, contentType string, headers map[string]string) (*http.Response, error) helper
• 429 retry logic
  • Read Retry-After header (seconds or HTTP-date)
  • Sleep and retry up to N times (configurable, default 3)
• Optional Referer injection (configurable per client)

---

1.4 FlareSolverr — internal/httpclient/flaresolverr.go

• FlareSolverrClient struct
  • endpoint string (FlareSolverr v1 base URL, e.g. http://localhost:8191)
  • Underlying *http.Client for talking to FlareSolverr itself
• Get(ctx context.Context, url string, cookies []Cookie) (html string, cookies []Cookie, err error)
  • POST {"cmd":"request.get","url":"...","maxTimeout":60000} to /v1
  • Extract solution.response (rendered HTML body)
  • Extract solution.cookies → convert to []*http.Cookie
• Cookie reuse strategy — cookies returned to caller; caller injects into source's cookie jar
• FlareSolverrResponse JSON struct (full response shape)
• Config: FLARESOLVERR_URL env var (disable if empty — NewFlareSolverrClient returns error)

---

1.5 GraphQL Helper — internal/httpclient/graphql.go

Used only to talk to upstream sources that expose GraphQL (mangahub, senkuro, allanime, luscious, stashapp). Our own API is plain REST.

• GraphQLRequest struct — Query string, Variables any
• Post[T any](ctx context.Context, client *http.Client, url string, req GraphQLRequest, headers map[string]string) (T, error)
  • Marshal request to JSON
  • POST with Content-Type: application/json
  • Unmarshal data field of response into T
  • Surface errors[] as a Go error if present
• graphQLResponse[T any] envelope struct

---

1.6 Header Builders — internal/httpclient/headers.go

• AndroidUA() string — Chrome/Android user-agent string (matches Tachiyomi default)
• DesktopUA() string — desktop Chrome user-agent (for sources requiring desktop)
• JSONHeaders() map[string]string — Content-Type: application/json, Accept: application/json
• FormHeaders() map[string]string — Content-Type: application/x-www-form-urlencoded
• WithRefererHeader(headers map[string]string, referer string) map[string]string
• WithOrigin(headers map[string]string, origin string) map[string]string

---

1.7 HTML Parser — internal/parser/html.go

Thin wrappers over github.com/PuerkitoBio/goquery (Go equivalent of JSoup used in Tachiyomi extensions).

• Parse(html string) (*goquery.Document, error) — parse raw HTML string
• ParseResponse(resp *http.Response) (*goquery.Document, error) — parse from HTTP response body
• Select(doc *goquery.Document, css string) *goquery.Selection
• SelectFrom(sel *goquery.Selection, css string) *goquery.Selection
• Attr(sel *goquery.Selection, name string) string — returns empty string if not found
• AbsURL(sel *goquery.Selection, attr string, baseURL string) string — resolves relative URLs
• OwnText(sel *goquery.Selection) string — text of element excluding child elements
• TextTrim(sel *goquery.Selection) string — .Text() with strings.TrimSpace
• First(sel *goquery.Selection) *goquery.Selection
• Each(sel *goquery.Selection, fn func(i int, s *goquery.Selection)) — convenience wrapper

---

1.8 Registry — internal/registry/registry.go

• Package-level map[int64]source.CatalogueSource protected by sync.RWMutex
• Register(s source.CatalogueSource) — panics on duplicate ID (caught at startup)
• Get(id int64) (source.CatalogueSource, bool)
• All() []source.CatalogueSource — returns sorted slice (by ID) for deterministic listing
• Each source package must call registry.Register(NewXxx()) in its init() function
• All source packages blank-imported in cmd/server/main.go (done when sources exist)

---

1.9 Docker — Dockerfile + compose.yml

• Dockerfile — multi-stage build (golang:1.26-alpine builder + distroless/static-debian12 runtime)
• compose.yml — services: app, postgres, flaresolverr
  • postgres — official image, healthcheck, named volume
  • flaresolverr — ghcr.io/flaresolverr/flaresolverr, depends on nothing
  • app — builds from Dockerfile, depends on postgres healthcheck, env vars wired

---

Checklist: Phase 1 Done When

• go build ./internal/... succeeds with no errors
• GenerateSourceID matches Tachiyomi IDs for at least 3 known sources
• FlareSolverrClient.Get returns rendered HTML for a Cloudflare-protected URL (manual test — requires running FlareSolverr)
• GraphQLPost works against a public GraphQL endpoint (manual test)
• Registry panics on duplicate source ID (unit test)