achmad
6953aa7833
Document the unified httpclient.Client design, FlareSolverr integration, cookie jar sharing, and Kotlin vs Go mapping.
797 lines
24 KiB
Markdown
Executable File
797 lines
24 KiB
Markdown
Executable File
# Phase 4 — Standalone Sources
|
|
|
|
Complete port checklist. Check a box when the source passes a basic smoke test
|
|
(popular/latest list returns ≥1 result, or detail+pages resolve for a known URL).
|
|
|
|
**After implementing each source:** mark its checkbox `[x]` in this doc and add it to the import list in `cmd/server/main.go`.
|
|
|
|
## Important: Source Types
|
|
|
|
### Base Sources (`lib-multisrc`)
|
|
Shared implementations in:
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/lib-multisrc/`
|
|
- `/Users/achmad/Documents/Belajar/Web/goyomi/sources/base/`
|
|
|
|
These are the core implementations that standalone sources may use.
|
|
|
|
### Standalone Sources (`all/` and `en/`)
|
|
Individual extensions in:
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/all/`
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/en/`
|
|
|
|
These may or may not use a base source from lib-multisrc — and even when they do, they often add their own
|
|
implementation on top (overriding methods, adding extra parsing, custom filters, etc.).
|
|
When porting, check if the Kotlin source extends a base class (e.g., `MangaReader`, `Madara`, `ComicGeek`) —
|
|
if so, use the corresponding Go base implementation as the foundation, then layer the source-specific overrides.
|
|
Otherwise, implement fully standalone.
|
|
|
|
Reference:
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/all/`
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/en/`
|
|
|
|
## HTTP Client Architecture
|
|
|
|
### Cookie Jar
|
|
|
|
The `httpclient.Client` creates a `cookiejar.Jar` for every instance. All cookies from
|
|
direct HTTP responses and from FlareSolverr are stored in this shared jar. Sources can
|
|
read cookies from the jar using the `Cookie(name, host)` method:
|
|
|
|
```go
|
|
// After a request, check the jar for a specific cookie (like Kotlin's
|
|
// client.cookieJar.loadForRequest()):
|
|
value := client.Cookie("mhub_access", "mangahub.io")
|
|
```
|
|
|
|
### Response Headers from FlareSolverr
|
|
|
|
When `Do()` falls back to FlareSolverr, the `doFS()` method now properly propagates
|
|
the actual response headers from FlareSolverr (including `Set-Cookie`) instead of
|
|
copying request headers into the response. Cookies from FS are also explicitly added
|
|
as `Set-Cookie` headers and fed into the shared cookie jar.
|
|
|
|
### Single Unified Client
|
|
|
|
All sources share one unified HTTP client (`internal/httpclient.Client`) that handles both
|
|
direct requests and Cloudflare/DDoS bypass transparently:
|
|
|
|
```
|
|
httpclient.DefaultClient()
|
|
├── Direct HTTP (net/http + cookie jar + rate limiter)
|
|
└── FlareSolverr fallback (auto-detected from FLARESOLVERR_URL env var)
|
|
```
|
|
|
|
The `Do(req)` method implements **adaptive logic** matching the Kotlin `cloudflareClient`:
|
|
|
|
1. **Try direct** — normal HTTP request with shared cookie jar + rate limiting
|
|
2. **If 403/503** (Cloudflare/DDoS challenge) — falls back to **FlareSolverr raw mode**
|
|
3. **FlareSolverr solves the challenge** — returns actual server response + cookies
|
|
4. **Cookies fed into shared jar** — subsequent requests to the same host skip FS
|
|
5. **Chrome HTML wrapper stripped** — FlareSolverr wraps responses in
|
|
`<html><head>...<meta...></head><body><pre>body</pre></body></html>` — the client
|
|
strips this wrapper so callers receive the actual server body (JSON or HTML).
|
|
|
|
### Kotlin vs Go Mapping
|
|
|
|
| Kotlin (`cloudflareClient`) | Go (`httpclient.Client`) |
|
|
|---|---|
|
|
| `OkHttpClient` with Cloudflare interceptor | `net/http.Client` with FlareSolverr fallback |
|
|
| Intercepts 503 → solves via WebView → retries with cookies | Tries direct first → on 403/503 → FS raw → strips wrapper |
|
|
| Returns raw server response body | Returns stripped FS body or direct body |
|
|
| Shared across all sources | `DefaultClient()` singleton shared across all sources |
|
|
|
|
### How Sources Use the Client
|
|
|
|
```go
|
|
import "goyomi/internal/httpclient"
|
|
|
|
// Default: use shared singleton (preferred for most sources)
|
|
func (s *Source) fetch(ctx context.Context) error {
|
|
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
|
req.Header.Set("Accept", "application/json")
|
|
resp, err := httpclient.DefaultClient().Do(req)
|
|
// ...
|
|
}
|
|
|
|
// Custom rate limit: create a dedicated client
|
|
func New() *Source {
|
|
c := httpclient.NewClient(httpclient.WithRateLimit(5, 10))
|
|
return &Source{client: c}
|
|
}
|
|
|
|
// Custom headers: set on the request (no need for a custom client)
|
|
func (s *Source) fetch(ctx context.Context) error {
|
|
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
|
req.Header.Set("Accept", "text/css") // DDOS-Guard bypass
|
|
resp, err := httpclient.DefaultClient().Do(req)
|
|
}
|
|
```
|
|
|
|
### FlareSolverr Integration
|
|
|
|
FlareSolverr is auto-configured from the `FLARESOLVERR_URL` environment variable.
|
|
If unset, the client works in direct-only mode (no Cloudflare bypass).
|
|
|
|
The `flare` package (`internal/httpclient/flare/`) is a backward-compatible shim —
|
|
all it does is alias `httpclient.Client` and `httpclient.NewClient`. Sources that
|
|
imported `flare` before continue to compile without changes.
|
|
|
|
### Design Decisions
|
|
|
|
1. **Why a singleton?** — Shared cookie jar means cookies from FlareSolverr (solved
|
|
challenges) benefit all sources. Shared rate limiter prevents hammering the same host.
|
|
2. **Why strip FS wrapper?** — FlareSolverr routes requests through Chrome, which wraps
|
|
JSON responses in `<html><head>...<pre>JSON</pre>...`. Without stripping, JSON parsers
|
|
fail. For HTML responses, Chrome adds `<meta charset>` tags but the page content
|
|
remains parseable by goquery.
|
|
3. **Why direct first?** — Most sites don't have Cloudflare. Direct requests are faster
|
|
(no Chrome overhead) and return clean response bodies. FS is only invoked on actual
|
|
challenge responses (403/503).
|
|
4. **Why not Chrome rendering?** — The Kotlin `cloudflareClient` does NOT render through
|
|
Chrome. It intercepts 503, solves the challenge, and returns the raw server response.
|
|
Our approach matches this behavior.
|
|
|
|
Detailed implementation notes for complex sources are in the **Notes** section at the bottom.
|
|
|
|
---
|
|
|
|
## `sources/all/` — 125 sources
|
|
|
|
- [x] `all/ahottie`
|
|
- [x] `all/akuma`
|
|
- [x] `all/allporncomicsco`
|
|
- [x] `all/asmhentai`
|
|
- [x] `all/baobua`
|
|
- [x] `all/beauty3600000`
|
|
- [x] `all/buondua`
|
|
- [x] `all/comicfury`
|
|
- [x] `all/comicgrowl`
|
|
- [x] `all/comicklive`
|
|
- [ ] `all/comicskingdom` ⚠️ no Kotlin reference found — skip
|
|
- [x] `all/comicsvalley`
|
|
- [x] `all/comikey`
|
|
- [x] `all/commitstrip`
|
|
- [x] `all/coomer`
|
|
- [ ] `all/coronaex`
|
|
- [ ] `all/cosplaytele`
|
|
- [ ] `all/cubari`
|
|
- [ ] `all/danbooru` ⚠️ see notes
|
|
- [ ] `all/deviantart`
|
|
- [ ] `all/dragonballmultiverse`
|
|
- [ ] `all/e621` ⚠️ see notes
|
|
- [x] `all/elitebabes`
|
|
- [ ] `all/everiaclub`
|
|
- [ ] `all/everiaclubcom`
|
|
- [x] `all/femjoyhunter`
|
|
- [ ] `all/foamgirl`
|
|
- [ ] `all/foolslidecustomizable`
|
|
- [ ] `all/fourkhd`
|
|
- [x] `all/ftvhunter`
|
|
- [ ] `all/fuwayomi` ⚠️ skip — server-side Fuwayomi integration, not a scraper
|
|
- [ ] `all/globalcomix` ⚠️ see notes
|
|
- [ ] `all/grabberzone`
|
|
- [ ] `all/hdoujin`
|
|
- [ ] `all/hennojin`
|
|
- [ ] `all/hentai3`
|
|
- [ ] `all/hentaicosplay`
|
|
- [ ] `all/hentaienvy`
|
|
- [ ] `all/hentaiera`
|
|
- [ ] `all/hentaifox`
|
|
- [x] `all/hentaihand`
|
|
- [ ] `all/hentairox`
|
|
- [ ] `all/hentaizap`
|
|
- [x] `all/hniscantrad`
|
|
- [ ] `all/holonometria`
|
|
- [ ] `all/honeytoon`
|
|
- [ ] `all/imhentai`
|
|
- [ ] `all/izneo`
|
|
- [ ] `all/jjcos`
|
|
- [x] `all/joymiihub`
|
|
- [ ] `all/junmeitu`
|
|
- [x] `all/kemono` ⚠️ see notes
|
|
- [ ] `all/kiutaku`
|
|
- [ ] `all/kodokustudio`
|
|
- [ ] `all/koharu`
|
|
- [ ] `all/komga` ⚠️ see notes
|
|
- [ ] `all/lanraragi`
|
|
- [ ] `all/leagueoflegends`
|
|
- [ ] `all/lunaranime`
|
|
- [ ] `all/luscious` ⚠️ see notes
|
|
- [ ] `all/magicaltranslators`
|
|
- [ ] `all/manga18me`
|
|
- [ ] `all/mangaball`
|
|
- [x] `all/mangacrazy`
|
|
- [ ] `all/mangadex` ⚠️ see notes
|
|
- [ ] `all/mangadraft`
|
|
- [ ] `all/mangafire`
|
|
- [ ] `all/mangaforfree`
|
|
- [ ] `all/mangaplus` ⚠️ see notes
|
|
- [ ] `all/mangapluscreators`
|
|
- [x] `all/mangataro`
|
|
- [ ] `all/mangatoon`
|
|
- [ ] `all/mangaup`
|
|
- [ ] `all/mango`
|
|
- [ ] `all/manhuarm`
|
|
- [ ] `all/manhwa18cc`
|
|
- [ ] `all/manhwa18net`
|
|
- [ ] `all/manhwa18uncensored`
|
|
- [ ] `all/manhwaclubnet`
|
|
- [ ] `all/manhwadashraw`
|
|
- [ ] `all/mayotune`
|
|
- [x] `all/metarthunter`
|
|
- [ ] `all/miauscan`
|
|
- [ ] `all/misskon`
|
|
- [ ] `all/mitaku`
|
|
- [ ] `all/myreadingmanga`
|
|
- [ ] `all/namicomi`
|
|
- [ ] `all/nhentaicom` ⚠️ see notes
|
|
- [ ] `all/nhentaixxx`
|
|
- [ ] `all/niadd`
|
|
- [ ] `all/ninemanga`
|
|
- [ ] `all/novelcool`
|
|
- [ ] `all/ososedki`
|
|
- [ ] `all/pandachaika`
|
|
- [ ] `all/peppercarrot`
|
|
- [ ] `all/photos18`
|
|
- [ ] `all/pixiv` ⚠️ see notes
|
|
- [x] `all/playmatehunter`
|
|
- [ ] `all/pornpics`
|
|
- [ ] `all/projectsuki`
|
|
- [ ] `all/qtoon`
|
|
- [ ] `all/rokuhentai`
|
|
- [ ] `all/sakuramanhwa`
|
|
- [ ] `all/sandraandwoo`
|
|
- [ ] `all/seraphicdeviltry`
|
|
- [ ] `all/simplycosplay`
|
|
- [ ] `all/simplyhentai`
|
|
- [ ] `all/stashapp` ⚠️ see notes
|
|
- [ ] `all/taddyink`
|
|
- [ ] `all/tappytoon`
|
|
- [ ] `all/thelibraryofohara`
|
|
- [ ] `all/thunderscans`
|
|
- [ ] `all/toomics`
|
|
- [ ] `all/twicomi`
|
|
- [ ] `all/uncensoredmanhwa`
|
|
- [ ] `all/vinnieVeritas`
|
|
- [ ] `all/webtoons` ⚠️ see notes
|
|
- [x] `all/xarthunter`
|
|
- [ ] `all/xasiatalbums`
|
|
- [ ] `all/xgmn`
|
|
- [ ] `all/xinmeitulu`
|
|
- [ ] `all/xiutaku`
|
|
- [ ] `all/xkcd`
|
|
- [ ] `all/yabai`
|
|
- [ ] `all/yaoimangaonline`
|
|
- [ ] `all/yellownote`
|
|
- [ ] `all/yskcomics`
|
|
|
|
---
|
|
|
|
## `sources/en/` — 430 sources
|
|
|
|
- [ ] `en/akaicomic`
|
|
- [ ] `en/alandal`
|
|
- [ ] `en/allanime` ⚠️ see notes
|
|
- [ ] `en/allporncomic`
|
|
- [ ] `en/allporncomicio`
|
|
- [ ] `en/anisascans`
|
|
- [ ] `en/apcomics`
|
|
- [x] `en/aquamanga`
|
|
- [x] `en/arcrelight` → base: MangAdventure
|
|
- [x] `en/arenascans`
|
|
- [ ] `en/armageddon`
|
|
- [x] `en/artlapsa`
|
|
- [ ] `en/arvencomics`
|
|
- [ ] `en/arvenscans`
|
|
- [ ] `en/aryascans`
|
|
- [ ] `en/asiatoon`
|
|
- [ ] `en/asmotoon`
|
|
- [x] `en/assortedscans`
|
|
- [ ] `en/asurascans` ⚠️ see notes
|
|
- [ ] `en/athreascans`
|
|
- [ ] `en/atsumaru` → standalone HttpSource
|
|
- [ ] `en/aurora` → standalone HttpSource
|
|
- [ ] `en/azcomic` → standalone HttpSource
|
|
- [ ] `en/azuki`
|
|
- [x] `en/baektoons`
|
|
- [x] `en/bakkin` ⚠️ see notes
|
|
- [ ] `en/bakkinselfhosted`
|
|
- [ ] `en/batcave`
|
|
- [ ] `en/battleinfivesecondsaftermeeting`
|
|
- [x] `en/beehentai` → base: MadTheme
|
|
- [ ] `en/bookwalker` ⚠️ see notes
|
|
- [x] `en/boratscans` → base: Madara
|
|
- [x] `en/boxmanhwa` → base: MadTheme
|
|
- [ ] `en/broccolisoup` → standalone HttpSource
|
|
- [x] `en/bunmanga` → base: Madara
|
|
- [ ] `en/buttsmithy` → standalone HttpSource
|
|
- [ ] `en/clonemanga` → standalone HttpSource
|
|
- [ ] `en/clowncorps` → standalone HttpSource
|
|
- [ ] `en/cmanhua`
|
|
- [x] `en/cocomic`
|
|
- [ ] `en/coffeemanga` → base: Madara (complex)
|
|
- [ ] `en/collectedcurios`
|
|
- [ ] `en/comicasura`
|
|
- [ ] `en/comiccx`
|
|
- [ ] `en/comichubfree`
|
|
- [ ] `en/comickfan`
|
|
- [ ] `en/comickiba`
|
|
- [ ] `en/comicland`
|
|
- [x] `en/comicsland`
|
|
- [ ] `en/comix`
|
|
- [x] `en/crowscans`
|
|
- [x] `en/cucumbermanga` → base: Madara
|
|
- [ ] `en/culturedworks`
|
|
- [ ] `en/cutiecomics`
|
|
- [x] `en/dankefurslesen`
|
|
- [ ] `en/darklegacycomics`
|
|
- [ ] `en/darkscans`
|
|
- [ ] `en/darkscience`
|
|
- [ ] `en/darthsdroids`
|
|
- [ ] `en/deathtollscans`
|
|
- [x] `en/decadencescans` → base: Madara
|
|
- [ ] `en/dflowscans`
|
|
- [ ] `en/digitalcomicmuseum`
|
|
- [x] `en/divascans`
|
|
- [ ] `en/doujinio`
|
|
- [ ] `en/doujins`
|
|
- [ ] `en/dragontea`
|
|
- [ ] `en/drakescans`
|
|
- [ ] `en/dynasty`
|
|
- [ ] `en/eggporncomics`
|
|
- [ ] `en/egscomics`
|
|
- [ ] `en/eighteenporncomic`
|
|
- [x] `en/eightmuses`
|
|
- [ ] `en/elanschool`
|
|
- [x] `en/elftoon`
|
|
- [x] `en/epicmanga`
|
|
- [x] `en/erisscans`
|
|
- [x] `en/ero18x`
|
|
- [ ] `en/erofus`
|
|
- [ ] `en/erosscans`
|
|
- [ ] `en/evascans`
|
|
- [x] `en/evilflowers`
|
|
- [ ] `en/existentialcomics`
|
|
- [ ] `en/explosm`
|
|
- [ ] `en/ezmanga`
|
|
- [x] `en/fablescans`
|
|
- [ ] `en/fairyscans`
|
|
- [ ] `en/firescans` → base: Madara (complex)
|
|
- [ ] `en/flamecomics`
|
|
- [ ] `en/frierenonline`
|
|
- [x] `en/gakamangas`
|
|
- [x] `en/galaxydegenscans`
|
|
- [ ] `en/galaxymanga`
|
|
- [ ] `en/gedecomix`
|
|
- [ ] `en/gingertoon`
|
|
- [ ] `en/goda`
|
|
- [ ] `en/gourmetscans`
|
|
- [ ] `en/greedscans`
|
|
- [ ] `en/grimscans`
|
|
- [ ] `en/grrlpower`
|
|
- [ ] `en/gunnerkriggcourt`
|
|
- [x] `en/guya` ⚠️ see notes
|
|
- [ ] `en/gwtb`
|
|
- [x] `en/hachirumi`
|
|
- [ ] `en/hadesscans`
|
|
- [ ] `en/harimanga`
|
|
- [ ] `en/hentai3zcc`
|
|
- [x] `en/hentai4free`
|
|
- [ ] `en/hentaidex`
|
|
- [ ] `en/hentaihere`
|
|
- [ ] `en/hentaikun`
|
|
- [ ] `en/hentainexus`
|
|
- [ ] `en/hentairead` → base: Madara (complex)
|
|
- [ ] `en/hentaireadio`
|
|
- [ ] `en/hentaisco`
|
|
- [ ] `en/hentaixcomic`
|
|
- [ ] `en/hentaixdickgirl`
|
|
- [ ] `en/hentaixyuri`
|
|
- [ ] `en/hentara`
|
|
- [ ] `en/heytoon`
|
|
- [x] `en/hijalascans`
|
|
- [ ] `en/hiperdex`
|
|
- [ ] `en/hiveworks`
|
|
- [ ] `en/hm2d`
|
|
- [ ] `en/honkaiimpact`
|
|
- [ ] `en/hotcomics`
|
|
- [ ] `en/hyakuro`
|
|
- [ ] `en/infernalvoidscans`
|
|
- [ ] `en/infinityscans`
|
|
- [ ] `en/irovedout`
|
|
- [ ] `en/isekaiscantop`
|
|
- [x] `en/jinmangas`
|
|
- [ ] `en/jnovel`
|
|
- [ ] `en/kagane`
|
|
- [x] `en/kaizenscan`
|
|
- [x] `en/kaliscancom` → base: MadTheme
|
|
- [x] `en/kaliscanio` → base: MadTheme
|
|
- [x] `en/kaliscanme`
|
|
- [ ] `en/kappabeast`
|
|
- [ ] `en/kaynscans`
|
|
- [ ] `en/keenspot`
|
|
- [ ] `en/kenscans`
|
|
- [x] `en/kewnscans`
|
|
- [ ] `en/killsixbilliondemons`
|
|
- [ ] `en/kingcomix`
|
|
- [x] `en/kingofshojo`
|
|
- [x] `en/kissmangain`
|
|
- [ ] `en/kmanga`
|
|
- [ ] `en/kodansha`
|
|
- [ ] `en/ksgroupscans`
|
|
- [x] `en/kunmanga`
|
|
- [ ] `en/kuramanga`
|
|
- [ ] `en/lagoonscans`
|
|
- [ ] `en/leslievictims`
|
|
- [x] `en/lhtranslation`
|
|
- [ ] `en/likemanga`
|
|
- [ ] `en/likemangain`
|
|
- [ ] `en/lilymanga`
|
|
- [x] `en/linkmanga` → base: Madara
|
|
- [ ] `en/loadingartist`
|
|
- [ ] `en/luascans`
|
|
- [ ] `en/luminaretranslations`
|
|
- [x] `en/lunatoons`
|
|
- [ ] `en/madaradex`
|
|
- [ ] `en/madarascans`
|
|
- [ ] `en/madokami`
|
|
- [ ] `en/magusmanga`
|
|
- [ ] `en/mahouirexnohentaikarte`
|
|
- [ ] `en/manga18club`
|
|
- [ ] `en/manga18free`
|
|
- [ ] `en/manga18fx`
|
|
- [x] `en/manga18x`
|
|
- [ ] `en/mangabat`
|
|
- [ ] `en/mangablaze`
|
|
- [ ] `en/mangabolt`
|
|
- [ ] `en/mangabtt`
|
|
- [x] `en/mangabuddy` → base: MadTheme
|
|
- [x] `en/mangabuddyme` → base: MadTheme
|
|
- [ ] `en/mangaclash`
|
|
- [ ] `en/mangacloud`
|
|
- [x] `en/mangacute` → base: MadTheme
|
|
- [ ] `en/mangadass`
|
|
- [ ] `en/mangade`
|
|
- [ ] `en/mangademon`
|
|
- [ ] `en/mangadia`
|
|
- [ ] `en/mangadistrict`
|
|
- [ ] `en/mangadotnet`
|
|
- [ ] `en/mangadrama`
|
|
- [x] `en/mangafab` → base: MadTheme
|
|
- [x] `en/mangaforest` → base: MadTheme
|
|
- [ ] `en/mangaforfreecom`
|
|
- [ ] `en/mangafox`
|
|
- [x] `en/mangafoxfun` → base: MangaHub
|
|
- [ ] `en/mangafreak`
|
|
- [ ] `en/mangafree`
|
|
- [x] `en/mangagg`
|
|
- [ ] `en/mangago`
|
|
- [ ] `en/mangagofun`
|
|
- [x] `en/mangahe`
|
|
- [ ] `en/mangahen`
|
|
- [x] `en/mangahentai`
|
|
- [ ] `en/mangahere`
|
|
- [x] `en/mangahereonl` → base: MangaHub
|
|
- [x] `en/mangahubio` → base: MangaHub
|
|
- [ ] `en/mangaka`
|
|
- [ ] `en/mangakakalot`
|
|
- [x] `en/mangakakalotfun` → base: MangaHub
|
|
- [ ] `en/mangakatana`
|
|
- [ ] `en/mangakiss`
|
|
- [ ] `en/mangamaniacs`
|
|
- [ ] `en/mangamo`
|
|
- [ ] `en/mangamob`
|
|
- [x] `en/mangamonk` → base: MadTheme
|
|
- [x] `en/manganel` → base: MangaHub
|
|
- [ ] `en/manganelo`
|
|
- [ ] `en/manganow`
|
|
- [x] `en/mangaonlinefun` → base: MangaHub
|
|
- [ ] `en/mangaowlio`
|
|
- [x] `en/mangapandaonl` → base: MangaHub
|
|
- [ ] `en/mangapill`
|
|
- [ ] `en/mangapuma`
|
|
- [ ] `en/mangarawclub`
|
|
- [ ] `en/mangaread`
|
|
- [ ] `en/mangareadercc`
|
|
- [x] `en/mangareadersite` → base: MangaHub
|
|
- [ ] `en/mangareadorg`
|
|
- [x] `en/mangasaga` → base: MadTheme
|
|
- [ ] `en/mangasect`
|
|
- [x] `en/mangaspin` → base: MadTheme
|
|
- [ ] `en/mangasushi`
|
|
- [ ] `en/mangatellers`
|
|
- [x] `en/mangatoday` → base: MangaHub
|
|
- [ ] `en/mangatown`
|
|
- [ ] `en/mangatrend`
|
|
- [ ] `en/mangatx`
|
|
- [x] `en/mangaxyz` → base: MadTheme
|
|
- [ ] `en/manhuafast`
|
|
- [ ] `en/manhuafastnet`
|
|
- [ ] `en/manhuahot`
|
|
- [ ] `en/manhuanext`
|
|
- [x] `en/manhuanow` → base: MadTheme
|
|
- [ ] `en/manhuaplus`
|
|
- [ ] `en/manhuaplusorg`
|
|
- [ ] `en/manhuarush`
|
|
- [ ] `en/manhuascanus`
|
|
- [x] `en/manhuasite` → base: MadTheme
|
|
- [ ] `en/manhuatop`
|
|
- [ ] `en/manhuaus`
|
|
- [ ] `en/manhuazonghe`
|
|
- [ ] `en/manhwa18`
|
|
- [ ] `en/manhwa18org`
|
|
- [ ] `en/manhwa68`
|
|
- [ ] `en/manhwabuddy`
|
|
- [ ] `en/manhwaclan`
|
|
- [ ] `en/manhwacomics`
|
|
- [ ] `en/manhwaden`
|
|
- [ ] `en/manhwaget`
|
|
- [ ] `en/manhwahub`
|
|
- [ ] `en/manhwajoy`
|
|
- [ ] `en/manhwalike`
|
|
- [x] `en/manhwalover`
|
|
- [ ] `en/manhwamanhua`
|
|
- [ ] `en/manhwaread`
|
|
- [x] `en/manhwareads`
|
|
- [ ] `en/manhwatoon`
|
|
- [x] `en/manhwatop`
|
|
- [x] `en/manhwax`
|
|
- [ ] `en/manhwaxxl`
|
|
- [ ] `en/manhwaz`
|
|
- [ ] `en/manhwazone`
|
|
- [ ] `en/manta`
|
|
- [ ] `en/megatokyo`
|
|
- [ ] `en/mehgazone`
|
|
- [ ] `en/meitoon`
|
|
- [x] `en/mgjinx`
|
|
- [ ] `en/mgreadio`
|
|
- [ ] `en/milftoon`
|
|
- [x] `en/mistscans`
|
|
- [ ] `en/mlbblore`
|
|
- [ ] `en/monochromecustom`
|
|
- [x] `en/monochromescans`
|
|
- [ ] `en/multporn`
|
|
- [ ] `en/murimscan`
|
|
- [ ] `en/myhentaicomics`
|
|
- [ ] `en/myhentaigallery`
|
|
- [x] `en/necroscans`
|
|
- [ ] `en/newmanhwa`
|
|
- [ ] `en/nexcomic`
|
|
- [ ] `en/nikatoons`
|
|
- [ ] `en/nineanime`
|
|
- [ ] `en/ninehentai`
|
|
- [ ] `en/ninekon`
|
|
- [ ] `en/novel24h`
|
|
- [ ] `en/novelcrow`
|
|
- [ ] `en/noxenscans`
|
|
- [ ] `en/nuxscans`
|
|
- [x] `en/nyanukafe`
|
|
- [ ] `en/nyrascans`
|
|
- [x] `en/nyxscans`
|
|
- [x] `en/octopusmanga`
|
|
- [ ] `en/oglaf`
|
|
- [ ] `en/ohjoysextoy`
|
|
- [ ] `en/omegascans`
|
|
- [x] `en/onemangaco` → base: MangaHub
|
|
- [x] `en/onemangainfo` → base: MangaHub
|
|
- [ ] `en/onepunchmanonline`
|
|
- [ ] `en/onlythebesthentai`
|
|
- [ ] `en/oots`
|
|
- [ ] `en/oppaistream`
|
|
- [ ] `en/orchisasia`
|
|
- [x] `en/orionscans`
|
|
- [ ] `en/paradisescans`
|
|
- [ ] `en/paragonscans`
|
|
- [ ] `en/paritehaber`
|
|
- [ ] `en/patchfriday`
|
|
- [x] `en/pawmanga`
|
|
- [ ] `en/petrotechsociety`
|
|
- [ ] `en/philiascans`
|
|
- [ ] `en/plutoscans`
|
|
- [ ] `en/pmscans`
|
|
- [ ] `en/porncomix`
|
|
- [ ] `en/qiscans`
|
|
- [ ] `en/questionablecontent`
|
|
- [ ] `en/ragescans`
|
|
- [ ] `en/randowiz`
|
|
- [x] `en/ravenscans`
|
|
- [ ] `en/razure`
|
|
- [ ] `en/rdscans`
|
|
- [ ] `en/readallcomicscom`
|
|
- [x] `en/readattackontitanshingekinokyojinmanga`
|
|
- [x] `en/readberserkmanga`
|
|
- [x] `en/readblackclovermangaonline`
|
|
- [x] `en/readbokunoheroacademiamyheroacademiamanga`
|
|
- [x] `en/readchainsawmanmangaonline`
|
|
- [ ] `en/readcomiconline`
|
|
- [ ] `en/readcomicsonline`
|
|
- [x] `en/readfairytailedenszeromangaonline`
|
|
- [x] `en/readhaikyuumangaonline`
|
|
- [x] `en/readjujutsukaisenmangaonline`
|
|
- [x] `en/readkingdommangaonline`
|
|
- [x] `en/readnanatsunotaizai7deadlysinsmangaonline`
|
|
- [x] `en/readnarutoborutosamurai8mangaonline`
|
|
- [x] `en/readonepiecemangaonline`
|
|
- [x] `en/readonepunchmanmangaonlinetwo`
|
|
- [x] `en/readsololevelingmangamanhwaonline`
|
|
- [x] `en/readtokyoghoulretokyoghoulmangaonline`
|
|
- [ ] `en/readvagabondmanga`
|
|
- [ ] `en/reallifecomics`
|
|
- [ ] `en/reimanga`
|
|
- [x] `en/renascans`
|
|
- [ ] `en/resetscans`
|
|
- [ ] `en/restscans`
|
|
- [ ] `en/retsu`
|
|
- [ ] `en/revivalscans`
|
|
- [ ] `en/rinkocomics`
|
|
- [x] `en/ritharscans`
|
|
- [ ] `en/rizzcomic`
|
|
- [ ] `en/rizzcomicunoriginal`
|
|
- [ ] `en/rokaricomics`
|
|
- [ ] `en/roliascan`
|
|
- [x] `en/rosesquadscans`
|
|
- [x] `en/ryumanga`
|
|
- [ ] `en/s2manga`
|
|
- [x] `en/sanascans`
|
|
- [ ] `en/saturdaymorningbreakfastcomics`
|
|
- [ ] `en/schlockmercenary`
|
|
- [ ] `en/setsuscans`
|
|
- [ ] `en/shibamanga`
|
|
- [ ] `en/shojoscans`
|
|
- [x] `en/sirenscans`
|
|
- [ ] `en/skymanga`
|
|
- [x] `en/sleepytranslations` → base: Madara
|
|
- [ ] `en/solarandsundry`
|
|
- [ ] `en/spmanhwa`
|
|
- [ ] `en/spyfakku`
|
|
- [ ] `en/stonescape`
|
|
- [ ] `en/sunshinebutterflyscans`
|
|
- [ ] `en/supermega`
|
|
- [ ] `en/suryascans`
|
|
- [ ] `en/swordscomic`
|
|
- [ ] `en/tapastic`
|
|
- [ ] `en/tcbscans`
|
|
- [x] `en/tcbscansunoriginal`
|
|
- [ ] `en/teamshadowi`
|
|
- [ ] `en/templescan`
|
|
- [ ] `en/theblank`
|
|
- [ ] `en/theduckwebcomics`
|
|
- [ ] `en/thepropertyofhate`
|
|
- [ ] `en/timelesstoons`
|
|
- [ ] `en/todaymanga`
|
|
- [ ] `en/toon18`
|
|
- [ ] `en/toongod`
|
|
- [ ] `en/toonily`
|
|
- [x] `en/toonilyme`
|
|
- [x] `en/toonitube`
|
|
- [ ] `en/toonizy`
|
|
- [x] `en/topmanhua`
|
|
- [ ] `en/topmanhuafan`
|
|
- [ ] `en/topmanhuanet`
|
|
- [ ] `en/tritiniascans`
|
|
- [ ] `en/utoon`
|
|
- [ ] `en/valirscans`
|
|
- [x] `en/vanillascans`
|
|
- [ ] `en/vgperson`
|
|
- [ ] `en/vizshonenjump`
|
|
- [ ] `en/voyceme`
|
|
- [ ] `en/vyvymanga`
|
|
- [ ] `en/vyvymangaorg`
|
|
- [ ] `en/warforrayuba`
|
|
- [ ] `en/wearehunger`
|
|
- [ ] `en/webcomics`
|
|
- [ ] `en/webdexscans`
|
|
- [ ] `en/webnovel`
|
|
- [ ] `en/webtoonscan`
|
|
- [ ] `en/webtoonxyz`
|
|
- [ ] `en/weebcentral`
|
|
- [ ] `en/whalemanga`
|
|
- [ ] `en/witchscans`
|
|
- [ ] `en/woopread`
|
|
- [ ] `en/writerscans`
|
|
- [ ] `en/wuxiaworld`
|
|
- [ ] `en/xlecx`
|
|
- [ ] `en/xomanga`
|
|
- [ ] `en/xoxocomics`
|
|
- [ ] `en/yakshacomics`
|
|
- [ ] `en/yaoihot`
|
|
- [x] `en/yaoihub`
|
|
- [ ] `en/yaoiscan`
|
|
- [ ] `en/yaoitoon`
|
|
- [ ] `en/yorai`
|
|
- [ ] `en/zazamanga`
|
|
- [ ] `en/zinchanmanga`
|
|
- [ ] `en/zinchanmangacom`
|
|
- [ ] `en/zinmanga`
|
|
- [ ] `en/zinmanganet`
|
|
|
|
---
|
|
|
|
## Notes — Complex Sources
|
|
|
|
### HTTP Client Selection
|
|
All sources use the same unified client — `httpclient.DefaultClient()`. You no longer need
|
|
to decide between direct HTTP and FlareSolverr; the client handles both automatically.
|
|
|
|
In Kotlin, `override val client = network.cloudflareClient` signals the source needs
|
|
Cloudflare bypass — but in Go, the adaptive `Do()` method provides it transparently.
|
|
|
|
### `all/mangadex` ⚠️
|
|
- Rate limit: 5 req/s (`golang.org/x/time/rate`)
|
|
- `GetPopularManga` — `GET /manga?order[rating]=desc&includes[]=cover_art&limit=20&offset={(n-1)*20}`
|
|
- `GetLatestUpdates` — `GET /manga?order[updatedAt]=desc&includes[]=cover_art`
|
|
- `GetSearchManga` — tag filters (`includedTags[]`, `excludedTags[]`), demographic, content rating, status, sort
|
|
- `GetMangaDetails` — `GET /manga/{id}?includes[]=cover_art&includes[]=author&includes[]=artist`; cover URL from relationships
|
|
- `GetChapterList` — `GET /manga/{id}/feed?translatedLanguage[]=en&limit=500`; paginate until all chapters fetched
|
|
- `GetPageList` — `GET /at-home/server/{chapterId}` → `{baseUrl}/{quality}/{hash}/{filename}`; quality = `data` or `data-saver`
|
|
- `GetFilterList` — `GET /manga/tag`; cache tag list
|
|
- Language configurable via `MANGADEX_LANG` env var (default `en`)
|
|
|
|
### `all/nhentaicom` ⚠️
|
|
- FlareSolverr required
|
|
- `GetPageList` — extract JSON blob from `document.getElementById(...)` in `<script>` tag; reconstruct URLs as `{imageServer}/galleries/{mediaId}/{pageNum}.{ext}`
|
|
|
|
### `all/kemono` ⚠️
|
|
- FlareSolverr required
|
|
- Service + creator = manga; posts = chapters; attachment files = pages
|
|
- File URLs: prefix relative paths with `{base}/data`
|
|
- `GetChapterList` — paginate in 50-post increments via `?o={offset}`
|
|
|
|
### `all/komga` ⚠️
|
|
- Self-hosted; `KOMGA_BASE_URL`, `KOMGA_USERNAME`, `KOMGA_PASSWORD` env vars; Basic Auth
|
|
- Series = manga; Books = chapters; pages via `GET /api/v1/books/{bookId}/pages/{n}/thumbnail`
|
|
|
|
### `all/e621` ⚠️
|
|
- Optional Basic Auth (`E621_USERNAME`, `E621_API_KEY`); required for Gold content
|
|
- Pools = manga; posts = pages; `GET /pools.json`, `GET /posts.json?tags=pool:{id}`
|
|
|
|
### `all/danbooru` ⚠️
|
|
- Optional Basic Auth; free accounts limited to 2 tags per search
|
|
- Pools = manga; pool post_ids = pages; image URL from `file_url` field
|
|
|
|
### `all/pixiv` ⚠️
|
|
- Auth: `PIXIV_PHPSESSID` env var → Cookie header
|
|
- `GetPageList` — `GET /ajax/illust/{illustId}/pages`; extract `urls.original`
|
|
- **Must set `Referer: https://www.pixiv.net/`** on all image requests or get 403
|
|
|
|
### `all/luscious` ⚠️
|
|
- GraphQL POST to `https://members.luscious.net/graphql/playground`
|
|
- Albums = manga; `SearchAlbumPictures` paginated for page list
|
|
|
|
### `all/mangaplus` ⚠️
|
|
- JSON endpoint: `GET https://jumpg-webapi.tokyo-cdn.com/api/title_detail?title_id={id}`
|
|
- `GetPageList` — XOR page URL decryption using per-page `encryptionKey` (hex string); implement in `GetImageURL`
|
|
- Required header: `Origin: https://mangaplus.shueisha.co.jp`
|
|
|
|
### `all/stashapp` ⚠️
|
|
- Self-hosted; `STASHAPP_BASE_URL`, `STASHAPP_API_KEY` env vars
|
|
- GraphQL POST to `{base}/graphql`; `ApiKey` header
|
|
|
|
### `all/globalcomix` ⚠️
|
|
- Signed CDN URLs with short expiry — **do NOT cache `image_url` in DB**; always fetch fresh page URLs
|
|
|
|
### `all/webtoons` ⚠️
|
|
- HMAC-SHA256 `Sec-Webtoon-Client-Data` header: `HMAC-SHA256(fixedKey, "{url}_{timestamp}")` → base64; fixed key in extension source
|
|
- Web API: `https://global.apis.naver.com/webtoonSvc/v1/...`
|
|
|
|
### `en/allanime` ⚠️
|
|
- GraphQL POST to `https://api.allanime.day/api`
|
|
- Episode-based chapters; `episodeString` used as chapter number
|
|
- Multiple CDN quality tiers in page response; pick highest
|
|
|
|
### `en/asurascans` ⚠️
|
|
- FlareSolverr required
|
|
- Page selector varies by layout version; check extension source for current selector
|
|
|
|
### `en/bakkin` ⚠️
|
|
- No list/search; all manga from a single JSON URL; enumerate from object keys
|
|
- `GetSearchManga` does client-side title filtering only
|
|
|
|
### `en/bookwalker` ⚠️
|
|
- DRM-protected; metadata only
|
|
- `GetPageList` returns empty slice; pages not accessible without purchase
|
|
|
|
### `en/guya` ⚠️
|
|
- `GET {base}/api/get_all_series/` returns all manga at once; no pagination
|
|
- Scanlation group filter applied client-side in response parsing
|