achmad
d432a3948f
- Replace old net/http + FS diagram with httpcloak hybrid approach - Document TLS fingerprint problem and why httpcloak fixes it - Add FLARESOLVERR_SESSION env var docs - Add goquery :has() + attribute selector limitation workaround - Update design decisions to reflect current architecture
807 lines
25 KiB
Markdown
Executable File
807 lines
25 KiB
Markdown
Executable File
# Phase 4 — Standalone Sources
|
|
|
|
Complete port checklist. Check a box when the source passes a basic smoke test
|
|
(popular/latest list returns ≥1 result, or detail+pages resolve for a known URL).
|
|
|
|
**After implementing each source:** mark its checkbox `[x]` in this doc and add it to the import list in `cmd/server/main.go`.
|
|
|
|
## Important: Source Types
|
|
|
|
### Base Sources (`lib-multisrc`)
|
|
Shared implementations in:
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/lib-multisrc/`
|
|
- `/Users/achmad/Documents/Belajar/Web/goyomi/sources/base/`
|
|
|
|
These are the core implementations that standalone sources may use.
|
|
|
|
### Standalone Sources (`all/` and `en/`)
|
|
Individual extensions in:
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/all/`
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/en/`
|
|
|
|
These may or may not use a base source from lib-multisrc — and even when they do, they often add their own
|
|
implementation on top (overriding methods, adding extra parsing, custom filters, etc.).
|
|
When porting, check if the Kotlin source extends a base class (e.g., `MangaReader`, `Madara`, `ComicGeek`) —
|
|
if so, use the corresponding Go base implementation as the foundation, then layer the source-specific overrides.
|
|
Otherwise, implement fully standalone.
|
|
|
|
Reference:
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/all/`
|
|
- `/Users/achmad/Documents/Belajar/Android/extensions-source/src/en/`
|
|
|
|
## HTTP Client Architecture
|
|
|
|
The HTTP client uses a **hybrid** approach: [httpcloak](https://github.com/sardanioss/httpcloak)
|
|
for Chrome TLS fingerprint emulation, with FlareSolverr as a fallback for challenge solving.
|
|
|
|
### TLS Fingerprint Problem
|
|
|
|
Go's `net/http` has a different TLS fingerprint (JA3/JA4 ciphers, HTTP/2 settings) than Chrome.
|
|
When Go sends a `cf_clearance` cookie obtained from FlareSolverr's Chrome, Cloudflare **rejects**
|
|
it because the TLS fingerprint doesn't match Chrome's. Every direct request gets re-challenged.
|
|
|
|
The fix: use **httpcloak** as the transport, which mimics Chrome's TLS fingerprint perfectly.
|
|
Now when FlareSolverr solves a challenge and returns `cf_clearance`, the cookie is fed into
|
|
httpcloak's session. Subsequent requests via httpcloak (with matching TLS fingerprint + cookie)
|
|
pass Cloudflare without re-challenge.
|
|
|
|
### Architecture
|
|
|
|
```
|
|
httpclient.DefaultClient()
|
|
├── httpcloak.Session (Chrome TLS fingerprint + cookie jar)
|
|
│ ├── 200 → return response (fast path, no re-challenge)
|
|
│ └── 403/503 → FlareSolverr fallback
|
|
└── FlareSolverr raw mode (auto-detected from FLARESOLVERR_URL env var)
|
|
└── Cookies fed back into httpcloak session for next request
|
|
```
|
|
|
|
### Flows
|
|
|
|
- **Non-Cloudflare site**: httpcloak → 200. Fast, ~0.5s.
|
|
- **Cloudflare site, first request**: httpcloak → 403 → FS solves challenge (~12-60s) →
|
|
`cf_clearance` cookie stored in httpcloak session.
|
|
- **Cloudflare site, subsequent requests**: httpcloak (with Chrome TLS + clearance cookie) →
|
|
200. Fast, ~1-2s.
|
|
|
|
### FlareSolverr Integration
|
|
|
|
FlareSolverr is auto-configured from `FLARESOLVERR_URL` env var (e.g.,
|
|
`http://localhost:8191`). If unset, the client works in direct-only mode.
|
|
|
|
| Env Var | Default | Description |
|
|
|---|---|---|
|
|
| `FLARESOLVERR_URL` | — | FlareSolverr endpoint |
|
|
| `FLARESOLVERR_SESSION` | `goyomi` | FS browser session ID; reuses one Chrome instance. Set empty for per-request sessions (more parallel, more memory) |
|
|
| `FLARESOLVERR_LOG_LEVEL` | `info` | FS log verbosity |
|
|
|
|
The `flare` package (`internal/httpclient/flare/`) is a backward-compatible alias for
|
|
`httpclient.Client`. Sources that already import `flare` continue to compile.
|
|
|
|
### Kotlin vs Go Mapping
|
|
|
|
| Kotlin (`cloudflareClient`) | Go (`httpclient.Client`) |
|
|
|---|---|
|
|
| Android WebView (Chrome) → solves 503 → retries with cookies | httpcloak (Chrome TLS fingerprint) → on 403 → FS fallback → cookies fed to httpcloak |
|
|
| WebView + OkHttp share Android's TLS stack | httpcloak mimics Chrome's JA3 fingerprint |
|
|
| Challenge solved once, cookies reused | Challenge solved once, cookies reused via httpcloak session |
|
|
| Returns raw server response | Returns FS raw body or httpcloak body |
|
|
| Shared across all extensions | `DefaultClient()` singleton shared across all sources |
|
|
|
|
### Design Decisions
|
|
|
|
1. **Why httpcloak?** — Go's net/http TLS fingerprint doesn't match Chrome's, so
|
|
Cloudflare clearance cookies from FS are rejected on direct requests. httpcloak
|
|
emulates Chrome's JA3/JA4 fingerprint, making subsequent requests pass Cloudflare.
|
|
2. **Why not go-cfscraper?** — The pure-Go JS challenge solver (goja) and external
|
|
runtimes (Node.js) both lack browser DOM APIs (`location`, `document`, `window`)
|
|
that Cloudflare challenge scripts require. The DOM shims are fragile and break
|
|
when Cloudflare updates. FlareSolverr with real Chrome is the only reliable solver.
|
|
3. **Why FlareSolverr at all?** — The first request to a Cloudflare site always gets
|
|
challenged (no `cf_clearance` cookie yet). FlareSolverr's real Chrome solves it.
|
|
4. **Why a singleton?** — Shared httpcloak session = cookies from FS (solved challenges)
|
|
benefit all sources. Shared rate limiter prevents hammering the same host.
|
|
5. **Why strip FS wrapper?** — FS routes requests through Chrome, which wraps JSON
|
|
responses in `<html>…<pre>…</pre>…</html>`. Without stripping, JSON parsers fail.
|
|
6. **Why FS session matters** — With `FLARESOLVERR_SESSION`, FS reuses one Chrome
|
|
instance. After the first challenge, `cf_clearance` is cached. Subsequent requests
|
|
to the same domain are near-instant. Without a session, each request spawns a
|
|
fresh Chrome, solving the challenge from scratch every time (~13s each).
|
|
|
|
### Known goquery Limitation — `:has()` + Attribute Selectors
|
|
|
|
goquery (via the cascadia CSS engine) does **not** support `:has()` combined with
|
|
attribute selectors like `a[href*=/video/]`. This works in Jsoup (Kotlin) but silently
|
|
returns 0 matches in Go.
|
|
|
|
**Wrong** (CSS-only, works in Kotlin, returns 0 in Go):
|
|
```go
|
|
doc.Find("figure:not(:has(a[href*=/video/]))")
|
|
```
|
|
|
|
**Correct** (programmatic filtering):
|
|
```go
|
|
doc.Find("figure").Each(func(_ int, el *goquery.Selection) {
|
|
if hasAttr(el, "a", "href", "/video/") { return }
|
|
// process entry
|
|
})
|
|
|
|
func hasAttr(el *goquery.Selection, tag, attr, substr string) bool {
|
|
found := false
|
|
el.Find(tag).EachWithBreak(func(_ int, a *goquery.Selection) bool {
|
|
if v, ok := a.Attr(attr); ok && strings.Contains(v, substr) {
|
|
found = true; return false
|
|
}
|
|
return true
|
|
})
|
|
return found
|
|
}
|
|
```
|
|
|
|
Always check Kotlin references for `:has()` selectors and convert them to
|
|
programmatic filtering when porting.
|
|
|
|
Detailed implementation notes for complex sources are in the **Notes** section at the bottom.
|
|
|
|
---
|
|
|
|
## `sources/all/` — 125 sources
|
|
|
|
- [x] `all/ahottie`
|
|
- [x] `all/akuma`
|
|
- [x] `all/allporncomicsco`
|
|
- [x] `all/asmhentai`
|
|
- [x] `all/baobua`
|
|
- [x] `all/beauty3600000`
|
|
- [x] `all/buondua`
|
|
- [x] `all/comicfury`
|
|
- [x] `all/comicgrowl`
|
|
- [x] `all/comicklive`
|
|
- [ ] `all/comicskingdom` ⚠️ no Kotlin reference found — skip
|
|
- [x] `all/comicsvalley`
|
|
- [x] `all/comikey`
|
|
- [x] `all/commitstrip`
|
|
- [x] `all/coomer`
|
|
- [ ] `all/coronaex`
|
|
- [ ] `all/cosplaytele`
|
|
- [ ] `all/cubari`
|
|
- [ ] `all/danbooru` ⚠️ see notes
|
|
- [ ] `all/deviantart`
|
|
- [ ] `all/dragonballmultiverse`
|
|
- [ ] `all/e621` ⚠️ see notes
|
|
- [x] `all/elitebabes`
|
|
- [ ] `all/everiaclub`
|
|
- [ ] `all/everiaclubcom`
|
|
- [x] `all/femjoyhunter`
|
|
- [ ] `all/foamgirl`
|
|
- [ ] `all/foolslidecustomizable`
|
|
- [ ] `all/fourkhd`
|
|
- [x] `all/ftvhunter`
|
|
- [ ] `all/fuwayomi` ⚠️ skip — server-side Fuwayomi integration, not a scraper
|
|
- [ ] `all/globalcomix` ⚠️ see notes
|
|
- [ ] `all/grabberzone`
|
|
- [ ] `all/hdoujin`
|
|
- [ ] `all/hennojin`
|
|
- [ ] `all/hentai3`
|
|
- [ ] `all/hentaicosplay`
|
|
- [ ] `all/hentaienvy`
|
|
- [ ] `all/hentaiera`
|
|
- [ ] `all/hentaifox`
|
|
- [x] `all/hentaihand`
|
|
- [ ] `all/hentairox`
|
|
- [ ] `all/hentaizap`
|
|
- [x] `all/hniscantrad`
|
|
- [ ] `all/holonometria`
|
|
- [ ] `all/honeytoon`
|
|
- [ ] `all/imhentai`
|
|
- [ ] `all/izneo`
|
|
- [ ] `all/jjcos`
|
|
- [x] `all/joymiihub`
|
|
- [ ] `all/junmeitu`
|
|
- [x] `all/kemono` ⚠️ see notes
|
|
- [ ] `all/kiutaku`
|
|
- [ ] `all/kodokustudio`
|
|
- [ ] `all/koharu`
|
|
- [ ] `all/komga` ⚠️ see notes
|
|
- [ ] `all/lanraragi`
|
|
- [ ] `all/leagueoflegends`
|
|
- [ ] `all/lunaranime`
|
|
- [ ] `all/luscious` ⚠️ see notes
|
|
- [ ] `all/magicaltranslators`
|
|
- [ ] `all/manga18me`
|
|
- [ ] `all/mangaball`
|
|
- [x] `all/mangacrazy`
|
|
- [ ] `all/mangadex` ⚠️ see notes
|
|
- [ ] `all/mangadraft`
|
|
- [ ] `all/mangafire`
|
|
- [ ] `all/mangaforfree`
|
|
- [ ] `all/mangaplus` ⚠️ see notes
|
|
- [ ] `all/mangapluscreators`
|
|
- [x] `all/mangataro`
|
|
- [ ] `all/mangatoon`
|
|
- [ ] `all/mangaup`
|
|
- [ ] `all/mango`
|
|
- [ ] `all/manhuarm`
|
|
- [ ] `all/manhwa18cc`
|
|
- [ ] `all/manhwa18net`
|
|
- [ ] `all/manhwa18uncensored`
|
|
- [ ] `all/manhwaclubnet`
|
|
- [ ] `all/manhwadashraw`
|
|
- [ ] `all/mayotune`
|
|
- [x] `all/metarthunter`
|
|
- [ ] `all/miauscan`
|
|
- [ ] `all/misskon`
|
|
- [ ] `all/mitaku`
|
|
- [ ] `all/myreadingmanga`
|
|
- [ ] `all/namicomi`
|
|
- [ ] `all/nhentaicom` ⚠️ see notes
|
|
- [ ] `all/nhentaixxx`
|
|
- [ ] `all/niadd`
|
|
- [ ] `all/ninemanga`
|
|
- [ ] `all/novelcool`
|
|
- [ ] `all/ososedki`
|
|
- [ ] `all/pandachaika`
|
|
- [ ] `all/peppercarrot`
|
|
- [ ] `all/photos18`
|
|
- [ ] `all/pixiv` ⚠️ see notes
|
|
- [x] `all/playmatehunter`
|
|
- [ ] `all/pornpics`
|
|
- [ ] `all/projectsuki`
|
|
- [ ] `all/qtoon`
|
|
- [ ] `all/rokuhentai`
|
|
- [ ] `all/sakuramanhwa`
|
|
- [ ] `all/sandraandwoo`
|
|
- [ ] `all/seraphicdeviltry`
|
|
- [ ] `all/simplycosplay`
|
|
- [ ] `all/simplyhentai`
|
|
- [ ] `all/stashapp` ⚠️ see notes
|
|
- [ ] `all/taddyink`
|
|
- [ ] `all/tappytoon`
|
|
- [ ] `all/thelibraryofohara`
|
|
- [ ] `all/thunderscans`
|
|
- [ ] `all/toomics`
|
|
- [ ] `all/twicomi`
|
|
- [ ] `all/uncensoredmanhwa`
|
|
- [ ] `all/vinnieVeritas`
|
|
- [ ] `all/webtoons` ⚠️ see notes
|
|
- [x] `all/xarthunter`
|
|
- [ ] `all/xasiatalbums`
|
|
- [ ] `all/xgmn`
|
|
- [ ] `all/xinmeitulu`
|
|
- [ ] `all/xiutaku`
|
|
- [ ] `all/xkcd`
|
|
- [ ] `all/yabai`
|
|
- [ ] `all/yaoimangaonline`
|
|
- [ ] `all/yellownote`
|
|
- [ ] `all/yskcomics`
|
|
|
|
---
|
|
|
|
## `sources/en/` — 430 sources
|
|
|
|
- [ ] `en/akaicomic`
|
|
- [ ] `en/alandal`
|
|
- [ ] `en/allanime` ⚠️ see notes
|
|
- [ ] `en/allporncomic`
|
|
- [ ] `en/allporncomicio`
|
|
- [ ] `en/anisascans`
|
|
- [ ] `en/apcomics`
|
|
- [x] `en/aquamanga`
|
|
- [x] `en/arcrelight` → base: MangAdventure
|
|
- [x] `en/arenascans`
|
|
- [ ] `en/armageddon`
|
|
- [x] `en/artlapsa`
|
|
- [ ] `en/arvencomics`
|
|
- [ ] `en/arvenscans`
|
|
- [ ] `en/aryascans`
|
|
- [ ] `en/asiatoon`
|
|
- [ ] `en/asmotoon`
|
|
- [x] `en/assortedscans`
|
|
- [ ] `en/asurascans` ⚠️ see notes
|
|
- [ ] `en/athreascans`
|
|
- [ ] `en/atsumaru` → standalone HttpSource
|
|
- [ ] `en/aurora` → standalone HttpSource
|
|
- [ ] `en/azcomic` → standalone HttpSource
|
|
- [ ] `en/azuki`
|
|
- [x] `en/baektoons`
|
|
- [x] `en/bakkin` ⚠️ see notes
|
|
- [ ] `en/bakkinselfhosted`
|
|
- [ ] `en/batcave`
|
|
- [ ] `en/battleinfivesecondsaftermeeting`
|
|
- [x] `en/beehentai` → base: MadTheme
|
|
- [ ] `en/bookwalker` ⚠️ see notes
|
|
- [x] `en/boratscans` → base: Madara
|
|
- [x] `en/boxmanhwa` → base: MadTheme
|
|
- [ ] `en/broccolisoup` → standalone HttpSource
|
|
- [x] `en/bunmanga` → base: Madara
|
|
- [ ] `en/buttsmithy` → standalone HttpSource
|
|
- [ ] `en/clonemanga` → standalone HttpSource
|
|
- [ ] `en/clowncorps` → standalone HttpSource
|
|
- [ ] `en/cmanhua`
|
|
- [x] `en/cocomic`
|
|
- [ ] `en/coffeemanga` → base: Madara (complex)
|
|
- [ ] `en/collectedcurios`
|
|
- [ ] `en/comicasura`
|
|
- [ ] `en/comiccx`
|
|
- [ ] `en/comichubfree`
|
|
- [ ] `en/comickfan`
|
|
- [ ] `en/comickiba`
|
|
- [ ] `en/comicland`
|
|
- [x] `en/comicsland`
|
|
- [ ] `en/comix`
|
|
- [x] `en/crowscans`
|
|
- [x] `en/cucumbermanga` → base: Madara
|
|
- [ ] `en/culturedworks`
|
|
- [ ] `en/cutiecomics`
|
|
- [x] `en/dankefurslesen`
|
|
- [ ] `en/darklegacycomics`
|
|
- [ ] `en/darkscans`
|
|
- [ ] `en/darkscience`
|
|
- [ ] `en/darthsdroids`
|
|
- [ ] `en/deathtollscans`
|
|
- [x] `en/decadencescans` → base: Madara
|
|
- [ ] `en/dflowscans`
|
|
- [ ] `en/digitalcomicmuseum`
|
|
- [x] `en/divascans`
|
|
- [ ] `en/doujinio`
|
|
- [ ] `en/doujins`
|
|
- [ ] `en/dragontea`
|
|
- [ ] `en/drakescans`
|
|
- [ ] `en/dynasty`
|
|
- [ ] `en/eggporncomics`
|
|
- [ ] `en/egscomics`
|
|
- [ ] `en/eighteenporncomic`
|
|
- [x] `en/eightmuses`
|
|
- [ ] `en/elanschool`
|
|
- [x] `en/elftoon`
|
|
- [x] `en/epicmanga`
|
|
- [x] `en/erisscans`
|
|
- [x] `en/ero18x`
|
|
- [ ] `en/erofus`
|
|
- [ ] `en/erosscans`
|
|
- [ ] `en/evascans`
|
|
- [x] `en/evilflowers`
|
|
- [ ] `en/existentialcomics`
|
|
- [ ] `en/explosm`
|
|
- [ ] `en/ezmanga`
|
|
- [x] `en/fablescans`
|
|
- [ ] `en/fairyscans`
|
|
- [ ] `en/firescans` → base: Madara (complex)
|
|
- [ ] `en/flamecomics`
|
|
- [ ] `en/frierenonline`
|
|
- [x] `en/gakamangas`
|
|
- [x] `en/galaxydegenscans`
|
|
- [ ] `en/galaxymanga`
|
|
- [ ] `en/gedecomix`
|
|
- [ ] `en/gingertoon`
|
|
- [ ] `en/goda`
|
|
- [ ] `en/gourmetscans`
|
|
- [ ] `en/greedscans`
|
|
- [ ] `en/grimscans`
|
|
- [ ] `en/grrlpower`
|
|
- [ ] `en/gunnerkriggcourt`
|
|
- [x] `en/guya` ⚠️ see notes
|
|
- [ ] `en/gwtb`
|
|
- [x] `en/hachirumi`
|
|
- [ ] `en/hadesscans`
|
|
- [ ] `en/harimanga`
|
|
- [ ] `en/hentai3zcc`
|
|
- [x] `en/hentai4free`
|
|
- [ ] `en/hentaidex`
|
|
- [ ] `en/hentaihere`
|
|
- [ ] `en/hentaikun`
|
|
- [ ] `en/hentainexus`
|
|
- [ ] `en/hentairead` → base: Madara (complex)
|
|
- [ ] `en/hentaireadio`
|
|
- [ ] `en/hentaisco`
|
|
- [ ] `en/hentaixcomic`
|
|
- [ ] `en/hentaixdickgirl`
|
|
- [ ] `en/hentaixyuri`
|
|
- [ ] `en/hentara`
|
|
- [ ] `en/heytoon`
|
|
- [x] `en/hijalascans`
|
|
- [ ] `en/hiperdex`
|
|
- [ ] `en/hiveworks`
|
|
- [ ] `en/hm2d`
|
|
- [ ] `en/honkaiimpact`
|
|
- [ ] `en/hotcomics`
|
|
- [ ] `en/hyakuro`
|
|
- [ ] `en/infernalvoidscans`
|
|
- [ ] `en/infinityscans`
|
|
- [ ] `en/irovedout`
|
|
- [ ] `en/isekaiscantop`
|
|
- [x] `en/jinmangas`
|
|
- [ ] `en/jnovel`
|
|
- [ ] `en/kagane`
|
|
- [x] `en/kaizenscan`
|
|
- [x] `en/kaliscancom` → base: MadTheme
|
|
- [x] `en/kaliscanio` → base: MadTheme
|
|
- [x] `en/kaliscanme`
|
|
- [ ] `en/kappabeast`
|
|
- [ ] `en/kaynscans`
|
|
- [ ] `en/keenspot`
|
|
- [ ] `en/kenscans`
|
|
- [x] `en/kewnscans`
|
|
- [ ] `en/killsixbilliondemons`
|
|
- [ ] `en/kingcomix`
|
|
- [x] `en/kingofshojo`
|
|
- [x] `en/kissmangain`
|
|
- [ ] `en/kmanga`
|
|
- [ ] `en/kodansha`
|
|
- [ ] `en/ksgroupscans`
|
|
- [x] `en/kunmanga`
|
|
- [ ] `en/kuramanga`
|
|
- [ ] `en/lagoonscans`
|
|
- [ ] `en/leslievictims`
|
|
- [x] `en/lhtranslation`
|
|
- [ ] `en/likemanga`
|
|
- [ ] `en/likemangain`
|
|
- [ ] `en/lilymanga`
|
|
- [x] `en/linkmanga` → base: Madara
|
|
- [ ] `en/loadingartist`
|
|
- [ ] `en/luascans`
|
|
- [ ] `en/luminaretranslations`
|
|
- [x] `en/lunatoons`
|
|
- [ ] `en/madaradex`
|
|
- [ ] `en/madarascans`
|
|
- [ ] `en/madokami`
|
|
- [ ] `en/magusmanga`
|
|
- [ ] `en/mahouirexnohentaikarte`
|
|
- [ ] `en/manga18club`
|
|
- [ ] `en/manga18free`
|
|
- [ ] `en/manga18fx`
|
|
- [x] `en/manga18x`
|
|
- [ ] `en/mangabat`
|
|
- [ ] `en/mangablaze`
|
|
- [ ] `en/mangabolt`
|
|
- [ ] `en/mangabtt`
|
|
- [x] `en/mangabuddy` → base: MadTheme
|
|
- [x] `en/mangabuddyme` → base: MadTheme
|
|
- [ ] `en/mangaclash`
|
|
- [ ] `en/mangacloud`
|
|
- [x] `en/mangacute` → base: MadTheme
|
|
- [ ] `en/mangadass`
|
|
- [ ] `en/mangade`
|
|
- [ ] `en/mangademon`
|
|
- [ ] `en/mangadia`
|
|
- [ ] `en/mangadistrict`
|
|
- [ ] `en/mangadotnet`
|
|
- [ ] `en/mangadrama`
|
|
- [x] `en/mangafab` → base: MadTheme
|
|
- [x] `en/mangaforest` → base: MadTheme
|
|
- [ ] `en/mangaforfreecom`
|
|
- [ ] `en/mangafox`
|
|
- [x] `en/mangafoxfun` → base: MangaHub
|
|
- [ ] `en/mangafreak`
|
|
- [ ] `en/mangafree`
|
|
- [x] `en/mangagg`
|
|
- [ ] `en/mangago`
|
|
- [ ] `en/mangagofun`
|
|
- [x] `en/mangahe`
|
|
- [ ] `en/mangahen`
|
|
- [x] `en/mangahentai`
|
|
- [ ] `en/mangahere`
|
|
- [x] `en/mangahereonl` → base: MangaHub
|
|
- [x] `en/mangahubio` → base: MangaHub
|
|
- [ ] `en/mangaka`
|
|
- [ ] `en/mangakakalot`
|
|
- [x] `en/mangakakalotfun` → base: MangaHub
|
|
- [ ] `en/mangakatana`
|
|
- [ ] `en/mangakiss`
|
|
- [ ] `en/mangamaniacs`
|
|
- [ ] `en/mangamo`
|
|
- [ ] `en/mangamob`
|
|
- [x] `en/mangamonk` → base: MadTheme
|
|
- [x] `en/manganel` → base: MangaHub
|
|
- [ ] `en/manganelo`
|
|
- [ ] `en/manganow`
|
|
- [x] `en/mangaonlinefun` → base: MangaHub
|
|
- [ ] `en/mangaowlio`
|
|
- [x] `en/mangapandaonl` → base: MangaHub
|
|
- [ ] `en/mangapill`
|
|
- [ ] `en/mangapuma`
|
|
- [ ] `en/mangarawclub`
|
|
- [ ] `en/mangaread`
|
|
- [ ] `en/mangareadercc`
|
|
- [x] `en/mangareadersite` → base: MangaHub
|
|
- [ ] `en/mangareadorg`
|
|
- [x] `en/mangasaga` → base: MadTheme
|
|
- [ ] `en/mangasect`
|
|
- [x] `en/mangaspin` → base: MadTheme
|
|
- [ ] `en/mangasushi`
|
|
- [ ] `en/mangatellers`
|
|
- [x] `en/mangatoday` → base: MangaHub
|
|
- [ ] `en/mangatown`
|
|
- [ ] `en/mangatrend`
|
|
- [ ] `en/mangatx`
|
|
- [x] `en/mangaxyz` → base: MadTheme
|
|
- [ ] `en/manhuafast`
|
|
- [ ] `en/manhuafastnet`
|
|
- [ ] `en/manhuahot`
|
|
- [ ] `en/manhuanext`
|
|
- [x] `en/manhuanow` → base: MadTheme
|
|
- [ ] `en/manhuaplus`
|
|
- [ ] `en/manhuaplusorg`
|
|
- [ ] `en/manhuarush`
|
|
- [ ] `en/manhuascanus`
|
|
- [x] `en/manhuasite` → base: MadTheme
|
|
- [ ] `en/manhuatop`
|
|
- [ ] `en/manhuaus`
|
|
- [ ] `en/manhuazonghe`
|
|
- [ ] `en/manhwa18`
|
|
- [ ] `en/manhwa18org`
|
|
- [ ] `en/manhwa68`
|
|
- [ ] `en/manhwabuddy`
|
|
- [ ] `en/manhwaclan`
|
|
- [ ] `en/manhwacomics`
|
|
- [ ] `en/manhwaden`
|
|
- [ ] `en/manhwaget`
|
|
- [ ] `en/manhwahub`
|
|
- [ ] `en/manhwajoy`
|
|
- [ ] `en/manhwalike`
|
|
- [x] `en/manhwalover`
|
|
- [ ] `en/manhwamanhua`
|
|
- [ ] `en/manhwaread`
|
|
- [x] `en/manhwareads`
|
|
- [ ] `en/manhwatoon`
|
|
- [x] `en/manhwatop`
|
|
- [x] `en/manhwax`
|
|
- [ ] `en/manhwaxxl`
|
|
- [ ] `en/manhwaz`
|
|
- [ ] `en/manhwazone`
|
|
- [ ] `en/manta`
|
|
- [ ] `en/megatokyo`
|
|
- [ ] `en/mehgazone`
|
|
- [ ] `en/meitoon`
|
|
- [x] `en/mgjinx`
|
|
- [ ] `en/mgreadio`
|
|
- [ ] `en/milftoon`
|
|
- [x] `en/mistscans`
|
|
- [ ] `en/mlbblore`
|
|
- [ ] `en/monochromecustom`
|
|
- [x] `en/monochromescans`
|
|
- [ ] `en/multporn`
|
|
- [ ] `en/murimscan`
|
|
- [ ] `en/myhentaicomics`
|
|
- [ ] `en/myhentaigallery`
|
|
- [x] `en/necroscans`
|
|
- [ ] `en/newmanhwa`
|
|
- [ ] `en/nexcomic`
|
|
- [ ] `en/nikatoons`
|
|
- [ ] `en/nineanime`
|
|
- [ ] `en/ninehentai`
|
|
- [ ] `en/ninekon`
|
|
- [ ] `en/novel24h`
|
|
- [ ] `en/novelcrow`
|
|
- [ ] `en/noxenscans`
|
|
- [ ] `en/nuxscans`
|
|
- [x] `en/nyanukafe`
|
|
- [ ] `en/nyrascans`
|
|
- [x] `en/nyxscans`
|
|
- [x] `en/octopusmanga`
|
|
- [ ] `en/oglaf`
|
|
- [ ] `en/ohjoysextoy`
|
|
- [ ] `en/omegascans`
|
|
- [x] `en/onemangaco` → base: MangaHub
|
|
- [x] `en/onemangainfo` → base: MangaHub
|
|
- [ ] `en/onepunchmanonline`
|
|
- [ ] `en/onlythebesthentai`
|
|
- [ ] `en/oots`
|
|
- [ ] `en/oppaistream`
|
|
- [ ] `en/orchisasia`
|
|
- [x] `en/orionscans`
|
|
- [ ] `en/paradisescans`
|
|
- [ ] `en/paragonscans`
|
|
- [ ] `en/paritehaber`
|
|
- [ ] `en/patchfriday`
|
|
- [x] `en/pawmanga`
|
|
- [ ] `en/petrotechsociety`
|
|
- [ ] `en/philiascans`
|
|
- [ ] `en/plutoscans`
|
|
- [ ] `en/pmscans`
|
|
- [ ] `en/porncomix`
|
|
- [ ] `en/qiscans`
|
|
- [ ] `en/questionablecontent`
|
|
- [ ] `en/ragescans`
|
|
- [ ] `en/randowiz`
|
|
- [x] `en/ravenscans`
|
|
- [ ] `en/razure`
|
|
- [ ] `en/rdscans`
|
|
- [ ] `en/readallcomicscom`
|
|
- [x] `en/readattackontitanshingekinokyojinmanga`
|
|
- [x] `en/readberserkmanga`
|
|
- [x] `en/readblackclovermangaonline`
|
|
- [x] `en/readbokunoheroacademiamyheroacademiamanga`
|
|
- [x] `en/readchainsawmanmangaonline`
|
|
- [ ] `en/readcomiconline`
|
|
- [ ] `en/readcomicsonline`
|
|
- [x] `en/readfairytailedenszeromangaonline`
|
|
- [x] `en/readhaikyuumangaonline`
|
|
- [x] `en/readjujutsukaisenmangaonline`
|
|
- [x] `en/readkingdommangaonline`
|
|
- [x] `en/readnanatsunotaizai7deadlysinsmangaonline`
|
|
- [x] `en/readnarutoborutosamurai8mangaonline`
|
|
- [x] `en/readonepiecemangaonline`
|
|
- [x] `en/readonepunchmanmangaonlinetwo`
|
|
- [x] `en/readsololevelingmangamanhwaonline`
|
|
- [x] `en/readtokyoghoulretokyoghoulmangaonline`
|
|
- [ ] `en/readvagabondmanga`
|
|
- [ ] `en/reallifecomics`
|
|
- [ ] `en/reimanga`
|
|
- [x] `en/renascans`
|
|
- [ ] `en/resetscans`
|
|
- [ ] `en/restscans`
|
|
- [ ] `en/retsu`
|
|
- [ ] `en/revivalscans`
|
|
- [ ] `en/rinkocomics`
|
|
- [x] `en/ritharscans`
|
|
- [ ] `en/rizzcomic`
|
|
- [ ] `en/rizzcomicunoriginal`
|
|
- [ ] `en/rokaricomics`
|
|
- [ ] `en/roliascan`
|
|
- [x] `en/rosesquadscans`
|
|
- [x] `en/ryumanga`
|
|
- [ ] `en/s2manga`
|
|
- [x] `en/sanascans`
|
|
- [ ] `en/saturdaymorningbreakfastcomics`
|
|
- [ ] `en/schlockmercenary`
|
|
- [ ] `en/setsuscans`
|
|
- [ ] `en/shibamanga`
|
|
- [ ] `en/shojoscans`
|
|
- [x] `en/sirenscans`
|
|
- [ ] `en/skymanga`
|
|
- [x] `en/sleepytranslations` → base: Madara
|
|
- [ ] `en/solarandsundry`
|
|
- [ ] `en/spmanhwa`
|
|
- [ ] `en/spyfakku`
|
|
- [ ] `en/stonescape`
|
|
- [ ] `en/sunshinebutterflyscans`
|
|
- [ ] `en/supermega`
|
|
- [ ] `en/suryascans`
|
|
- [ ] `en/swordscomic`
|
|
- [ ] `en/tapastic`
|
|
- [ ] `en/tcbscans`
|
|
- [x] `en/tcbscansunoriginal`
|
|
- [ ] `en/teamshadowi`
|
|
- [ ] `en/templescan`
|
|
- [ ] `en/theblank`
|
|
- [ ] `en/theduckwebcomics`
|
|
- [ ] `en/thepropertyofhate`
|
|
- [ ] `en/timelesstoons`
|
|
- [ ] `en/todaymanga`
|
|
- [ ] `en/toon18`
|
|
- [ ] `en/toongod`
|
|
- [ ] `en/toonily`
|
|
- [x] `en/toonilyme`
|
|
- [x] `en/toonitube`
|
|
- [ ] `en/toonizy`
|
|
- [x] `en/topmanhua`
|
|
- [ ] `en/topmanhuafan`
|
|
- [ ] `en/topmanhuanet`
|
|
- [ ] `en/tritiniascans`
|
|
- [ ] `en/utoon`
|
|
- [ ] `en/valirscans`
|
|
- [x] `en/vanillascans`
|
|
- [ ] `en/vgperson`
|
|
- [ ] `en/vizshonenjump`
|
|
- [ ] `en/voyceme`
|
|
- [ ] `en/vyvymanga`
|
|
- [ ] `en/vyvymangaorg`
|
|
- [ ] `en/warforrayuba`
|
|
- [ ] `en/wearehunger`
|
|
- [ ] `en/webcomics`
|
|
- [ ] `en/webdexscans`
|
|
- [ ] `en/webnovel`
|
|
- [ ] `en/webtoonscan`
|
|
- [ ] `en/webtoonxyz`
|
|
- [ ] `en/weebcentral`
|
|
- [ ] `en/whalemanga`
|
|
- [ ] `en/witchscans`
|
|
- [ ] `en/woopread`
|
|
- [ ] `en/writerscans`
|
|
- [ ] `en/wuxiaworld`
|
|
- [ ] `en/xlecx`
|
|
- [ ] `en/xomanga`
|
|
- [ ] `en/xoxocomics`
|
|
- [ ] `en/yakshacomics`
|
|
- [ ] `en/yaoihot`
|
|
- [x] `en/yaoihub`
|
|
- [ ] `en/yaoiscan`
|
|
- [ ] `en/yaoitoon`
|
|
- [ ] `en/yorai`
|
|
- [ ] `en/zazamanga`
|
|
- [ ] `en/zinchanmanga`
|
|
- [ ] `en/zinchanmangacom`
|
|
- [ ] `en/zinmanga`
|
|
- [ ] `en/zinmanganet`
|
|
|
|
---
|
|
|
|
## Notes — Complex Sources
|
|
|
|
### HTTP Client Selection
|
|
All sources use the same unified client — `httpclient.DefaultClient()`. You no longer need
|
|
to decide between direct HTTP and FlareSolverr; the client handles both automatically.
|
|
|
|
In Kotlin, `override val client = network.cloudflareClient` signals the source needs
|
|
Cloudflare bypass — but in Go, the adaptive `Do()` method provides it transparently.
|
|
|
|
### `all/mangadex` ⚠️
|
|
- Rate limit: 5 req/s (`golang.org/x/time/rate`)
|
|
- `GetPopularManga` — `GET /manga?order[rating]=desc&includes[]=cover_art&limit=20&offset={(n-1)*20}`
|
|
- `GetLatestUpdates` — `GET /manga?order[updatedAt]=desc&includes[]=cover_art`
|
|
- `GetSearchManga` — tag filters (`includedTags[]`, `excludedTags[]`), demographic, content rating, status, sort
|
|
- `GetMangaDetails` — `GET /manga/{id}?includes[]=cover_art&includes[]=author&includes[]=artist`; cover URL from relationships
|
|
- `GetChapterList` — `GET /manga/{id}/feed?translatedLanguage[]=en&limit=500`; paginate until all chapters fetched
|
|
- `GetPageList` — `GET /at-home/server/{chapterId}` → `{baseUrl}/{quality}/{hash}/{filename}`; quality = `data` or `data-saver`
|
|
- `GetFilterList` — `GET /manga/tag`; cache tag list
|
|
- Language configurable via `MANGADEX_LANG` env var (default `en`)
|
|
|
|
### `all/nhentaicom` ⚠️
|
|
- FlareSolverr required
|
|
- `GetPageList` — extract JSON blob from `document.getElementById(...)` in `<script>` tag; reconstruct URLs as `{imageServer}/galleries/{mediaId}/{pageNum}.{ext}`
|
|
|
|
### `all/kemono` ⚠️
|
|
- FlareSolverr required
|
|
- Service + creator = manga; posts = chapters; attachment files = pages
|
|
- File URLs: prefix relative paths with `{base}/data`
|
|
- `GetChapterList` — paginate in 50-post increments via `?o={offset}`
|
|
|
|
### `all/komga` ⚠️
|
|
- Self-hosted; `KOMGA_BASE_URL`, `KOMGA_USERNAME`, `KOMGA_PASSWORD` env vars; Basic Auth
|
|
- Series = manga; Books = chapters; pages via `GET /api/v1/books/{bookId}/pages/{n}/thumbnail`
|
|
|
|
### `all/e621` ⚠️
|
|
- Optional Basic Auth (`E621_USERNAME`, `E621_API_KEY`); required for Gold content
|
|
- Pools = manga; posts = pages; `GET /pools.json`, `GET /posts.json?tags=pool:{id}`
|
|
|
|
### `all/danbooru` ⚠️
|
|
- Optional Basic Auth; free accounts limited to 2 tags per search
|
|
- Pools = manga; pool post_ids = pages; image URL from `file_url` field
|
|
|
|
### `all/pixiv` ⚠️
|
|
- Auth: `PIXIV_PHPSESSID` env var → Cookie header
|
|
- `GetPageList` — `GET /ajax/illust/{illustId}/pages`; extract `urls.original`
|
|
- **Must set `Referer: https://www.pixiv.net/`** on all image requests or get 403
|
|
|
|
### `all/luscious` ⚠️
|
|
- GraphQL POST to `https://members.luscious.net/graphql/playground`
|
|
- Albums = manga; `SearchAlbumPictures` paginated for page list
|
|
|
|
### `all/mangaplus` ⚠️
|
|
- JSON endpoint: `GET https://jumpg-webapi.tokyo-cdn.com/api/title_detail?title_id={id}`
|
|
- `GetPageList` — XOR page URL decryption using per-page `encryptionKey` (hex string); implement in `GetImageURL`
|
|
- Required header: `Origin: https://mangaplus.shueisha.co.jp`
|
|
|
|
### `all/stashapp` ⚠️
|
|
- Self-hosted; `STASHAPP_BASE_URL`, `STASHAPP_API_KEY` env vars
|
|
- GraphQL POST to `{base}/graphql`; `ApiKey` header
|
|
|
|
### `all/globalcomix` ⚠️
|
|
- Signed CDN URLs with short expiry — **do NOT cache `image_url` in DB**; always fetch fresh page URLs
|
|
|
|
### `all/webtoons` ⚠️
|
|
- HMAC-SHA256 `Sec-Webtoon-Client-Data` header: `HMAC-SHA256(fixedKey, "{url}_{timestamp}")` → base64; fixed key in extension source
|
|
- Web API: `https://global.apis.naver.com/webtoonSvc/v1/...`
|
|
|
|
### `en/allanime` ⚠️
|
|
- GraphQL POST to `https://api.allanime.day/api`
|
|
- Episode-based chapters; `episodeString` used as chapter number
|
|
- Multiple CDN quality tiers in page response; pick highest
|
|
|
|
### `en/asurascans` ⚠️
|
|
- FlareSolverr required
|
|
- Page selector varies by layout version; check extension source for current selector
|
|
|
|
### `en/bakkin` ⚠️
|
|
- No list/search; all manga from a single JSON URL; enumerate from object keys
|
|
- `GetSearchManga` does client-side title filtering only
|
|
|
|
### `en/bookwalker` ⚠️
|
|
- DRM-protected; metadata only
|
|
- `GetPageList` returns empty slice; pages not accessible without purchase
|
|
|
|
### `en/guya` ⚠️
|
|
- `GET {base}/api/get_all_series/` returns all manga at once; no pagination
|
|
- Scanlation group filter applied client-side in response parsing
|