Some anti-bot challenge pages return HTTP 200 instead of 403/503,
bypassing the FS fallback. Read response body on 200, check for
challenge indicators, and fall through to FlareSolverr if detected.
- Use httpcloak.Session (Chrome JA3/JA4 fingerprint) as primary transport
- Adaptive: direct request via httpcloak first; FlareSolverr fallback on 403/503
- FS cookies fed into httpcloak session so subsequent requests reuse
cf_clearance (Chrome fingerprint + cookie = no re-challenge)
- FlareSolverr timeout increased to 120s for slow challenges
- Sanitize FS cookie values (strip quotes/newlines to avoid Go cookie warnings)
- Remove go-cfscraper dependency (pure JS solver was fragile)
Re-enables the direct-then-FS approach removed earlier due to TLS
fingerprint mismatch. With FS sessions configured, the fallback path
is fast after the first request per domain (Chrome caches cf_clearance).
Non-Cloudflare sites still benefit from direct HTTP speed.
go test -json implicitly sets testing.Verbose()=true, so the old guard
always printed manga lists through the script. Switched to an env var
(SOURCETEST_VERBOSE=1) set by the script only when -v is passed.
- Guard isCloudflareChallenge with directStatus >= 400 to prevent
overriding status to 0 when no direct request was made
- When FS returns challenge page without a prior direct status,
return an error instead of silently passing HTTP 0
- Restore default FS session ID to 'goyomi' — without a session,
each request spawns a new Chrome, causing timeouts under load
- Add Message field to FlareSolverrResponse for better error reporting
- Document FLARESOLVERR_SESSION env var: shared session = fast after
1st request, but serializes. Set empty for parallel (resource-heavy).
When -v is passed, test-sources.sh passes it through to go test -v.
sourcetest.Run uses testing.Verbose() to print the full manga list
from GetPopularManga and GetLatestUpdates, showing title + URL.
- Remove global ProxyEnabled() logic from httpclient
- Each source now explicitly chooses client at import time:
- flare client: for JS-rendering/cloudflare sources
- normal httpclient: for REST API sources
- Updated 29 base sources based on Kotlin reference (network.cloudflareClient)
- Add config table for storing FlareSolverr proxy setting
- Add HTTP endpoints to get/set proxy status (GET/POST /api/config/flaresolverr)
- Refactor httpclient to support proxy mode (requests go through FlareSolverr)
- Add verbose logging for debugging
- Add POST support to FlareSolverr client
Usage:
GET /api/config/flaresolverr - returns {flaresolverr_proxy: bool}
POST /api/config/flaresolverr - body: {enabled: true/false}
- PostgreSQL schema: sources, manga, chapters, pages, source_meta with indexes
- golang-migrate runner with embedded SQL via go:embed (pgx5:// scheme)
- sqlc-generated type-safe queries for all tables (pgx/v5 native)
- Config package with all env vars including TTL durations
- Wire DB init and config into cmd/server/main.go
achmad