Release v1.1.1

* Compare webUI version with bundled webUI version

The bundled webUI version was incorrectly compared with the minimum server version.
This worked until the latest release, because the bundled webUI version had a lower revision number than the server revision, however, with the latest release, it is now higher, resulting in no compatible webUI version to be found

* Consider bundled webUI version only for default flavor

* Add "isDefault" util function to WebUIFlavor

.github/ISSUE_TEMPLATE/bug_report.md

@@ -23,7 +23,7 @@ Note that the issue will be automatically closed if you do not fill out the titl
 ---
 
 ## Device information
-- Suwayomi-Server version: (Example: v1.0.0-r1438-win32)
+- Suwayomi-Server version: (Example: v1.1.1-r1535-win32)
 - Server Operating System: (Example: Ubuntu 20.04)
 - Server Desktop Environment: N/A or (Example: Gnome 40)
 - Server JVM version: bundled with win32 or (Example: Java 8 Update 281 or OpenJDK 8u281)

CHANGELOG.md

@@ -1,3 +1,58 @@
+# Server: v1.1.1 + WevUI: v1.1.0
+## TL;DR
+- WebUI update bugfixes
+
+## Suwayomi-Server Changelog
+- ([r1534](https://github.com/Suwayomi/Suwayomi-Server/commit/d9cb54b28593e4df87522090f03a6e5b9c7d9fa2)) Compare webUI version with bundled webUI version ([#969](https://github.com/Suwayomi/Suwayomi-Server/pull/969) by @schroda)
+- ([r1533](https://github.com/Suwayomi/Suwayomi-Server/commit/f738a162d3cd4582612d4986b3d3887e1c309bdd)) Support for "STABLEPREVIEW" webUI version ([#970](https://github.com/Suwayomi/Suwayomi-Server/pull/970) by @schroda)
+
+# Server: v1.1.0 + WevUI: v1.1.0
+## TL;DR
+- Update Manga Info in browse
+- Full Tracking support
+- Import Tracking from backups
+- Improved support for library filters
+- Improved thumbnail handling
+- Many minor bugfixes
+- WebUI changes: https://github.com/Suwayomi/Suwayomi-WebUI/releases/tag/v1.1.0
+
+## Suwayomi-Server Changelog
+- ([r1531](https://github.com/Suwayomi/Suwayomi-Server/commit/ecd1604e25a17a6ef68d568b5d81e69f6e9f7702)) Update metadata in source browse only if new data is not null ([#962](https://github.com/Suwayomi/Suwayomi-Server/pull/962) by @schroda)
+- ([r1530](https://github.com/Suwayomi/Suwayomi-Server/commit/0f061900afbd4036b4d081bcb3f39e4f60160ac3)) Fix browse source ([#961](https://github.com/Suwayomi/Suwayomi-Server/pull/961) by @Syer10)
+- ([r1529](https://github.com/Suwayomi/Suwayomi-Server/commit/c47f5ea85e75c7119d828a19eda392b2fb9faecd)) [skip ci] doc: add NixOS installation ([#959](https://github.com/Suwayomi/Suwayomi-Server/pull/959) by @RatCornu)
+- ([r1528](https://github.com/Suwayomi/Suwayomi-Server/commit/306eb0e3c774b5a8d0d2d4ade2d7091904fe6e58)) Update manga info when browsing if not in library ([#958](https://github.com/Suwayomi/Suwayomi-Server/pull/958) by @Syer10)
+- ([r1527](https://github.com/Suwayomi/Suwayomi-Server/commit/e64025ded814a15129999586133d84085e0c5779)) Correctly set name of logger ([#956](https://github.com/Suwayomi/Suwayomi-Server/pull/956) by @schroda)
+- ([r1526](https://github.com/Suwayomi/Suwayomi-Server/commit/c1fe2da636b675091ec0ac93162764883938ffc6)) Fix/failing thumbnail requests with http 410 ([#955](https://github.com/Suwayomi/Suwayomi-Server/pull/955) by @schroda)
+- ([r1525](https://github.com/Suwayomi/Suwayomi-Server/commit/ff23f58a4f4e8e0b4d459957f0e0701265e0c364)) Support partial mutation responses ([#954](https://github.com/Suwayomi/Suwayomi-Server/pull/954) by @schroda)
+- ([r1524](https://github.com/Suwayomi/Suwayomi-Server/commit/fc2f5ffdf9c1e8675f9b978031a49fb4ce3af601)) Fix/failing track progress update for logged out trackers ([#953](https://github.com/Suwayomi/Suwayomi-Server/pull/953) by @schroda)
+- ([r1523](https://github.com/Suwayomi/Suwayomi-Server/commit/6dd9ed7fb0816b2f163bec43d04c433099e7e529)) Fix/prevent importing unsupported trackers from backup II ([#945](https://github.com/Suwayomi/Suwayomi-Server/pull/945) by @schroda)
+- ([r1522](https://github.com/Suwayomi/Suwayomi-Server/commit/2f362abb91be875e943b1364eb86d70a4144dd6f)) Prevent importing unsupported tracker from backup ([#944](https://github.com/Suwayomi/Suwayomi-Server/pull/944) by @schroda)
+- ([r1521](https://github.com/Suwayomi/Suwayomi-Server/commit/96807a64cf1b13b6db655d46e90c42717170ce62)) [skip ci] Update README.md ([#941](https://github.com/Suwayomi/Suwayomi-Server/pull/941) by @FumoVite)
+- ([r1520](https://github.com/Suwayomi/Suwayomi-Server/commit/7df5f1c4c4408cfbbd56697ba10f018393df2b4a)) Feature/backup tracking ([#940](https://github.com/Suwayomi/Suwayomi-Server/pull/940) by @schroda)
+- ([r1519](https://github.com/Suwayomi/Suwayomi-Server/commit/cf1ede9cf70a2d72a7ff84b9ead24a394ceee2ce)) Update lastPageRead on chapter update ([#939](https://github.com/Suwayomi/Suwayomi-Server/pull/939) by @schroda)
+- ([r1518](https://github.com/Suwayomi/Suwayomi-Server/commit/729385588a3d8e06ec8be38865a12c47e88f6bcb)) Prevent greater last page read than page count ([#938](https://github.com/Suwayomi/Suwayomi-Server/pull/938) by @schroda)
+- ([r1517](https://github.com/Suwayomi/Suwayomi-Server/commit/668d5cf8f02e35cc53d1430a239ae67837c64f51)) Prevent IndexOutOfBoundsException when removing duplicated chapters ([#935](https://github.com/Suwayomi/Suwayomi-Server/pull/935) by @schroda)
+- ([r1516](https://github.com/Suwayomi/Suwayomi-Server/commit/72b1b5b0f9b86f82a0e203802d9a4b6339277c01)) Exit track progress update early in case new chapter is same as current local ([#937](https://github.com/Suwayomi/Suwayomi-Server/pull/937) by @schroda)
+- ([r1515](https://github.com/Suwayomi/Suwayomi-Server/commit/fbf726c17434212cdf94b39f52a25a0050d77287)) Use "AsyncExecutionStrategy" for mutations ([#932](https://github.com/Suwayomi/Suwayomi-Server/pull/932) by @schroda)
+- ([r1514](https://github.com/Suwayomi/Suwayomi-Server/commit/c441eed84773fdc295e6d004e4f4628453b54659)) Exclude duplicated chapters from auto download limit ([#923](https://github.com/Suwayomi/Suwayomi-Server/pull/923) by @schroda)
+- ([r1513](https://github.com/Suwayomi/Suwayomi-Server/commit/e8e83ed49caac2d25f29073d1bd3b5b385aa2d98)) Remove duplicated mangas from gql "mangas" query ([#924](https://github.com/Suwayomi/Suwayomi-Server/pull/924) by @schroda)
+- ([r1512](https://github.com/Suwayomi/Suwayomi-Server/commit/cdc21b067c1a341d68ea7a9c1ee565dc3959f552)) Fix/recognition of already downloaded chapters ([#922](https://github.com/Suwayomi/Suwayomi-Server/pull/922) by @schroda)
+- ([r1511](https://github.com/Suwayomi/Suwayomi-Server/commit/48e19f7914fee1ea1789b217d5df9b05acb49203)) Feature/auto download of new chapters improve handling of unhandable reuploads ([#921](https://github.com/Suwayomi/Suwayomi-Server/pull/921) by @schroda)
+- ([r1510](https://github.com/Suwayomi/Suwayomi-Server/commit/89dd570b3057bee34643858b4a42bfac7d88a82b)) Add mutation to fetch the latest track data from the tracker ([#920](https://github.com/Suwayomi/Suwayomi-Server/pull/920) by @schroda, @Syer10)
+- ([r1509](https://github.com/Suwayomi/Suwayomi-Server/commit/16474d4328651f1236722556b7f59628a0f9dbda)) Feature/tracking gql add option to delete remote binding on tracker ([#919](https://github.com/Suwayomi/Suwayomi-Server/pull/919) by @schroda, @Syer10)
+- ([r1508](https://github.com/Suwayomi/Suwayomi-Server/commit/9db612bf0317950d0291047b9ee64a0787e49bf2)) Move trigger for track progress update to client ([#918](https://github.com/Suwayomi/Suwayomi-Server/pull/918) by @schroda)
+- ([r1507](https://github.com/Suwayomi/Suwayomi-Server/commit/7d92dbc5c0a47176099eb310eaf17a4788ba2ce4)) Fix/tracking progress update in case local chapter is smaller than remote ([#917](https://github.com/Suwayomi/Suwayomi-Server/pull/917) by @schroda)
+- ([r1506](https://github.com/Suwayomi/Suwayomi-Server/commit/a9efca86870cec6d74f58535e2e007eb6c8831c2)) Add chapter bookmark count field to MangaType ([#912](https://github.com/Suwayomi/Suwayomi-Server/pull/912) by @schroda)
+- ([r1505](https://github.com/Suwayomi/Suwayomi-Server/commit/dbfea5d02b898884fdeb2be2959fe8a73a465704)) Update inLibraryAt timestamp when adding manga to library ([#911](https://github.com/Suwayomi/Suwayomi-Server/pull/911) by @schroda)
+- ([r1504](https://github.com/Suwayomi/Suwayomi-Server/commit/a6b05c4a2759d0d5f834a54cad6c8417fe49a0d2)) Feature/refresh outdated thumbnail url on fetch failure ([#910](https://github.com/Suwayomi/Suwayomi-Server/pull/910) by @schroda)
+- ([r1503](https://github.com/Suwayomi/Suwayomi-Server/commit/6d539d34040c4e95692b57ce4fedfbeaa73083d0)) Fix/update subscription clear data loader cache ([#908](https://github.com/Suwayomi/Suwayomi-Server/pull/908) by @schroda)
+- ([r1502](https://github.com/Suwayomi/Suwayomi-Server/commit/b2aff1efc9e6527e70ba519e5171096394e6ccf7)) Fix MAL after restarting the server ([#903](https://github.com/Suwayomi/Suwayomi-Server/pull/903) by @Syer10)
+- ([r1501](https://github.com/Suwayomi/Suwayomi-Server/commit/8a20a1ef5094efc05426ed420bbde40358fdf2dd)) Add first unread chapter field to MangaType ([#900](https://github.com/Suwayomi/Suwayomi-Server/pull/900) by @schroda)
+- ([r1500](https://github.com/Suwayomi/Suwayomi-Server/commit/33cbfa9751c3ef7a6babfcff9595782cbac5acae)) Fix/electron launch error not logged ([#895](https://github.com/Suwayomi/Suwayomi-Server/pull/895) by @schroda)
+- ([r1499](https://github.com/Suwayomi/Suwayomi-Server/commit/b95a8d44d4bb7c94a04e66b3d6cc0fc101f4880b)) Always fetch thumbnail of manga from local source ([#898](https://github.com/Suwayomi/Suwayomi-Server/pull/898) by @schroda)
+
+## [Suwayomi-WebUI Changelog](https://github.com/Suwayomi/Suwayomi-WebUI/blob/master/CHANGELOG.md#v110-r1689)
+
 # Server: v1.0.0 + WevUI: r1409
 ## TL;DR
 - GraphQL API

README.md

@@ -48,7 +48,7 @@ Here's a list of known clients/user interfaces for Suwayomi-Server:
 - [Tachidesk-Sorayomi](https://github.com/Suwayomi/Tachidesk-Sorayomi): A Flutter front-end for Desktop(Linux, windows, etc.), Web and Android with a User Interface inspired by Tachiyomi.
 - [Tachidesk-VaadinUI](https://github.com/Suwayomi/Tachidesk-VaadinUI): A Web front-end for Suwayomi-Server built with Vaadin.
 - [Suwayomi-VUI](https://github.com/Suwayomi/Suwayomi-VUI): A preview focused web frontend built with svelte with some features the other UIs might not have (migration)
-##### Inctive/Abandoned Clients
+##### Inactive/Abandoned Clients
 - [Tachidesk-qtui](https://github.com/Suwayomi/Tachidesk-qtui): A C++/Qt front-end for mobile devices(Android/linux), feature support is basic.
 - [Tachidesk-GTK](https://github.com/mahor1221/Tachidesk-GTK): A native Rust/GTK desktop client.
 - [Equinox](https://github.com/Suwayomi/Equinox): A web user interface made with Vue.js.
@@ -108,6 +108,21 @@ sudo apt update
 sudo apt install suwayomi-server
 ```
 
+### NixOS
+You can deploy Suwayomi on NixOS using the module `services.suwayomi-server` in your configuration:
+
+```
+{
+  services.suwayomi-server = {
+    enable = true;
+  };
+}
+```
+
+For more information, see [the NixOS manual](https://nixos.org/manual/nixos/stable/#module-services-suwayomi-server).
+
+You can also directly use the package from [nixpkgs](https://search.nixos.org/packages?channel=unstable&type=packages&query=suwayomi-server).
+
 ### Docker
 Check our Official Docker release [Suwayomi Container](https://github.com/orgs/Suwayomi/packages/container/package/tachidesk) for running Suwayomi Server in a docker container. Source code for our container is available at [docker-tachidesk](https://github.com/Suwayomi/docker-tachidesk). By default, the server will be running on http://localhost:4567 open this url in your browser.
 

buildSrc/src/main/kotlin/Constants.kt

@@ -10,9 +10,9 @@ import java.io.BufferedReader
 const val MainClass = "suwayomi.tachidesk.MainKt"
 
 // should be bumped with each stable release
-val tachideskVersion = System.getenv("ProductVersion") ?: "v1.0.0"
+val tachideskVersion = System.getenv("ProductVersion") ?: "v1.1.1"
 
-val webUIRevisionTag = System.getenv("WebUIRevision") ?: "r1409"
+val webUIRevisionTag = System.getenv("WebUIRevision") ?: "r1689"
 
 // counts commits on the current checked out branch
 val getTachideskRevision = {

server/src/main/kotlin/suwayomi/tachidesk/graphql/AsDataFetcherResult.kt

@@ -0,0 +1,25 @@
+package suwayomi.tachidesk.graphql
+
+import com.expediagroup.graphql.server.extensions.toGraphQLError
+import graphql.execution.DataFetcherResult
+import mu.KotlinLogging
+
+val logger = KotlinLogging.logger { }
+
+inline fun <T> asDataFetcherResult(block: () -> T): DataFetcherResult<T?> {
+    val result =
+        runCatching {
+            block()
+        }
+
+    if (result.isFailure) {
+        logger.error(result.exceptionOrNull()) { "asDataFetcherResult: failed due to" }
+        return DataFetcherResult.newResult<T?>()
+            .error(result.exceptionOrNull()?.toGraphQLError())
+            .build()
+    }
+
+    return DataFetcherResult.newResult<T?>()
+        .data(result.getOrNull())
+        .build()
+}

server/src/main/kotlin/suwayomi/tachidesk/graphql/cache/CustomCacheMap.kt

@@ -0,0 +1,46 @@
+package suwayomi.tachidesk.graphql.cache
+
+import org.dataloader.CacheMap
+import java.util.concurrent.CompletableFuture
+
+class CustomCacheMap<K, V> : CacheMap<K, V> {
+    private val cache: MutableMap<K, CompletableFuture<V>>
+
+    init {
+        cache = HashMap()
+    }
+
+    override fun containsKey(key: K): Boolean {
+        return cache.containsKey(key)
+    }
+
+    override fun get(key: K): CompletableFuture<V> {
+        return cache[key]!!
+    }
+
+    fun getKeys(): Collection<K> {
+        return cache.keys.toSet()
+    }
+
+    override fun getAll(): Collection<CompletableFuture<V>> {
+        return cache.values
+    }
+
+    override fun set(
+        key: K,
+        value: CompletableFuture<V>,
+    ): CacheMap<K, V> {
+        cache[key] = value
+        return this
+    }
+
+    override fun delete(key: K): CacheMap<K, V> {
+        cache.remove(key)
+        return this
+    }
+
+    override fun clear(): CacheMap<K, V> {
+        cache.clear()
+        return this
+    }
+}

server/src/main/kotlin/suwayomi/tachidesk/graphql/dataLoaders/ChapterDataLoader.kt

@@ -99,6 +99,26 @@ class UnreadChapterCountForMangaDataLoader : KotlinDataLoader<Int, Int> {
         }
 }
 
+class BookmarkedChapterCountForMangaDataLoader : KotlinDataLoader<Int, Int> {
+    override val dataLoaderName = "BookmarkedChapterCountForMangaDataLoader"
+
+    override fun getDataLoader(): DataLoader<Int, Int> =
+        DataLoaderFactory.newDataLoader<Int, Int> { ids ->
+            future {
+                transaction {
+                    addLogger(Slf4jSqlDebugLogger)
+                    val bookmarkedChapterCountByMangaId =
+                        ChapterTable
+                            .slice(ChapterTable.manga, ChapterTable.isBookmarked.count())
+                            .select { (ChapterTable.manga inList ids) and (ChapterTable.isBookmarked eq true) }
+                            .groupBy(ChapterTable.manga)
+                            .associate { it[ChapterTable.manga].value to it[ChapterTable.isBookmarked.count()] }
+                    ids.map { bookmarkedChapterCountByMangaId[it]?.toInt() ?: 0 }
+                }
+            }
+        }
+}
+
 class LastReadChapterForMangaDataLoader : KotlinDataLoader<Int, ChapterType?> {
     override val dataLoaderName = "LastReadChapterForMangaDataLoader"
 
@@ -174,3 +194,22 @@ class LatestUploadedChapterForMangaDataLoader : KotlinDataLoader<Int, ChapterTyp
             }
         }
 }
+
+class FirstUnreadChapterForMangaDataLoader : KotlinDataLoader<Int, ChapterType?> {
+    override val dataLoaderName = "FirstUnreadChapterForMangaDataLoader"
+
+    override fun getDataLoader(): DataLoader<Int, ChapterType?> =
+        DataLoaderFactory.newDataLoader<Int, ChapterType?> { ids ->
+            future {
+                transaction {
+                    addLogger(Slf4jSqlDebugLogger)
+                    val firstUnreadChaptersByMangaId =
+                        ChapterTable
+                            .select { (ChapterTable.manga inList ids) and (ChapterTable.isRead eq false) }
+                            .orderBy(ChapterTable.sourceOrder to SortOrder.ASC)
+                            .groupBy { it[ChapterTable.manga].value }
+                    ids.map { id -> firstUnreadChaptersByMangaId[id]?.let { chapters -> ChapterType(chapters.first()) } }
+                }
+            }
+        }
+}

server/src/main/kotlin/suwayomi/tachidesk/graphql/dataLoaders/MangaDataLoader.kt

@@ -10,11 +10,13 @@ package suwayomi.tachidesk.graphql.dataLoaders
 import com.expediagroup.graphql.dataloader.KotlinDataLoader
 import org.dataloader.DataLoader
 import org.dataloader.DataLoaderFactory
+import org.dataloader.DataLoaderOptions
 import org.jetbrains.exposed.sql.Slf4jSqlDebugLogger
 import org.jetbrains.exposed.sql.addLogger
 import org.jetbrains.exposed.sql.andWhere
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
+import suwayomi.tachidesk.graphql.cache.CustomCacheMap
 import suwayomi.tachidesk.graphql.types.MangaNodeList
 import suwayomi.tachidesk.graphql.types.MangaNodeList.Companion.toNodeList
 import suwayomi.tachidesk.graphql.types.MangaType
@@ -95,18 +97,21 @@ class MangaForIdsDataLoader : KotlinDataLoader<List<Int>, MangaNodeList> {
     override val dataLoaderName = "MangaForIdsDataLoader"
 
     override fun getDataLoader(): DataLoader<List<Int>, MangaNodeList> =
-        DataLoaderFactory.newDataLoader { mangaIds ->
+        DataLoaderFactory.newDataLoader(
-            future {
+            { mangaIds ->
-                transaction {
+                future {
-                    addLogger(Slf4jSqlDebugLogger)
+                    transaction {
-                    val ids = mangaIds.flatten().distinct()
+                        addLogger(Slf4jSqlDebugLogger)
-                    val manga =
+                        val ids = mangaIds.flatten().distinct()
-                        MangaTable.select { MangaTable.id inList ids }
+                        val manga =
-                            .map { MangaType(it) }
+                            MangaTable.select { MangaTable.id inList ids }
-                    mangaIds.map { mangaIds ->
+                                .map { MangaType(it) }
-                        manga.filter { it.id in mangaIds }.toNodeList()
+                        mangaIds.map { mangaIds ->
+                            manga.filter { it.id in mangaIds }.toNodeList()
+                        }
                     }
                 }
-            }
+            },
-        }
+            DataLoaderOptions.newOptions().setCacheMap(CustomCacheMap<List<Int>, MangaNodeList>()),
+        )
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/CategoryMutation.kt

@@ -1,5 +1,6 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import graphql.execution.DataFetcherResult
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.minus
@@ -12,6 +13,7 @@ import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.selectAll
 import org.jetbrains.exposed.sql.transactions.transaction
 import org.jetbrains.exposed.sql.update
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.CategoryMetaType
 import suwayomi.tachidesk.graphql.types.CategoryType
 import suwayomi.tachidesk.graphql.types.MangaType
@@ -36,12 +38,14 @@ class CategoryMutation {
         val meta: CategoryMetaType,
     )
 
-    fun setCategoryMeta(input: SetCategoryMetaInput): SetCategoryMetaPayload {
+    fun setCategoryMeta(input: SetCategoryMetaInput): DataFetcherResult<SetCategoryMetaPayload?> {
-        val (clientMutationId, meta) = input
+        return asDataFetcherResult {
+            val (clientMutationId, meta) = input
 
-        Category.modifyMeta(meta.categoryId, meta.key, meta.value)
+            Category.modifyMeta(meta.categoryId, meta.key, meta.value)
 
-        return SetCategoryMetaPayload(clientMutationId, meta)
+            SetCategoryMetaPayload(clientMutationId, meta)
+        }
     }
 
     data class DeleteCategoryMetaInput(
@@ -56,30 +60,32 @@ class CategoryMutation {
         val category: CategoryType,
     )
 
-    fun deleteCategoryMeta(input: DeleteCategoryMetaInput): DeleteCategoryMetaPayload {
+    fun deleteCategoryMeta(input: DeleteCategoryMetaInput): DataFetcherResult<DeleteCategoryMetaPayload?> {
-        val (clientMutationId, categoryId, key) = input
+        return asDataFetcherResult {
+            val (clientMutationId, categoryId, key) = input
 
-        val (meta, category) =
+            val (meta, category) =
-            transaction {
+                transaction {
-                val meta =
+                    val meta =
-                    CategoryMetaTable.select { (CategoryMetaTable.ref eq categoryId) and (CategoryMetaTable.key eq key) }
+                        CategoryMetaTable.select { (CategoryMetaTable.ref eq categoryId) and (CategoryMetaTable.key eq key) }
-                        .firstOrNull()
+                            .firstOrNull()
 
-                CategoryMetaTable.deleteWhere { (CategoryMetaTable.ref eq categoryId) and (CategoryMetaTable.key eq key) }
+                    CategoryMetaTable.deleteWhere { (CategoryMetaTable.ref eq categoryId) and (CategoryMetaTable.key eq key) }
 
-                val category =
+                    val category =
-                    transaction {
+                        transaction {
-                        CategoryType(CategoryTable.select { CategoryTable.id eq categoryId }.first())
+                            CategoryType(CategoryTable.select { CategoryTable.id eq categoryId }.first())
-                    }
+                        }
 
-                if (meta != null) {
+                    if (meta != null) {
-                    CategoryMetaType(meta)
+                        CategoryMetaType(meta)
-                } else {
+                    } else {
-                    null
+                        null
-                } to category
+                    } to category
-            }
+                }
 
-        return DeleteCategoryMetaPayload(clientMutationId, meta, category)
+            DeleteCategoryMetaPayload(clientMutationId, meta, category)
+        }
     }
 
     data class UpdateCategoryPatch(
@@ -147,36 +153,40 @@ class CategoryMutation {
         }
     }
 
-    fun updateCategory(input: UpdateCategoryInput): UpdateCategoryPayload {
+    fun updateCategory(input: UpdateCategoryInput): DataFetcherResult<UpdateCategoryPayload?> {
-        val (clientMutationId, id, patch) = input
+        return asDataFetcherResult {
+            val (clientMutationId, id, patch) = input
 
-        updateCategories(listOf(id), patch)
+            updateCategories(listOf(id), patch)
 
-        val category =
+            val category =
-            transaction {
+                transaction {
-                CategoryType(CategoryTable.select { CategoryTable.id eq id }.first())
+                    CategoryType(CategoryTable.select { CategoryTable.id eq id }.first())
-            }
+                }
 
-        return UpdateCategoryPayload(
+            UpdateCategoryPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            category = category,
+                category = category,
-        )
+            )
+        }
     }
 
-    fun updateCategories(input: UpdateCategoriesInput): UpdateCategoriesPayload {
+    fun updateCategories(input: UpdateCategoriesInput): DataFetcherResult<UpdateCategoriesPayload?> {
-        val (clientMutationId, ids, patch) = input
+        return asDataFetcherResult {
+            val (clientMutationId, ids, patch) = input
 
-        updateCategories(ids, patch)
+            updateCategories(ids, patch)
 
-        val categories =
+            val categories =
-            transaction {
+                transaction {
-                CategoryTable.select { CategoryTable.id inList ids }.map { CategoryType(it) }
+                    CategoryTable.select { CategoryTable.id inList ids }.map { CategoryType(it) }
-            }
+                }
 
-        return UpdateCategoriesPayload(
+            UpdateCategoriesPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            categories = categories,
+                categories = categories,
-        )
+            )
+        }
     }
 
     data class UpdateCategoryOrderPayload(
@@ -190,46 +200,48 @@ class CategoryMutation {
         val position: Int,
     )
 
-    fun updateCategoryOrder(input: UpdateCategoryOrderInput): UpdateCategoryOrderPayload {
+    fun updateCategoryOrder(input: UpdateCategoryOrderInput): DataFetcherResult<UpdateCategoryOrderPayload?> {
-        val (clientMutationId, categoryId, position) = input
+        return asDataFetcherResult {
-        require(position > 0) {
+            val (clientMutationId, categoryId, position) = input
-            "'order' must not be <= 0"
+            require(position > 0) {
-        }
+                "'order' must not be <= 0"
-
-        transaction {
-            val currentOrder =
-                CategoryTable
-                    .select { CategoryTable.id eq categoryId }
-                    .first()[CategoryTable.order]
-
-            if (currentOrder != position) {
-                if (position < currentOrder) {
-                    CategoryTable.update({ CategoryTable.order greaterEq position }) {
-                        it[CategoryTable.order] = CategoryTable.order + 1
-                    }
-                } else {
-                    CategoryTable.update({ CategoryTable.order lessEq position }) {
-                        it[CategoryTable.order] = CategoryTable.order - 1
-                    }
-                }
-
-                CategoryTable.update({ CategoryTable.id eq categoryId }) {
-                    it[CategoryTable.order] = position
-                }
             }
-        }
 
-        Category.normalizeCategories()
-
-        val categories =
             transaction {
-                CategoryTable.selectAll().orderBy(CategoryTable.order).map { CategoryType(it) }
+                val currentOrder =
+                    CategoryTable
+                        .select { CategoryTable.id eq categoryId }
+                        .first()[CategoryTable.order]
+
+                if (currentOrder != position) {
+                    if (position < currentOrder) {
+                        CategoryTable.update({ CategoryTable.order greaterEq position }) {
+                            it[CategoryTable.order] = CategoryTable.order + 1
+                        }
+                    } else {
+                        CategoryTable.update({ CategoryTable.order lessEq position }) {
+                            it[CategoryTable.order] = CategoryTable.order - 1
+                        }
+                    }
+
+                    CategoryTable.update({ CategoryTable.id eq categoryId }) {
+                        it[CategoryTable.order] = position
+                    }
+                }
             }
 
-        return UpdateCategoryOrderPayload(
+            Category.normalizeCategories()
-            clientMutationId = clientMutationId,
+
-            categories = categories,
+            val categories =
-        )
+                transaction {
+                    CategoryTable.selectAll().orderBy(CategoryTable.order).map { CategoryType(it) }
+                }
+
+            UpdateCategoryOrderPayload(
+                clientMutationId = clientMutationId,
+                categories = categories,
+            )
+        }
     }
 
     data class CreateCategoryInput(
@@ -246,51 +258,53 @@ class CategoryMutation {
         val category: CategoryType,
     )
 
-    fun createCategory(input: CreateCategoryInput): CreateCategoryPayload {
+    fun createCategory(input: CreateCategoryInput): DataFetcherResult<CreateCategoryPayload?> {
-        val (clientMutationId, name, order, default, includeInUpdate, includeInDownload) = input
+        return asDataFetcherResult {
-        transaction {
+            val (clientMutationId, name, order, default, includeInUpdate, includeInDownload) = input
-            require(CategoryTable.select { CategoryTable.name eq input.name }.isEmpty()) {
-                "'name' must be unique"
-            }
-        }
-        require(!name.equals(Category.DEFAULT_CATEGORY_NAME, ignoreCase = true)) {
-            "'name' must not be ${Category.DEFAULT_CATEGORY_NAME}"
-        }
-        if (order != null) {
-            require(order > 0) {
-                "'order' must not be <= 0"
-            }
-        }
-
-        val category =
             transaction {
-                if (order != null) {
+                require(CategoryTable.select { CategoryTable.name eq input.name }.isEmpty()) {
-                    CategoryTable.update({ CategoryTable.order greaterEq order }) {
+                    "'name' must be unique"
-                        it[CategoryTable.order] = CategoryTable.order + 1
+                }
+            }
+            require(!name.equals(Category.DEFAULT_CATEGORY_NAME, ignoreCase = true)) {
+                "'name' must not be ${Category.DEFAULT_CATEGORY_NAME}"
+            }
+            if (order != null) {
+                require(order > 0) {
+                    "'order' must not be <= 0"
+                }
+            }
+
+            val category =
+                transaction {
+                    if (order != null) {
+                        CategoryTable.update({ CategoryTable.order greaterEq order }) {
+                            it[CategoryTable.order] = CategoryTable.order + 1
+                        }
                     }
+
+                    val id =
+                        CategoryTable.insertAndGetId {
+                            it[CategoryTable.name] = input.name
+                            it[CategoryTable.order] = order ?: Int.MAX_VALUE
+                            if (default != null) {
+                                it[CategoryTable.isDefault] = default
+                            }
+                            if (includeInUpdate != null) {
+                                it[CategoryTable.includeInUpdate] = includeInUpdate.value
+                            }
+                            if (includeInDownload != null) {
+                                it[CategoryTable.includeInDownload] = includeInDownload.value
+                            }
+                        }
+
+                    Category.normalizeCategories()
+
+                    CategoryType(CategoryTable.select { CategoryTable.id eq id }.first())
                 }
 
-                val id =
+            CreateCategoryPayload(clientMutationId, category)
-                    CategoryTable.insertAndGetId {
+        }
-                        it[CategoryTable.name] = input.name
-                        it[CategoryTable.order] = order ?: Int.MAX_VALUE
-                        if (default != null) {
-                            it[CategoryTable.isDefault] = default
-                        }
-                        if (includeInUpdate != null) {
-                            it[CategoryTable.includeInUpdate] = includeInUpdate.value
-                        }
-                        if (includeInDownload != null) {
-                            it[CategoryTable.includeInDownload] = includeInDownload.value
-                        }
-                    }
-
-                Category.normalizeCategories()
-
-                CategoryType(CategoryTable.select { CategoryTable.id eq id }.first())
-            }
-
-        return CreateCategoryPayload(clientMutationId, category)
     }
 
     data class DeleteCategoryInput(
@@ -304,41 +318,43 @@ class CategoryMutation {
         val mangas: List<MangaType>,
     )
 
-    fun deleteCategory(input: DeleteCategoryInput): DeleteCategoryPayload {
+    fun deleteCategory(input: DeleteCategoryInput): DataFetcherResult<DeleteCategoryPayload?> {
-        val (clientMutationId, categoryId) = input
+        return asDataFetcherResult {
-        if (categoryId == 0) { // Don't delete default category
+            val (clientMutationId, categoryId) = input
-            return DeleteCategoryPayload(
+            if (categoryId == 0) { // Don't delete default category
-                clientMutationId,
+                return@asDataFetcherResult DeleteCategoryPayload(
-                null,
+                    clientMutationId,
-                emptyList(),
+                    null,
-            )
+                    emptyList(),
-        }
+                )
-
-        val (category, mangas) =
-            transaction {
-                val category =
-                    CategoryTable.select { CategoryTable.id eq categoryId }
-                        .firstOrNull()
-
-                val mangas =
-                    transaction {
-                        MangaTable.innerJoin(CategoryMangaTable)
-                            .select { CategoryMangaTable.category eq categoryId }
-                            .map { MangaType(it) }
-                    }
-
-                CategoryTable.deleteWhere { CategoryTable.id eq categoryId }
-
-                Category.normalizeCategories()
-
-                if (category != null) {
-                    CategoryType(category)
-                } else {
-                    null
-                } to mangas
             }
 
-        return DeleteCategoryPayload(clientMutationId, category, mangas)
+            val (category, mangas) =
+                transaction {
+                    val category =
+                        CategoryTable.select { CategoryTable.id eq categoryId }
+                            .firstOrNull()
+
+                    val mangas =
+                        transaction {
+                            MangaTable.innerJoin(CategoryMangaTable)
+                                .select { CategoryMangaTable.category eq categoryId }
+                                .map { MangaType(it) }
+                        }
+
+                    CategoryTable.deleteWhere { CategoryTable.id eq categoryId }
+
+                    Category.normalizeCategories()
+
+                    if (category != null) {
+                        CategoryType(category)
+                    } else {
+                        null
+                    } to mangas
+                }
+
+            DeleteCategoryPayload(clientMutationId, category, mangas)
+        }
     }
 
     data class UpdateMangaCategoriesPatch(
@@ -406,35 +422,39 @@ class CategoryMutation {
         }
     }
 
-    fun updateMangaCategories(input: UpdateMangaCategoriesInput): UpdateMangaCategoriesPayload {
+    fun updateMangaCategories(input: UpdateMangaCategoriesInput): DataFetcherResult<UpdateMangaCategoriesPayload?> {
-        val (clientMutationId, id, patch) = input
+        return asDataFetcherResult {
+            val (clientMutationId, id, patch) = input
 
-        updateMangas(listOf(id), patch)
+            updateMangas(listOf(id), patch)
 
-        val manga =
+            val manga =
-            transaction {
+                transaction {
-                MangaType(MangaTable.select { MangaTable.id eq id }.first())
+                    MangaType(MangaTable.select { MangaTable.id eq id }.first())
-            }
+                }
 
-        return UpdateMangaCategoriesPayload(
+            UpdateMangaCategoriesPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            manga = manga,
+                manga = manga,
-        )
+            )
+        }
     }
 
-    fun updateMangasCategories(input: UpdateMangasCategoriesInput): UpdateMangasCategoriesPayload {
+    fun updateMangasCategories(input: UpdateMangasCategoriesInput): DataFetcherResult<UpdateMangasCategoriesPayload?> {
-        val (clientMutationId, ids, patch) = input
+        return asDataFetcherResult {
+            val (clientMutationId, ids, patch) = input
 
-        updateMangas(ids, patch)
+            updateMangas(ids, patch)
 
-        val mangas =
+            val mangas =
-            transaction {
+                transaction {
-                MangaTable.select { MangaTable.id inList ids }.map { MangaType(it) }
+                    MangaTable.select { MangaTable.id inList ids }.map { MangaType(it) }
-            }
+                }
 
-        return UpdateMangasCategoriesPayload(
+            UpdateMangasCategoriesPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            mangas = mangas,
+                mangas = mangas,
-        )
+            )
+        }
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/ChapterMutation.kt

@@ -1,16 +1,18 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import graphql.execution.DataFetcherResult
+import org.jetbrains.exposed.dao.id.EntityID
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
 import org.jetbrains.exposed.sql.and
 import org.jetbrains.exposed.sql.deleteWhere
 import org.jetbrains.exposed.sql.select
+import org.jetbrains.exposed.sql.statements.BatchUpdateStatement
 import org.jetbrains.exposed.sql.transactions.transaction
-import org.jetbrains.exposed.sql.update
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.ChapterMetaType
 import suwayomi.tachidesk.graphql.types.ChapterType
 import suwayomi.tachidesk.manga.impl.Chapter
 import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReadyById
-import suwayomi.tachidesk.manga.impl.track.Track
 import suwayomi.tachidesk.manga.model.table.ChapterMetaTable
 import suwayomi.tachidesk.manga.model.table.ChapterTable
 import suwayomi.tachidesk.server.JavalinSetup.future
@@ -56,61 +58,73 @@ class ChapterMutation {
         patch: UpdateChapterPatch,
     ) {
         transaction {
+            val chapterIdToPageCount =
+                if (patch.lastPageRead != null) {
+                    ChapterTable
+                        .slice(ChapterTable.id, ChapterTable.pageCount)
+                        .select { ChapterTable.id inList ids }
+                        .groupBy { it[ChapterTable.id].value }
+                        .mapValues { it.value.firstOrNull()?.let { it[ChapterTable.pageCount] } }
+                } else {
+                    emptyMap()
+                }
             if (patch.isRead != null || patch.isBookmarked != null || patch.lastPageRead != null) {
                 val now = Instant.now().epochSecond
-                ChapterTable.update({ ChapterTable.id inList ids }) { update ->
+
-                    patch.isRead?.also {
+                BatchUpdateStatement(ChapterTable).apply {
-                        update[isRead] = it
+                    ids.forEach { chapterId ->
+                        addBatch(EntityID(chapterId, ChapterTable))
+                        patch.isRead?.also {
+                            this[ChapterTable.isRead] = it
+                        }
+                        patch.isBookmarked?.also {
+                            this[ChapterTable.isBookmarked] = it
+                        }
+                        patch.lastPageRead?.also {
+                            this[ChapterTable.lastPageRead] = it.coerceAtMost(chapterIdToPageCount[chapterId] ?: 0).coerceAtLeast(0)
+                            this[ChapterTable.lastReadAt] = now
+                        }
                     }
-                    patch.isBookmarked?.also {
+                    execute(this@transaction)
-                        update[isBookmarked] = it
-                    }
-                    patch.lastPageRead?.also {
-                        update[lastPageRead] = it
-                        update[lastReadAt] = now
-                    }
-                }
-                if (patch.isRead == true) {
-                    val mangaIds =
-                        ChapterTable.slice(ChapterTable.manga).select { ChapterTable.id inList ids }
-                            .map { it[ChapterTable.manga].value }
-                            .toSet()
-                    Track.asyncTrackChapter(mangaIds)
                 }
             }
         }
     }
 
-    fun updateChapter(input: UpdateChapterInput): UpdateChapterPayload {
+    fun updateChapter(input: UpdateChapterInput): DataFetcherResult<UpdateChapterPayload?> {
-        val (clientMutationId, id, patch) = input
+        return asDataFetcherResult {
+            val (clientMutationId, id, patch) = input
 
-        updateChapters(listOf(id), patch)
+            updateChapters(listOf(id), patch)
 
-        val chapter =
+            val chapter =
-            transaction {
+                transaction {
-                ChapterType(ChapterTable.select { ChapterTable.id eq id }.first())
+                    ChapterType(ChapterTable.select { ChapterTable.id eq id }.first())
-            }
+                }
 
-        return UpdateChapterPayload(
+            UpdateChapterPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            chapter = chapter,
+                chapter = chapter,
-        )
+            )
+        }
     }
 
-    fun updateChapters(input: UpdateChaptersInput): UpdateChaptersPayload {
+    fun updateChapters(input: UpdateChaptersInput): DataFetcherResult<UpdateChaptersPayload?> {
-        val (clientMutationId, ids, patch) = input
+        return asDataFetcherResult {
+            val (clientMutationId, ids, patch) = input
 
-        updateChapters(ids, patch)
+            updateChapters(ids, patch)
 
-        val chapters =
+            val chapters =
-            transaction {
+                transaction {
-                ChapterTable.select { ChapterTable.id inList ids }.map { ChapterType(it) }
+                    ChapterTable.select { ChapterTable.id inList ids }.map { ChapterType(it) }
-            }
+                }
 
-        return UpdateChaptersPayload(
+            UpdateChaptersPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            chapters = chapters,
+                chapters = chapters,
-        )
+            )
+        }
     }
 
     data class FetchChaptersInput(
@@ -123,23 +137,25 @@ class ChapterMutation {
         val chapters: List<ChapterType>,
     )
 
-    fun fetchChapters(input: FetchChaptersInput): CompletableFuture<FetchChaptersPayload> {
+    fun fetchChapters(input: FetchChaptersInput): CompletableFuture<DataFetcherResult<FetchChaptersPayload?>> {
         val (clientMutationId, mangaId) = input
 
         return future {
-            Chapter.fetchChapterList(mangaId)
+            asDataFetcherResult {
-        }.thenApply {
+                Chapter.fetchChapterList(mangaId)
-            val chapters =
-                transaction {
-                    ChapterTable.select { ChapterTable.manga eq mangaId }
-                        .orderBy(ChapterTable.sourceOrder)
-                        .map { ChapterType(it) }
-                }
 
-            FetchChaptersPayload(
+                val chapters =
-                clientMutationId = clientMutationId,
+                    transaction {
-                chapters = chapters,
+                        ChapterTable.select { ChapterTable.manga eq mangaId }
-            )
+                            .orderBy(ChapterTable.sourceOrder)
+                            .map { ChapterType(it) }
+                    }
+
+                FetchChaptersPayload(
+                    clientMutationId = clientMutationId,
+                    chapters = chapters,
+                )
+            }
         }
     }
 
@@ -153,12 +169,14 @@ class ChapterMutation {
         val meta: ChapterMetaType,
     )
 
-    fun setChapterMeta(input: SetChapterMetaInput): SetChapterMetaPayload {
+    fun setChapterMeta(input: SetChapterMetaInput): DataFetcherResult<SetChapterMetaPayload?> {
-        val (clientMutationId, meta) = input
+        return asDataFetcherResult {
+            val (clientMutationId, meta) = input
 
-        Chapter.modifyChapterMeta(meta.chapterId, meta.key, meta.value)
+            Chapter.modifyChapterMeta(meta.chapterId, meta.key, meta.value)
 
-        return SetChapterMetaPayload(clientMutationId, meta)
+            SetChapterMetaPayload(clientMutationId, meta)
+        }
     }
 
     data class DeleteChapterMetaInput(
@@ -173,30 +191,32 @@ class ChapterMutation {
         val chapter: ChapterType,
     )
 
-    fun deleteChapterMeta(input: DeleteChapterMetaInput): DeleteChapterMetaPayload {
+    fun deleteChapterMeta(input: DeleteChapterMetaInput): DataFetcherResult<DeleteChapterMetaPayload?> {
-        val (clientMutationId, chapterId, key) = input
+        return asDataFetcherResult {
+            val (clientMutationId, chapterId, key) = input
 
-        val (meta, chapter) =
+            val (meta, chapter) =
-            transaction {
+                transaction {
-                val meta =
+                    val meta =
-                    ChapterMetaTable.select { (ChapterMetaTable.ref eq chapterId) and (ChapterMetaTable.key eq key) }
+                        ChapterMetaTable.select { (ChapterMetaTable.ref eq chapterId) and (ChapterMetaTable.key eq key) }
-                        .firstOrNull()
+                            .firstOrNull()
 
-                ChapterMetaTable.deleteWhere { (ChapterMetaTable.ref eq chapterId) and (ChapterMetaTable.key eq key) }
+                    ChapterMetaTable.deleteWhere { (ChapterMetaTable.ref eq chapterId) and (ChapterMetaTable.key eq key) }
 
-                val chapter =
+                    val chapter =
-                    transaction {
+                        transaction {
-                        ChapterType(ChapterTable.select { ChapterTable.id eq chapterId }.first())
+                            ChapterType(ChapterTable.select { ChapterTable.id eq chapterId }.first())
-                    }
+                        }
 
-                if (meta != null) {
+                    if (meta != null) {
-                    ChapterMetaType(meta)
+                        ChapterMetaType(meta)
-                } else {
+                    } else {
-                    null
+                        null
-                } to chapter
+                    } to chapter
-            }
+                }
 
-        return DeleteChapterMetaPayload(clientMutationId, meta, chapter)
+            DeleteChapterMetaPayload(clientMutationId, meta, chapter)
+        }
     }
 
     data class FetchChapterPagesInput(
@@ -210,20 +230,22 @@ class ChapterMutation {
         val chapter: ChapterType,
     )
 
-    fun fetchChapterPages(input: FetchChapterPagesInput): CompletableFuture<FetchChapterPagesPayload> {
+    fun fetchChapterPages(input: FetchChapterPagesInput): CompletableFuture<DataFetcherResult<FetchChapterPagesPayload?>> {
         val (clientMutationId, chapterId) = input
 
         return future {
-            getChapterDownloadReadyById(chapterId)
+            asDataFetcherResult {
-        }.thenApply { chapter ->
+                val chapter = getChapterDownloadReadyById(chapterId)
-            FetchChapterPagesPayload(
+
-                clientMutationId = clientMutationId,
+                FetchChapterPagesPayload(
-                pages =
+                    clientMutationId = clientMutationId,
-                    List(chapter.pageCount) { index ->
+                    pages =
-                        "/api/v1/manga/${chapter.mangaId}/chapter/${chapter.index}/page/$index"
+                        List(chapter.pageCount) { index ->
-                    },
+                            "/api/v1/manga/${chapter.mangaId}/chapter/${chapter.index}/page/$index"
-                chapter = ChapterType(chapter),
+                        },
-            )
+                    chapter = ChapterType(chapter),
+                )
+            }
         }
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/DownloadMutation.kt

@@ -1,9 +1,11 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import graphql.execution.DataFetcherResult
 import kotlinx.coroutines.flow.first
 import kotlinx.coroutines.withTimeout
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.ChapterType
 import suwayomi.tachidesk.graphql.types.DownloadStatus
 import suwayomi.tachidesk.manga.impl.Chapter
@@ -25,19 +27,21 @@ class DownloadMutation {
         val chapters: List<ChapterType>,
     )
 
-    fun deleteDownloadedChapters(input: DeleteDownloadedChaptersInput): DeleteDownloadedChaptersPayload {
+    fun deleteDownloadedChapters(input: DeleteDownloadedChaptersInput): DataFetcherResult<DeleteDownloadedChaptersPayload?> {
         val (clientMutationId, chapters) = input
 
-        Chapter.deleteChapters(chapters)
+        return asDataFetcherResult {
+            Chapter.deleteChapters(chapters)
 
-        return DeleteDownloadedChaptersPayload(
+            DeleteDownloadedChaptersPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            chapters =
+                chapters =
-                transaction {
+                    transaction {
-                    ChapterTable.select { ChapterTable.id inList chapters }
+                        ChapterTable.select { ChapterTable.id inList chapters }
-                        .map { ChapterType(it) }
+                            .map { ChapterType(it) }
-                },
+                    },
-        )
+            )
+        }
     }
 
     data class DeleteDownloadedChapterInput(
@@ -50,18 +54,20 @@ class DownloadMutation {
         val chapters: ChapterType,
     )
 
-    fun deleteDownloadedChapter(input: DeleteDownloadedChapterInput): DeleteDownloadedChapterPayload {
+    fun deleteDownloadedChapter(input: DeleteDownloadedChapterInput): DataFetcherResult<DeleteDownloadedChapterPayload?> {
         val (clientMutationId, chapter) = input
 
-        Chapter.deleteChapters(listOf(chapter))
+        return asDataFetcherResult {
+            Chapter.deleteChapters(listOf(chapter))
 
-        return DeleteDownloadedChapterPayload(
+            DeleteDownloadedChapterPayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            chapters =
+                chapters =
-                transaction {
+                    transaction {
-                    ChapterType(ChapterTable.select { ChapterTable.id eq chapter }.first())
+                        ChapterType(ChapterTable.select { ChapterTable.id eq chapter }.first())
-                },
+                    },
-        )
+            )
+        }
     }
 
     data class EnqueueChapterDownloadsInput(
@@ -74,19 +80,23 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun enqueueChapterDownloads(input: EnqueueChapterDownloadsInput): CompletableFuture<EnqueueChapterDownloadsPayload> {
+    fun enqueueChapterDownloads(
+        input: EnqueueChapterDownloadsInput,
+    ): CompletableFuture<DataFetcherResult<EnqueueChapterDownloadsPayload?>> {
         val (clientMutationId, chapters) = input
 
-        DownloadManager.enqueue(DownloadManager.EnqueueInput(chapters))
-
         return future {
-            EnqueueChapterDownloadsPayload(
+            asDataFetcherResult {
-                clientMutationId = clientMutationId,
+                DownloadManager.enqueue(DownloadManager.EnqueueInput(chapters))
-                downloadStatus =
+
-                    withTimeout(30.seconds) {
+                EnqueueChapterDownloadsPayload(
-                        DownloadStatus(DownloadManager.status.first { it.queue.any { it.chapter.id in chapters } })
+                    clientMutationId = clientMutationId,
-                    },
+                    downloadStatus =
-            )
+                        withTimeout(30.seconds) {
+                            DownloadStatus(DownloadManager.status.first { it.queue.any { it.chapter.id in chapters } })
+                        },
+                )
+            }
         }
     }
 
@@ -100,19 +110,21 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun enqueueChapterDownload(input: EnqueueChapterDownloadInput): CompletableFuture<EnqueueChapterDownloadPayload> {
+    fun enqueueChapterDownload(input: EnqueueChapterDownloadInput): CompletableFuture<DataFetcherResult<EnqueueChapterDownloadPayload?>> {
         val (clientMutationId, chapter) = input
 
-        DownloadManager.enqueue(DownloadManager.EnqueueInput(listOf(chapter)))
-
         return future {
-            EnqueueChapterDownloadPayload(
+            asDataFetcherResult {
-                clientMutationId = clientMutationId,
+                DownloadManager.enqueue(DownloadManager.EnqueueInput(listOf(chapter)))
-                downloadStatus =
+
-                    withTimeout(30.seconds) {
+                EnqueueChapterDownloadPayload(
-                        DownloadStatus(DownloadManager.status.first { it.queue.any { it.chapter.id == chapter } })
+                    clientMutationId = clientMutationId,
-                    },
+                    downloadStatus =
-            )
+                        withTimeout(30.seconds) {
+                            DownloadStatus(DownloadManager.status.first { it.queue.any { it.chapter.id == chapter } })
+                        },
+                )
+            }
         }
     }
 
@@ -126,19 +138,23 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun dequeueChapterDownloads(input: DequeueChapterDownloadsInput): CompletableFuture<DequeueChapterDownloadsPayload> {
+    fun dequeueChapterDownloads(
+        input: DequeueChapterDownloadsInput,
+    ): CompletableFuture<DataFetcherResult<DequeueChapterDownloadsPayload?>> {
         val (clientMutationId, chapters) = input
 
-        DownloadManager.dequeue(DownloadManager.EnqueueInput(chapters))
-
         return future {
-            DequeueChapterDownloadsPayload(
+            asDataFetcherResult {
-                clientMutationId = clientMutationId,
+                DownloadManager.dequeue(DownloadManager.EnqueueInput(chapters))
-                downloadStatus =
+
-                    withTimeout(30.seconds) {
+                DequeueChapterDownloadsPayload(
-                        DownloadStatus(DownloadManager.status.first { it.queue.none { it.chapter.id in chapters } })
+                    clientMutationId = clientMutationId,
-                    },
+                    downloadStatus =
-            )
+                        withTimeout(30.seconds) {
+                            DownloadStatus(DownloadManager.status.first { it.queue.none { it.chapter.id in chapters } })
+                        },
+                )
+            }
         }
     }
 
@@ -152,19 +168,21 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun dequeueChapterDownload(input: DequeueChapterDownloadInput): CompletableFuture<DequeueChapterDownloadPayload> {
+    fun dequeueChapterDownload(input: DequeueChapterDownloadInput): CompletableFuture<DataFetcherResult<DequeueChapterDownloadPayload?>> {
         val (clientMutationId, chapter) = input
 
-        DownloadManager.dequeue(DownloadManager.EnqueueInput(listOf(chapter)))
-
         return future {
-            DequeueChapterDownloadPayload(
+            asDataFetcherResult {
-                clientMutationId = clientMutationId,
+                DownloadManager.dequeue(DownloadManager.EnqueueInput(listOf(chapter)))
-                downloadStatus =
+
-                    withTimeout(30.seconds) {
+                DequeueChapterDownloadPayload(
-                        DownloadStatus(DownloadManager.status.first { it.queue.none { it.chapter.id == chapter } })
+                    clientMutationId = clientMutationId,
-                    },
+                    downloadStatus =
-            )
+                        withTimeout(30.seconds) {
+                            DownloadStatus(DownloadManager.status.first { it.queue.none { it.chapter.id == chapter } })
+                        },
+                )
+            }
         }
     }
 
@@ -177,19 +195,21 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun startDownloader(input: StartDownloaderInput): CompletableFuture<StartDownloaderPayload> {
+    fun startDownloader(input: StartDownloaderInput): CompletableFuture<DataFetcherResult<StartDownloaderPayload?>> {
-        DownloadManager.start()
-
         return future {
-            StartDownloaderPayload(
+            asDataFetcherResult {
-                input.clientMutationId,
+                DownloadManager.start()
-                downloadStatus =
+
-                    withTimeout(30.seconds) {
+                StartDownloaderPayload(
-                        DownloadStatus(
+                    input.clientMutationId,
-                            DownloadManager.status.first { it.status == Status.Started },
+                    downloadStatus =
-                        )
+                        withTimeout(30.seconds) {
-                    },
+                            DownloadStatus(
-            )
+                                DownloadManager.status.first { it.status == Status.Started },
+                            )
+                        },
+                )
+            }
         }
     }
 
@@ -202,18 +222,21 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun stopDownloader(input: StopDownloaderInput): CompletableFuture<StopDownloaderPayload> {
+    fun stopDownloader(input: StopDownloaderInput): CompletableFuture<DataFetcherResult<StopDownloaderPayload?>> {
         return future {
-            DownloadManager.stop()
+            asDataFetcherResult {
-            StopDownloaderPayload(
+                DownloadManager.stop()
-                input.clientMutationId,
+
-                downloadStatus =
+                StopDownloaderPayload(
-                    withTimeout(30.seconds) {
+                    input.clientMutationId,
-                        DownloadStatus(
+                    downloadStatus =
-                            DownloadManager.status.first { it.status == Status.Stopped },
+                        withTimeout(30.seconds) {
-                        )
+                            DownloadStatus(
-                    },
+                                DownloadManager.status.first { it.status == Status.Stopped },
-            )
+                            )
+                        },
+                )
+            }
         }
     }
 
@@ -226,18 +249,21 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun clearDownloader(input: ClearDownloaderInput): CompletableFuture<ClearDownloaderPayload> {
+    fun clearDownloader(input: ClearDownloaderInput): CompletableFuture<DataFetcherResult<ClearDownloaderPayload?>> {
         return future {
-            DownloadManager.clear()
+            asDataFetcherResult {
-            ClearDownloaderPayload(
+                DownloadManager.clear()
-                input.clientMutationId,
+
-                downloadStatus =
+                ClearDownloaderPayload(
-                    withTimeout(30.seconds) {
+                    input.clientMutationId,
-                        DownloadStatus(
+                    downloadStatus =
-                            DownloadManager.status.first { it.status == Status.Stopped && it.queue.isEmpty() },
+                        withTimeout(30.seconds) {
-                        )
+                            DownloadStatus(
-                    },
+                                DownloadManager.status.first { it.status == Status.Stopped && it.queue.isEmpty() },
-            )
+                            )
+                        },
+                )
+            }
         }
     }
 
@@ -252,20 +278,23 @@ class DownloadMutation {
         val downloadStatus: DownloadStatus,
     )
 
-    fun reorderChapterDownload(input: ReorderChapterDownloadInput): CompletableFuture<ReorderChapterDownloadPayload> {
+    fun reorderChapterDownload(input: ReorderChapterDownloadInput): CompletableFuture<DataFetcherResult<ReorderChapterDownloadPayload?>> {
         val (clientMutationId, chapter, to) = input
-        DownloadManager.reorder(chapter, to)
 
         return future {
-            ReorderChapterDownloadPayload(
+            asDataFetcherResult {
-                clientMutationId,
+                DownloadManager.reorder(chapter, to)
-                downloadStatus =
+
-                    withTimeout(30.seconds) {
+                ReorderChapterDownloadPayload(
-                        DownloadStatus(
+                    clientMutationId,
-                            DownloadManager.status.first { it.queue.indexOfFirst { it.chapter.id == chapter } <= to },
+                    downloadStatus =
-                        )
+                        withTimeout(30.seconds) {
-                    },
+                            DownloadStatus(
-            )
+                                DownloadManager.status.first { it.queue.indexOfFirst { it.chapter.id == chapter } <= to },
+                            )
+                        },
+                )
+            }
         }
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/ExtensionMutation.kt

@@ -1,9 +1,11 @@
 package suwayomi.tachidesk.graphql.mutations
 
 import eu.kanade.tachiyomi.source.local.LocalSource
+import graphql.execution.DataFetcherResult
 import io.javalin.http.UploadedFile
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.ExtensionType
 import suwayomi.tachidesk.manga.impl.extension.Extension
 import suwayomi.tachidesk.manga.impl.extension.ExtensionsList
@@ -69,41 +71,45 @@ class ExtensionMutation {
         }
     }
 
-    fun updateExtension(input: UpdateExtensionInput): CompletableFuture<UpdateExtensionPayload> {
+    fun updateExtension(input: UpdateExtensionInput): CompletableFuture<DataFetcherResult<UpdateExtensionPayload?>> {
         val (clientMutationId, id, patch) = input
 
         return future {
-            updateExtensions(listOf(id), patch)
+            asDataFetcherResult {
-        }.thenApply {
+                updateExtensions(listOf(id), patch)
-            val extension =
-                transaction {
-                    ExtensionTable.select { ExtensionTable.pkgName eq id }.firstOrNull()
-                        ?.let { ExtensionType(it) }
-                }
 
-            UpdateExtensionPayload(
+                val extension =
-                clientMutationId = clientMutationId,
+                    transaction {
-                extension = extension,
+                        ExtensionTable.select { ExtensionTable.pkgName eq id }.firstOrNull()
-            )
+                            ?.let { ExtensionType(it) }
+                    }
+
+                UpdateExtensionPayload(
+                    clientMutationId = clientMutationId,
+                    extension = extension,
+                )
+            }
         }
     }
 
-    fun updateExtensions(input: UpdateExtensionsInput): CompletableFuture<UpdateExtensionsPayload> {
+    fun updateExtensions(input: UpdateExtensionsInput): CompletableFuture<DataFetcherResult<UpdateExtensionsPayload?>> {
         val (clientMutationId, ids, patch) = input
 
         return future {
-            updateExtensions(ids, patch)
+            asDataFetcherResult {
-        }.thenApply {
+                updateExtensions(ids, patch)
-            val extensions =
-                transaction {
-                    ExtensionTable.select { ExtensionTable.pkgName inList ids }
-                        .map { ExtensionType(it) }
-                }
 
-            UpdateExtensionsPayload(
+                val extensions =
-                clientMutationId = clientMutationId,
+                    transaction {
-                extensions = extensions,
+                        ExtensionTable.select { ExtensionTable.pkgName inList ids }
-            )
+                            .map { ExtensionType(it) }
+                    }
+
+                UpdateExtensionsPayload(
+                    clientMutationId = clientMutationId,
+                    extensions = extensions,
+                )
+            }
         }
     }
 
@@ -116,22 +122,24 @@ class ExtensionMutation {
         val extensions: List<ExtensionType>,
     )
 
-    fun fetchExtensions(input: FetchExtensionsInput): CompletableFuture<FetchExtensionsPayload> {
+    fun fetchExtensions(input: FetchExtensionsInput): CompletableFuture<DataFetcherResult<FetchExtensionsPayload?>> {
         val (clientMutationId) = input
 
         return future {
-            ExtensionsList.fetchExtensions()
+            asDataFetcherResult {
-        }.thenApply {
+                ExtensionsList.fetchExtensions()
-            val extensions =
-                transaction {
-                    ExtensionTable.select { ExtensionTable.name neq LocalSource.EXTENSION_NAME }
-                        .map { ExtensionType(it) }
-                }
 
-            FetchExtensionsPayload(
+                val extensions =
-                clientMutationId = clientMutationId,
+                    transaction {
-                extensions = extensions,
+                        ExtensionTable.select { ExtensionTable.name neq LocalSource.EXTENSION_NAME }
-            )
+                            .map { ExtensionType(it) }
+                    }
+
+                FetchExtensionsPayload(
+                    clientMutationId = clientMutationId,
+                    extensions = extensions,
+                )
+            }
         }
     }
 
@@ -145,18 +153,22 @@ class ExtensionMutation {
         val extension: ExtensionType,
     )
 
-    fun installExternalExtension(input: InstallExternalExtensionInput): CompletableFuture<InstallExternalExtensionPayload> {
+    fun installExternalExtension(
+        input: InstallExternalExtensionInput,
+    ): CompletableFuture<DataFetcherResult<InstallExternalExtensionPayload?>> {
         val (clientMutationId, extensionFile) = input
 
         return future {
-            Extension.installExternalExtension(extensionFile.content, extensionFile.filename)
+            asDataFetcherResult {
-        }.thenApply {
+                Extension.installExternalExtension(extensionFile.content, extensionFile.filename)
-            val dbExtension = transaction { ExtensionTable.select { ExtensionTable.apkName eq extensionFile.filename }.first() }
 
-            InstallExternalExtensionPayload(
+                val dbExtension = transaction { ExtensionTable.select { ExtensionTable.apkName eq extensionFile.filename }.first() }
-                clientMutationId,
+
-                extension = ExtensionType(dbExtension),
+                InstallExternalExtensionPayload(
-            )
+                    clientMutationId,
+                    extension = ExtensionType(dbExtension),
+                )
+            }
         }
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/InfoMutation.kt

@@ -1,7 +1,9 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import graphql.execution.DataFetcherResult
 import kotlinx.coroutines.flow.first
 import kotlinx.coroutines.withTimeout
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.UpdateState.DOWNLOADING
 import suwayomi.tachidesk.graphql.types.UpdateState.ERROR
 import suwayomi.tachidesk.graphql.types.UpdateState.IDLE
@@ -22,50 +24,54 @@ class InfoMutation {
         val updateStatus: WebUIUpdateStatus,
     )
 
-    fun updateWebUI(input: WebUIUpdateInput): CompletableFuture<WebUIUpdatePayload> {
+    fun updateWebUI(input: WebUIUpdateInput): CompletableFuture<DataFetcherResult<WebUIUpdatePayload?>> {
         return future {
-            withTimeout(30.seconds) {
+            asDataFetcherResult {
-                if (WebInterfaceManager.status.value.state === DOWNLOADING) {
+                withTimeout(30.seconds) {
-                    return@withTimeout WebUIUpdatePayload(input.clientMutationId, WebInterfaceManager.status.value)
+                    if (WebInterfaceManager.status.value.state === DOWNLOADING) {
-                }
+                        return@withTimeout WebUIUpdatePayload(input.clientMutationId, WebInterfaceManager.status.value)
+                    }
 
-                val flavor = WebUIFlavor.current
+                    val flavor = WebUIFlavor.current
 
-                val (version, updateAvailable) = WebInterfaceManager.isUpdateAvailable(flavor)
+                    val (version, updateAvailable) = WebInterfaceManager.isUpdateAvailable(flavor)
 
-                if (!updateAvailable) {
+                    if (!updateAvailable) {
-                    val didUpdateCheckFail = version.isEmpty()
+                        val didUpdateCheckFail = version.isEmpty()
 
-                    return@withTimeout WebUIUpdatePayload(
+                        return@withTimeout WebUIUpdatePayload(
+                            input.clientMutationId,
+                            WebInterfaceManager.getStatus(version, if (didUpdateCheckFail) ERROR else IDLE),
+                        )
+                    }
+                    try {
+                        WebInterfaceManager.startDownloadInScope(flavor, version)
+                    } catch (e: Exception) {
+                        // ignore since we use the status anyway
+                    }
+
+                    WebUIUpdatePayload(
                         input.clientMutationId,
-                        WebInterfaceManager.getStatus(version, if (didUpdateCheckFail) ERROR else IDLE),
+                        updateStatus = WebInterfaceManager.status.first { it.state == DOWNLOADING },
                     )
                 }
-                try {
-                    WebInterfaceManager.startDownloadInScope(flavor, version)
-                } catch (e: Exception) {
-                    // ignore since we use the status anyway
-                }
-
-                WebUIUpdatePayload(
-                    input.clientMutationId,
-                    updateStatus = WebInterfaceManager.status.first { it.state == DOWNLOADING },
-                )
             }
         }
     }
 
-    fun resetWebUIUpdateStatus(): CompletableFuture<WebUIUpdateStatus> {
+    fun resetWebUIUpdateStatus(): CompletableFuture<DataFetcherResult<WebUIUpdateStatus?>> {
         return future {
-            withTimeout(30.seconds) {
+            asDataFetcherResult {
-                val isUpdateFinished = WebInterfaceManager.status.value.state != DOWNLOADING
+                withTimeout(30.seconds) {
-                if (!isUpdateFinished) {
+                    val isUpdateFinished = WebInterfaceManager.status.value.state != DOWNLOADING
-                    throw Exception("Status reset is not allowed during status \"$DOWNLOADING\"")
+                    if (!isUpdateFinished) {
+                        throw Exception("Status reset is not allowed during status \"$DOWNLOADING\"")
+                    }
+
+                    WebInterfaceManager.resetStatus()
+
+                    WebInterfaceManager.status.first { it.state == IDLE }
                 }
-
-                WebInterfaceManager.resetStatus()
-
-                WebInterfaceManager.status.first { it.state == IDLE }
             }
         }
     }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/MangaMutation.kt

@@ -1,11 +1,13 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import graphql.execution.DataFetcherResult
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
 import org.jetbrains.exposed.sql.and
 import org.jetbrains.exposed.sql.deleteWhere
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
 import org.jetbrains.exposed.sql.update
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.MangaMetaType
 import suwayomi.tachidesk.graphql.types.MangaType
 import suwayomi.tachidesk.manga.impl.Library
@@ -13,6 +15,7 @@ import suwayomi.tachidesk.manga.impl.Manga
 import suwayomi.tachidesk.manga.model.table.MangaMetaTable
 import suwayomi.tachidesk.manga.model.table.MangaTable
 import suwayomi.tachidesk.server.JavalinSetup.future
+import java.time.Instant
 import java.util.concurrent.CompletableFuture
 
 /**
@@ -56,6 +59,7 @@ class MangaMutation {
                 MangaTable.update({ MangaTable.id inList ids }) { update ->
                     patch.inLibrary.also {
                         update[inLibrary] = it
+                        if (it) update[inLibraryAt] = Instant.now().epochSecond
                     }
                 }
             }
@@ -68,39 +72,43 @@ class MangaMutation {
         }
     }
 
-    fun updateManga(input: UpdateMangaInput): CompletableFuture<UpdateMangaPayload> {
+    fun updateManga(input: UpdateMangaInput): CompletableFuture<DataFetcherResult<UpdateMangaPayload?>> {
         val (clientMutationId, id, patch) = input
 
         return future {
-            updateMangas(listOf(id), patch)
+            asDataFetcherResult {
-        }.thenApply {
+                updateMangas(listOf(id), patch)
-            val manga =
-                transaction {
-                    MangaType(MangaTable.select { MangaTable.id eq id }.first())
-                }
 
-            UpdateMangaPayload(
+                val manga =
-                clientMutationId = clientMutationId,
+                    transaction {
-                manga = manga,
+                        MangaType(MangaTable.select { MangaTable.id eq id }.first())
-            )
+                    }
+
+                UpdateMangaPayload(
+                    clientMutationId = clientMutationId,
+                    manga = manga,
+                )
+            }
         }
     }
 
-    fun updateMangas(input: UpdateMangasInput): CompletableFuture<UpdateMangasPayload> {
+    fun updateMangas(input: UpdateMangasInput): CompletableFuture<DataFetcherResult<UpdateMangasPayload?>> {
         val (clientMutationId, ids, patch) = input
 
         return future {
-            updateMangas(ids, patch)
+            asDataFetcherResult {
-        }.thenApply {
+                updateMangas(ids, patch)
-            val mangas =
-                transaction {
-                    MangaTable.select { MangaTable.id inList ids }.map { MangaType(it) }
-                }
 
-            UpdateMangasPayload(
+                val mangas =
-                clientMutationId = clientMutationId,
+                    transaction {
-                mangas = mangas,
+                        MangaTable.select { MangaTable.id inList ids }.map { MangaType(it) }
-            )
+                    }
+
+                UpdateMangasPayload(
+                    clientMutationId = clientMutationId,
+                    mangas = mangas,
+                )
+            }
         }
     }
 
@@ -114,20 +122,22 @@ class MangaMutation {
         val manga: MangaType,
     )
 
-    fun fetchManga(input: FetchMangaInput): CompletableFuture<FetchMangaPayload> {
+    fun fetchManga(input: FetchMangaInput): CompletableFuture<DataFetcherResult<FetchMangaPayload?>> {
         val (clientMutationId, id) = input
 
         return future {
-            Manga.fetchManga(id)
+            asDataFetcherResult {
-        }.thenApply {
+                Manga.fetchManga(id)
-            val manga =
+
-                transaction {
+                val manga =
-                    MangaTable.select { MangaTable.id eq id }.first()
+                    transaction {
-                }
+                        MangaTable.select { MangaTable.id eq id }.first()
-            FetchMangaPayload(
+                    }
-                clientMutationId = clientMutationId,
+                FetchMangaPayload(
-                manga = MangaType(manga),
+                    clientMutationId = clientMutationId,
-            )
+                    manga = MangaType(manga),
+                )
+            }
         }
     }
 
@@ -141,12 +151,14 @@ class MangaMutation {
         val meta: MangaMetaType,
     )
 
-    fun setMangaMeta(input: SetMangaMetaInput): SetMangaMetaPayload {
+    fun setMangaMeta(input: SetMangaMetaInput): DataFetcherResult<SetMangaMetaPayload?> {
         val (clientMutationId, meta) = input
 
-        Manga.modifyMangaMeta(meta.mangaId, meta.key, meta.value)
+        return asDataFetcherResult {
+            Manga.modifyMangaMeta(meta.mangaId, meta.key, meta.value)
 
-        return SetMangaMetaPayload(clientMutationId, meta)
+            SetMangaMetaPayload(clientMutationId, meta)
+        }
     }
 
     data class DeleteMangaMetaInput(
@@ -161,29 +173,31 @@ class MangaMutation {
         val manga: MangaType,
     )
 
-    fun deleteMangaMeta(input: DeleteMangaMetaInput): DeleteMangaMetaPayload {
+    fun deleteMangaMeta(input: DeleteMangaMetaInput): DataFetcherResult<DeleteMangaMetaPayload?> {
         val (clientMutationId, mangaId, key) = input
 
-        val (meta, manga) =
+        return asDataFetcherResult {
-            transaction {
+            val (meta, manga) =
-                val meta =
+                transaction {
-                    MangaMetaTable.select { (MangaMetaTable.ref eq mangaId) and (MangaMetaTable.key eq key) }
+                    val meta =
-                        .firstOrNull()
+                        MangaMetaTable.select { (MangaMetaTable.ref eq mangaId) and (MangaMetaTable.key eq key) }
+                            .firstOrNull()
 
-                MangaMetaTable.deleteWhere { (MangaMetaTable.ref eq mangaId) and (MangaMetaTable.key eq key) }
+                    MangaMetaTable.deleteWhere { (MangaMetaTable.ref eq mangaId) and (MangaMetaTable.key eq key) }
 
-                val manga =
+                    val manga =
-                    transaction {
+                        transaction {
-                        MangaType(MangaTable.select { MangaTable.id eq mangaId }.first())
+                            MangaType(MangaTable.select { MangaTable.id eq mangaId }.first())
-                    }
+                        }
 
-                if (meta != null) {
+                    if (meta != null) {
-                    MangaMetaType(meta)
+                        MangaMetaType(meta)
-                } else {
+                    } else {
-                    null
+                        null
-                } to manga
+                    } to manga
-            }
+                }
 
-        return DeleteMangaMetaPayload(clientMutationId, meta, manga)
+            DeleteMangaMetaPayload(clientMutationId, meta, manga)
+        }
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/MetaMutation.kt

@@ -1,11 +1,13 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import graphql.execution.DataFetcherResult
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
 import org.jetbrains.exposed.sql.deleteWhere
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
 import suwayomi.tachidesk.global.impl.GlobalMeta
 import suwayomi.tachidesk.global.model.table.GlobalMetaTable
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.GlobalMetaType
 
 class MetaMutation {
@@ -19,12 +21,14 @@ class MetaMutation {
         val meta: GlobalMetaType,
     )
 
-    fun setGlobalMeta(input: SetGlobalMetaInput): SetGlobalMetaPayload {
+    fun setGlobalMeta(input: SetGlobalMetaInput): DataFetcherResult<SetGlobalMetaPayload?> {
         val (clientMutationId, meta) = input
 
-        GlobalMeta.modifyMeta(meta.key, meta.value)
+        return asDataFetcherResult {
+            GlobalMeta.modifyMeta(meta.key, meta.value)
 
-        return SetGlobalMetaPayload(clientMutationId, meta)
+            SetGlobalMetaPayload(clientMutationId, meta)
+        }
     }
 
     data class DeleteGlobalMetaInput(
@@ -37,24 +41,26 @@ class MetaMutation {
         val meta: GlobalMetaType?,
     )
 
-    fun deleteGlobalMeta(input: DeleteGlobalMetaInput): DeleteGlobalMetaPayload {
+    fun deleteGlobalMeta(input: DeleteGlobalMetaInput): DataFetcherResult<DeleteGlobalMetaPayload?> {
         val (clientMutationId, key) = input
 
-        val meta =
+        return asDataFetcherResult {
-            transaction {
+            val meta =
-                val meta =
+                transaction {
-                    GlobalMetaTable.select { GlobalMetaTable.key eq key }
+                    val meta =
-                        .firstOrNull()
+                        GlobalMetaTable.select { GlobalMetaTable.key eq key }
+                            .firstOrNull()
 
-                GlobalMetaTable.deleteWhere { GlobalMetaTable.key eq key }
+                    GlobalMetaTable.deleteWhere { GlobalMetaTable.key eq key }
 
-                if (meta != null) {
+                    if (meta != null) {
-                    GlobalMetaType(meta)
+                        GlobalMetaType(meta)
-                } else {
+                    } else {
-                    null
+                        null
+                    }
                 }
-            }
 
-        return DeleteGlobalMetaPayload(clientMutationId, meta)
+            DeleteGlobalMetaPayload(clientMutationId, meta)
+        }
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/SettingsMutation.kt

@@ -58,6 +58,7 @@ class SettingsMutation {
         updateSetting(settings.excludeEntryWithUnreadChapters, serverConfig.excludeEntryWithUnreadChapters)
         updateSetting(settings.autoDownloadAheadLimit, serverConfig.autoDownloadNewChaptersLimit) // deprecated
         updateSetting(settings.autoDownloadNewChaptersLimit, serverConfig.autoDownloadNewChaptersLimit)
+        updateSetting(settings.autoDownloadIgnoreReUploads, serverConfig.autoDownloadIgnoreReUploads)
 
         // extension
         updateSetting(settings.extensionRepos, serverConfig.extensionRepos)

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/SourceMutation.kt

@@ -5,11 +5,13 @@ import androidx.preference.EditTextPreference
 import androidx.preference.ListPreference
 import androidx.preference.MultiSelectListPreference
 import androidx.preference.SwitchPreferenceCompat
+import graphql.execution.DataFetcherResult
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
 import org.jetbrains.exposed.sql.and
 import org.jetbrains.exposed.sql.deleteWhere
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.FilterChange
 import suwayomi.tachidesk.graphql.types.MangaType
 import suwayomi.tachidesk.graphql.types.Preference
@@ -17,7 +19,7 @@ import suwayomi.tachidesk.graphql.types.SourceMetaType
 import suwayomi.tachidesk.graphql.types.SourceType
 import suwayomi.tachidesk.graphql.types.preferenceOf
 import suwayomi.tachidesk.graphql.types.updateFilterList
-import suwayomi.tachidesk.manga.impl.MangaList.insertOrGet
+import suwayomi.tachidesk.manga.impl.MangaList.insertOrUpdate
 import suwayomi.tachidesk.manga.impl.Source
 import suwayomi.tachidesk.manga.impl.util.source.GetCatalogueSource
 import suwayomi.tachidesk.manga.model.table.MangaTable
@@ -37,12 +39,14 @@ class SourceMutation {
         val meta: SourceMetaType,
     )
 
-    fun setSourceMeta(input: SetSourceMetaInput): SetSourceMetaPayload {
+    fun setSourceMeta(input: SetSourceMetaInput): DataFetcherResult<SetSourceMetaPayload?> {
         val (clientMutationId, meta) = input
 
-        Source.modifyMeta(meta.sourceId, meta.key, meta.value)
+        return asDataFetcherResult {
+            Source.modifyMeta(meta.sourceId, meta.key, meta.value)
 
-        return SetSourceMetaPayload(clientMutationId, meta)
+            SetSourceMetaPayload(clientMutationId, meta)
+        }
     }
 
     data class DeleteSourceMetaInput(
@@ -57,31 +61,33 @@ class SourceMutation {
         val source: SourceType?,
     )
 
-    fun deleteSourceMeta(input: DeleteSourceMetaInput): DeleteSourceMetaPayload {
+    fun deleteSourceMeta(input: DeleteSourceMetaInput): DataFetcherResult<DeleteSourceMetaPayload?> {
         val (clientMutationId, sourceId, key) = input
 
-        val (meta, source) =
+        return asDataFetcherResult {
-            transaction {
+            val (meta, source) =
-                val meta =
+                transaction {
-                    SourceMetaTable.select { (SourceMetaTable.ref eq sourceId) and (SourceMetaTable.key eq key) }
+                    val meta =
-                        .firstOrNull()
+                        SourceMetaTable.select { (SourceMetaTable.ref eq sourceId) and (SourceMetaTable.key eq key) }
+                            .firstOrNull()
 
-                SourceMetaTable.deleteWhere { (SourceMetaTable.ref eq sourceId) and (SourceMetaTable.key eq key) }
+                    SourceMetaTable.deleteWhere { (SourceMetaTable.ref eq sourceId) and (SourceMetaTable.key eq key) }
 
-                val source =
+                    val source =
-                    transaction {
+                        transaction {
-                        SourceTable.select { SourceTable.id eq sourceId }.firstOrNull()
+                            SourceTable.select { SourceTable.id eq sourceId }.firstOrNull()
-                            ?.let { SourceType(it) }
+                                ?.let { SourceType(it) }
-                    }
+                        }
 
-                if (meta != null) {
+                    if (meta != null) {
-                    SourceMetaType(meta)
+                        SourceMetaType(meta)
-                } else {
+                    } else {
-                    null
+                        null
-                } to source
+                    } to source
-            }
+                }
 
-        return DeleteSourceMetaPayload(clientMutationId, meta, source)
+            DeleteSourceMetaPayload(clientMutationId, meta, source)
+        }
     }
 
     enum class FetchSourceMangaType {
@@ -105,44 +111,46 @@ class SourceMutation {
         val hasNextPage: Boolean,
     )
 
-    fun fetchSourceManga(input: FetchSourceMangaInput): CompletableFuture<FetchSourceMangaPayload> {
+    fun fetchSourceManga(input: FetchSourceMangaInput): CompletableFuture<DataFetcherResult<FetchSourceMangaPayload?>> {
         val (clientMutationId, sourceId, type, page, query, filters) = input
 
         return future {
-            val source = GetCatalogueSource.getCatalogueSourceOrNull(sourceId)!!
+            asDataFetcherResult {
-            val mangasPage =
+                val source = GetCatalogueSource.getCatalogueSourceOrNull(sourceId)!!
-                when (type) {
+                val mangasPage =
-                    FetchSourceMangaType.SEARCH -> {
+                    when (type) {
-                        source.getSearchManga(
+                        FetchSourceMangaType.SEARCH -> {
-                            page = page,
+                            source.getSearchManga(
-                            query = query.orEmpty(),
+                                page = page,
-                            filters = updateFilterList(source, filters),
+                                query = query.orEmpty(),
-                        )
+                                filters = updateFilterList(source, filters),
+                            )
+                        }
+                        FetchSourceMangaType.POPULAR -> {
+                            source.getPopularManga(page)
+                        }
+                        FetchSourceMangaType.LATEST -> {
+                            if (!source.supportsLatest) throw Exception("Source does not support latest")
+                            source.getLatestUpdates(page)
+                        }
                     }
-                    FetchSourceMangaType.POPULAR -> {
+
-                        source.getPopularManga(page)
+                val mangaIds = mangasPage.insertOrUpdate(sourceId)
+
+                val mangas =
+                    transaction {
+                        MangaTable.select { MangaTable.id inList mangaIds }
+                            .map { MangaType(it) }
+                    }.sortedBy {
+                        mangaIds.indexOf(it.id)
                     }
-                    FetchSourceMangaType.LATEST -> {
-                        if (!source.supportsLatest) throw Exception("Source does not support latest")
-                        source.getLatestUpdates(page)
-                    }
-                }
 
-            val mangaIds = mangasPage.insertOrGet(sourceId)
+                FetchSourceMangaPayload(
-
+                    clientMutationId = clientMutationId,
-            val mangas =
+                    mangas = mangas,
-                transaction {
+                    hasNextPage = mangasPage.hasNextPage,
-                    MangaTable.select { MangaTable.id inList mangaIds }
+                )
-                        .map { MangaType(it) }
+            }
-                }.sortedBy {
-                    mangaIds.indexOf(it.id)
-                }
-
-            FetchSourceMangaPayload(
-                clientMutationId = clientMutationId,
-                mangas = mangas,
-                hasNextPage = mangasPage.hasNextPage,
-            )
         }
     }
 
@@ -167,27 +175,29 @@ class SourceMutation {
         val source: SourceType,
     )
 
-    fun updateSourcePreference(input: UpdateSourcePreferenceInput): UpdateSourcePreferencePayload {
+    fun updateSourcePreference(input: UpdateSourcePreferenceInput): DataFetcherResult<UpdateSourcePreferencePayload?> {
         val (clientMutationId, sourceId, change) = input
 
-        Source.setSourcePreference(sourceId, change.position, "") { preference ->
+        return asDataFetcherResult {
-            when (preference) {
+            Source.setSourcePreference(sourceId, change.position, "") { preference ->
-                is SwitchPreferenceCompat -> change.switchState
+                when (preference) {
-                is CheckBoxPreference -> change.checkBoxState
+                    is SwitchPreferenceCompat -> change.switchState
-                is EditTextPreference -> change.editTextState
+                    is CheckBoxPreference -> change.checkBoxState
-                is ListPreference -> change.listState
+                    is EditTextPreference -> change.editTextState
-                is MultiSelectListPreference -> change.multiSelectState?.toSet()
+                    is ListPreference -> change.listState
-                else -> throw RuntimeException("sealed class cannot have more subtypes!")
+                    is MultiSelectListPreference -> change.multiSelectState?.toSet()
-            } ?: throw Exception("Expected change to ${preference::class.simpleName}")
+                    else -> throw RuntimeException("sealed class cannot have more subtypes!")
-        }
+                } ?: throw Exception("Expected change to ${preference::class.simpleName}")
+            }
 
-        return UpdateSourcePreferencePayload(
+            UpdateSourcePreferencePayload(
-            clientMutationId = clientMutationId,
+                clientMutationId = clientMutationId,
-            preferences = Source.getSourcePreferencesRaw(sourceId).map { preferenceOf(it) },
+                preferences = Source.getSourcePreferencesRaw(sourceId).map { preferenceOf(it) },
-            source =
+                source =
-                transaction {
+                    transaction {
-                    SourceType(SourceTable.select { SourceTable.id eq sourceId }.first())!!
+                        SourceType(SourceTable.select { SourceTable.id eq sourceId }.first())!!
-                },
+                    },
-        )
+            )
+        }
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/TrackMutation.kt

@@ -1,8 +1,12 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import com.expediagroup.graphql.generator.annotations.GraphQLDeprecated
+import com.expediagroup.graphql.generator.annotations.GraphQLDescription
+import graphql.execution.DataFetcherResult
 import org.jetbrains.exposed.sql.and
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.TrackRecordType
 import suwayomi.tachidesk.graphql.types.TrackerType
 import suwayomi.tachidesk.manga.impl.track.Track
@@ -133,6 +137,93 @@ class TrackMutation {
         }
     }
 
+    data class FetchTrackInput(
+        val clientMutationId: String? = null,
+        val recordId: Int,
+    )
+
+    data class FetchTrackPayload(
+        val clientMutationId: String?,
+        val trackRecord: TrackRecordType,
+    )
+
+    fun fetchTrack(input: FetchTrackInput): CompletableFuture<FetchTrackPayload> {
+        val (clientMutationId, recordId) = input
+
+        return future {
+            Track.refresh(recordId)
+            val trackRecord =
+                transaction {
+                    TrackRecordTable.select {
+                        TrackRecordTable.id eq recordId
+                    }.first()
+                }
+            FetchTrackPayload(
+                clientMutationId,
+                TrackRecordType(trackRecord),
+            )
+        }
+    }
+
+    data class UnbindTrackInput(
+        val clientMutationId: String? = null,
+        val recordId: Int,
+        @GraphQLDescription("This will only work if the tracker of the track record supports deleting tracks")
+        val deleteRemoteTrack: Boolean? = null,
+    )
+
+    data class UnbindTrackPayload(
+        val clientMutationId: String?,
+        val trackRecord: TrackRecordType?,
+    )
+
+    fun unbindTrack(input: UnbindTrackInput): CompletableFuture<UnbindTrackPayload> {
+        val (clientMutationId, recordId, deleteRemoteTrack) = input
+
+        return future {
+            Track.unbind(recordId, deleteRemoteTrack)
+            val trackRecord =
+                transaction {
+                    TrackRecordTable.select {
+                        TrackRecordTable.id eq recordId
+                    }.firstOrNull()
+                }
+            UnbindTrackPayload(
+                clientMutationId,
+                trackRecord?.let { TrackRecordType(it) },
+            )
+        }
+    }
+
+    data class TrackProgressInput(
+        val clientMutationId: String? = null,
+        val mangaId: Int,
+    )
+
+    data class TrackProgressPayload(
+        val clientMutationId: String?,
+        val trackRecords: List<TrackRecordType>,
+    )
+
+    fun trackProgress(input: TrackProgressInput): CompletableFuture<DataFetcherResult<TrackProgressPayload?>> {
+        val (clientMutationId, mangaId) = input
+
+        return future {
+            asDataFetcherResult {
+                Track.trackChapter(mangaId)
+                val trackRecords =
+                    transaction {
+                        TrackRecordTable.select { TrackRecordTable.mangaId eq mangaId }
+                            .toList()
+                    }
+                TrackProgressPayload(
+                    clientMutationId,
+                    trackRecords.map { TrackRecordType(it) },
+                )
+            }
+        }
+    }
+
     data class UpdateTrackInput(
         val clientMutationId: String? = null,
         val recordId: Int,
@@ -141,6 +232,7 @@ class TrackMutation {
         val scoreString: String? = null,
         val startDate: Long? = null,
         val finishDate: Long? = null,
+        @GraphQLDeprecated("Replaced with \"unbindTrack\" mutation", replaceWith = ReplaceWith("unbindTrack"))
         val unbind: Boolean? = null,
     )
 

server/src/main/kotlin/suwayomi/tachidesk/graphql/mutations/UpdateMutation.kt

@@ -1,15 +1,22 @@
 package suwayomi.tachidesk.graphql.mutations
 
+import graphql.execution.DataFetcherResult
+import kotlinx.coroutines.flow.first
+import kotlinx.coroutines.withTimeout
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
 import org.kodein.di.DI
 import org.kodein.di.conf.global
 import org.kodein.di.instance
+import suwayomi.tachidesk.graphql.asDataFetcherResult
 import suwayomi.tachidesk.graphql.types.UpdateStatus
 import suwayomi.tachidesk.manga.impl.Category
 import suwayomi.tachidesk.manga.impl.update.IUpdater
 import suwayomi.tachidesk.manga.model.table.CategoryTable
 import suwayomi.tachidesk.manga.model.table.toDataClass
+import suwayomi.tachidesk.server.JavalinSetup.future
+import java.util.concurrent.CompletableFuture
+import kotlin.time.Duration.Companion.seconds
 
 class UpdateMutation {
     private val updater by DI.global.instance<IUpdater>()
@@ -23,14 +30,24 @@ class UpdateMutation {
         val updateStatus: UpdateStatus,
     )
 
-    fun updateLibraryManga(input: UpdateLibraryMangaInput): UpdateLibraryMangaPayload {
+    fun updateLibraryManga(input: UpdateLibraryMangaInput): CompletableFuture<DataFetcherResult<UpdateLibraryMangaPayload?>> {
         updater.addCategoriesToUpdateQueue(
             Category.getCategoryList(),
             clear = true,
             forceAll = false,
         )
 
-        return UpdateLibraryMangaPayload(input.clientMutationId, UpdateStatus(updater.status.value))
+        return future {
+            asDataFetcherResult {
+                UpdateLibraryMangaPayload(
+                    input.clientMutationId,
+                    updateStatus =
+                        withTimeout(30.seconds) {
+                            UpdateStatus(updater.status.first())
+                        },
+                )
+            }
+        }
     }
 
     data class UpdateCategoryMangaInput(
@@ -43,7 +60,7 @@ class UpdateMutation {
         val updateStatus: UpdateStatus,
     )
 
-    fun updateCategoryManga(input: UpdateCategoryMangaInput): UpdateCategoryMangaPayload {
+    fun updateCategoryManga(input: UpdateCategoryMangaInput): CompletableFuture<DataFetcherResult<UpdateCategoryMangaPayload?>> {
         val categories =
             transaction {
                 CategoryTable.select { CategoryTable.id inList input.categories }.map {
@@ -52,10 +69,17 @@ class UpdateMutation {
             }
         updater.addCategoriesToUpdateQueue(categories, clear = true, forceAll = true)
 
-        return UpdateCategoryMangaPayload(
+        return future {
-            clientMutationId = input.clientMutationId,
+            asDataFetcherResult {
-            updateStatus = UpdateStatus(updater.status.value),
+                UpdateCategoryMangaPayload(
-        )
+                    input.clientMutationId,
+                    updateStatus =
+                        withTimeout(30.seconds) {
+                            UpdateStatus(updater.status.first())
+                        },
+                )
+            }
+        }
     }
 
     data class UpdateStopInput(

server/src/main/kotlin/suwayomi/tachidesk/graphql/queries/BackupQuery.kt

@@ -16,14 +16,20 @@ class BackupQuery {
         val name: String,
     )
 
+    data class ValidateBackupTracker(
+        val name: String,
+    )
+
     data class ValidateBackupResult(
         val missingSources: List<ValidateBackupSource>,
+        val missingTrackers: List<ValidateBackupTracker>,
     )
 
     fun validateBackup(input: ValidateBackupInput): ValidateBackupResult {
         val result = ProtoBackupValidator.validate(input.backup.content)
         return ValidateBackupResult(
             result.missingSourceIds.map { ValidateBackupSource(it.first, it.second) },
+            result.missingTrackers.map { ValidateBackupTracker(it) },
         )
     }
 

server/src/main/kotlin/suwayomi/tachidesk/graphql/queries/MangaQuery.kt

@@ -28,6 +28,7 @@ import suwayomi.tachidesk.graphql.queries.filter.andFilterWithCompare
 import suwayomi.tachidesk.graphql.queries.filter.andFilterWithCompareEntity
 import suwayomi.tachidesk.graphql.queries.filter.andFilterWithCompareString
 import suwayomi.tachidesk.graphql.queries.filter.applyOps
+import suwayomi.tachidesk.graphql.queries.util.distinctOn
 import suwayomi.tachidesk.graphql.server.primitives.Cursor
 import suwayomi.tachidesk.graphql.server.primitives.OrderBy
 import suwayomi.tachidesk.graphql.server.primitives.PageInfo
@@ -217,7 +218,12 @@ class MangaQuery {
     ): MangaNodeList {
         val queryResults =
             transaction {
-                val res = MangaTable.leftJoin(CategoryMangaTable).selectAll()
+                val res =
+                    MangaTable.leftJoin(CategoryMangaTable).slice(
+                        distinctOn(MangaTable.id),
+                        *(MangaTable.columns).toTypedArray(),
+                        *(CategoryMangaTable.columns).toTypedArray(),
+                    ).selectAll()
 
                 res.applyOps(condition, filter)
 

server/src/main/kotlin/suwayomi/tachidesk/graphql/queries/UpdateQuery.kt

@@ -1,16 +1,19 @@
 package suwayomi.tachidesk.graphql.queries
 
+import kotlinx.coroutines.flow.first
 import org.kodein.di.DI
 import org.kodein.di.conf.global
 import org.kodein.di.instance
 import suwayomi.tachidesk.graphql.types.UpdateStatus
 import suwayomi.tachidesk.manga.impl.update.IUpdater
+import suwayomi.tachidesk.server.JavalinSetup.future
+import java.util.concurrent.CompletableFuture
 
 class UpdateQuery {
     private val updater by DI.global.instance<IUpdater>()
 
-    fun updateStatus(): UpdateStatus {
+    fun updateStatus(): CompletableFuture<UpdateStatus> {
-        return UpdateStatus(updater.status.value)
+        return future { UpdateStatus(updater.status.first()) }
     }
 
     data class LastUpdateTimestampPayload(val timestamp: Long)

server/src/main/kotlin/suwayomi/tachidesk/graphql/queries/util/CustomDistinctOn.kt

@@ -0,0 +1,31 @@
+package suwayomi.tachidesk.graphql.queries.util
+
+import org.jetbrains.exposed.sql.BooleanColumnType
+import org.jetbrains.exposed.sql.CustomFunction
+import org.jetbrains.exposed.sql.Expression
+import org.jetbrains.exposed.sql.QueryBuilder
+
+/**
+ * src: https://github.com/JetBrains/Exposed/issues/500#issuecomment-543574151 (2024-04-02 02:20)
+ */
+
+fun distinctOn(vararg expressions: Expression<*>): CustomFunction<Boolean?> =
+    customBooleanFunction(
+        functionName = "DISTINCT ON",
+        postfix = " TRUE",
+        params = expressions,
+    )
+
+fun customBooleanFunction(
+    functionName: String,
+    postfix: String = "",
+    vararg params: Expression<*>,
+): CustomFunction<Boolean?> =
+    object : CustomFunction<Boolean?>(functionName, BooleanColumnType(), *params) {
+        override fun toQueryBuilder(queryBuilder: QueryBuilder) {
+            super.toQueryBuilder(queryBuilder)
+            if (postfix.isNotEmpty()) {
+                queryBuilder.append(postfix)
+            }
+        }
+    }

server/src/main/kotlin/suwayomi/tachidesk/graphql/server/TachideskDataLoaderRegistryFactory.kt

@@ -8,6 +8,7 @@
 package suwayomi.tachidesk.graphql.server
 
 import com.expediagroup.graphql.dataloader.KotlinDataLoaderRegistryFactory
+import suwayomi.tachidesk.graphql.dataLoaders.BookmarkedChapterCountForMangaDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.CategoriesForMangaDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.CategoryDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.CategoryForIdsDataLoader
@@ -19,6 +20,7 @@ import suwayomi.tachidesk.graphql.dataLoaders.DisplayScoreForTrackRecordDataLoad
 import suwayomi.tachidesk.graphql.dataLoaders.DownloadedChapterCountForMangaDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.ExtensionDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.ExtensionForSourceDataLoader
+import suwayomi.tachidesk.graphql.dataLoaders.FirstUnreadChapterForMangaDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.GlobalMetaDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.LastReadChapterForMangaDataLoader
 import suwayomi.tachidesk.graphql.dataLoaders.LatestFetchedChapterForMangaDataLoader
@@ -50,10 +52,12 @@ class TachideskDataLoaderRegistryFactory {
                 ChaptersForMangaDataLoader(),
                 DownloadedChapterCountForMangaDataLoader(),
                 UnreadChapterCountForMangaDataLoader(),
+                BookmarkedChapterCountForMangaDataLoader(),
                 LastReadChapterForMangaDataLoader(),
                 LatestReadChapterForMangaDataLoader(),
                 LatestFetchedChapterForMangaDataLoader(),
                 LatestUploadedChapterForMangaDataLoader(),
+                FirstUnreadChapterForMangaDataLoader(),
                 GlobalMetaDataLoader(),
                 ChapterMetaDataLoader(),
                 MangaMetaDataLoader(),

server/src/main/kotlin/suwayomi/tachidesk/graphql/server/TachideskGraphQLServer.kt

@@ -12,6 +12,7 @@ import com.expediagroup.graphql.server.execution.GraphQLRequestHandler
 import com.expediagroup.graphql.server.execution.GraphQLServer
 import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
 import graphql.GraphQL
+import graphql.execution.AsyncExecutionStrategy
 import io.javalin.http.Context
 import io.javalin.websocket.WsCloseContext
 import io.javalin.websocket.WsMessageContext
@@ -47,6 +48,7 @@ class TachideskGraphQLServer(
         private fun getGraphQLObject(): GraphQL =
             GraphQL.newGraphQL(schema)
                 .subscriptionExecutionStrategy(FlowSubscriptionExecutionStrategy())
+                .mutationExecutionStrategy(AsyncExecutionStrategy())
                 .build()
 
         fun create(): TachideskGraphQLServer {

server/src/main/kotlin/suwayomi/tachidesk/graphql/types/MangaType.kt

@@ -11,6 +11,7 @@ import com.expediagroup.graphql.server.extensions.getValueFromDataLoader
 import eu.kanade.tachiyomi.source.model.UpdateStrategy
 import graphql.schema.DataFetchingEnvironment
 import org.jetbrains.exposed.sql.ResultRow
+import suwayomi.tachidesk.graphql.cache.CustomCacheMap
 import suwayomi.tachidesk.graphql.server.primitives.Cursor
 import suwayomi.tachidesk.graphql.server.primitives.Edge
 import suwayomi.tachidesk.graphql.server.primitives.Node
@@ -45,18 +46,33 @@ class MangaType(
     var chaptersLastFetchedAt: Long?, // todo
 ) : Node {
     companion object {
+        fun clearCacheFor(
+            mangaIds: List<Int>,
+            dataFetchingEnvironment: DataFetchingEnvironment,
+        ) {
+            mangaIds.forEach { clearCacheFor(it, dataFetchingEnvironment) }
+        }
+
         fun clearCacheFor(
             mangaId: Int,
             dataFetchingEnvironment: DataFetchingEnvironment,
         ) {
             dataFetchingEnvironment.getDataLoader<Int, MangaType>("MangaDataLoader").clear(mangaId)
-            dataFetchingEnvironment.getDataLoader<Int, MangaNodeList>("MangaForIdsDataLoader").clear(mangaId)
+
+            val mangaForIdsDataLoader =
+                dataFetchingEnvironment.getDataLoader<List<Int>, MangaNodeList>("MangaForIdsDataLoader")
+            @Suppress("UNCHECKED_CAST")
+            (mangaForIdsDataLoader.cacheMap as CustomCacheMap<List<Int>, MangaNodeList>).getKeys()
+                .filter { it.contains(mangaId) }.forEach { mangaForIdsDataLoader.clear(it) }
+
             dataFetchingEnvironment.getDataLoader<Int, Int>("DownloadedChapterCountForMangaDataLoader").clear(mangaId)
             dataFetchingEnvironment.getDataLoader<Int, Int>("UnreadChapterCountForMangaDataLoader").clear(mangaId)
+            dataFetchingEnvironment.getDataLoader<Int, Int>("BookmarkedChapterCountForMangaDataLoader").clear(mangaId)
             dataFetchingEnvironment.getDataLoader<Int, ChapterType>("LastReadChapterForMangaDataLoader").clear(mangaId)
             dataFetchingEnvironment.getDataLoader<Int, ChapterType>("LatestReadChapterForMangaDataLoader").clear(mangaId)
             dataFetchingEnvironment.getDataLoader<Int, ChapterType>("LatestFetchedChapterForMangaDataLoader").clear(mangaId)
             dataFetchingEnvironment.getDataLoader<Int, ChapterType>("LatestUploadedChapterForMangaDataLoader").clear(mangaId)
+            dataFetchingEnvironment.getDataLoader<Int, ChapterType>("FirstUnreadChapterForMangaDataLoader").clear(mangaId)
             dataFetchingEnvironment.getDataLoader<Int, ChapterNodeList>(
                 "ChaptersForMangaDataLoader",
             ).clear(mangaId)
@@ -115,6 +131,10 @@ class MangaType(
         return dataFetchingEnvironment.getValueFromDataLoader("UnreadChapterCountForMangaDataLoader", id)
     }
 
+    fun bookmarkCount(dataFetchingEnvironment: DataFetchingEnvironment): CompletableFuture<Int> {
+        return dataFetchingEnvironment.getValueFromDataLoader("BookmarkedChapterCountForMangaDataLoader", id)
+    }
+
     fun lastReadChapter(dataFetchingEnvironment: DataFetchingEnvironment): CompletableFuture<ChapterType?> {
         return dataFetchingEnvironment.getValueFromDataLoader("LastReadChapterForMangaDataLoader", id)
     }
@@ -131,6 +151,10 @@ class MangaType(
         return dataFetchingEnvironment.getValueFromDataLoader("LatestUploadedChapterForMangaDataLoader", id)
     }
 
+    fun firstUnreadChapter(dataFetchingEnvironment: DataFetchingEnvironment): CompletableFuture<ChapterType?> {
+        return dataFetchingEnvironment.getValueFromDataLoader("FirstUnreadChapterForMangaDataLoader", id)
+    }
+
     fun chapters(dataFetchingEnvironment: DataFetchingEnvironment): CompletableFuture<ChapterNodeList> {
         return dataFetchingEnvironment.getValueFromDataLoader<Int, ChapterNodeList>("ChaptersForMangaDataLoader", id)
     }

server/src/main/kotlin/suwayomi/tachidesk/graphql/types/SettingsType.kt

@@ -49,6 +49,7 @@ interface Settings : Node {
     )
     val autoDownloadAheadLimit: Int?
     val autoDownloadNewChaptersLimit: Int?
+    val autoDownloadIgnoreReUploads: Boolean?
 
     // extension
     val extensionRepos: List<String>?
@@ -118,6 +119,7 @@ data class PartialSettingsType(
     )
     override val autoDownloadAheadLimit: Int?,
     override val autoDownloadNewChaptersLimit: Int?,
+    override val autoDownloadIgnoreReUploads: Boolean?,
     // extension
     override val extensionRepos: List<String>?,
     // requests
@@ -179,6 +181,7 @@ class SettingsType(
     )
     override val autoDownloadAheadLimit: Int,
     override val autoDownloadNewChaptersLimit: Int,
+    override val autoDownloadIgnoreReUploads: Boolean?,
     // extension
     override val extensionRepos: List<String>,
     // requests
@@ -235,6 +238,7 @@ class SettingsType(
         config.excludeEntryWithUnreadChapters.value,
         config.autoDownloadNewChaptersLimit.value, // deprecated
         config.autoDownloadNewChaptersLimit.value,
+        config.autoDownloadIgnoreReUploads.value,
         // extension
         config.extensionRepos.value,
         // requests

server/src/main/kotlin/suwayomi/tachidesk/graphql/types/TrackType.kt

@@ -20,6 +20,7 @@ class TrackerType(
     val icon: String,
     val isLoggedIn: Boolean,
     val authUrl: String?,
+    val supportsTrackDeletion: Boolean?,
 ) : Node {
     constructor(tracker: Tracker) : this(
         tracker.isLoggedIn,
@@ -36,6 +37,7 @@ class TrackerType(
         } else {
             tracker.authUrl()
         },
+        tracker.supportsTrackDeletion,
     )
 
     fun statuses(dataFetchingEnvironment: DataFetchingEnvironment): CompletableFuture<List<TrackStatusType>> {

server/src/main/kotlin/suwayomi/tachidesk/manga/controller/UpdateController.kt

@@ -117,7 +117,7 @@ object UpdateController {
             },
             behaviorOf = { ctx ->
                 val updater by DI.global.instance<IUpdater>()
-                ctx.json(updater.status.value)
+                ctx.json(updater.statusDeprecated.value)
             },
             withResults = {
                 json<UpdateStatus>(HttpCode.OK)

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/Chapter.kt

@@ -20,7 +20,6 @@ import kotlinx.serialization.Serializable
 import mu.KotlinLogging
 import org.jetbrains.exposed.dao.id.EntityID
 import org.jetbrains.exposed.sql.Op
-import org.jetbrains.exposed.sql.ResultRow
 import org.jetbrains.exposed.sql.SortOrder
 import org.jetbrains.exposed.sql.SqlExpressionBuilder.inList
 import org.jetbrains.exposed.sql.and
@@ -37,7 +36,6 @@ import suwayomi.tachidesk.manga.impl.download.DownloadManager.EnqueueInput
 import suwayomi.tachidesk.manga.impl.track.Track
 import suwayomi.tachidesk.manga.impl.util.source.GetCatalogueSource.getCatalogueSourceOrStub
 import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
-import suwayomi.tachidesk.manga.model.dataclass.IncludeOrExclude
 import suwayomi.tachidesk.manga.model.dataclass.MangaChapterDataClass
 import suwayomi.tachidesk.manga.model.dataclass.PaginatedList
 import suwayomi.tachidesk.manga.model.dataclass.paginatedFrom
@@ -52,6 +50,15 @@ import java.util.TreeSet
 import java.util.concurrent.TimeUnit
 import kotlin.math.max
 
+private fun List<ChapterDataClass>.removeDuplicates(currentChapter: ChapterDataClass): List<ChapterDataClass> {
+    return groupBy { it.chapterNumber }
+        .map { (_, chapters) ->
+            chapters.find { it.id == currentChapter.id }
+                ?: chapters.find { it.scanlator == currentChapter.scanlator }
+                ?: chapters.first()
+        }
+}
+
 object Chapter {
     private val logger = KotlinLogging.logger { }
 
@@ -136,6 +143,7 @@ object Chapter {
                         url = manga.url
                     }
 
+                val currentLatestChapterNumber = Manga.getLatestChapter(mangaId)?.chapterNumber ?: 0f
                 val numberOfCurrentChapters = getCountOfMangaChapters(mangaId)
                 val chapterList = source.getChapterList(sManga)
 
@@ -164,7 +172,10 @@ object Chapter {
                             .toList()
                     }
 
-                val chaptersToInsert = mutableListOf<ChapterDataClass>()
+                // new chapters after they have been added to the database for auto downloads
+                val insertedChapters = mutableListOf<ChapterDataClass>()
+
+                val chaptersToInsert = mutableListOf<ChapterDataClass>() // do not yet have an ID from the database
                 val chaptersToUpdate = mutableListOf<ChapterDataClass>()
 
                 chapterList.reversed().forEachIndexed { index, fetchedChapter ->
@@ -260,7 +271,7 @@ object Chapter {
                                     this[ChapterTable.fetchedAt] = it
                                 }
                             }
-                        }
+                        }.forEach { insertedChapters.add(ChapterTable.toDataClass(it)) }
                     }
 
                     if (chaptersToUpdate.isNotEmpty()) {
@@ -283,14 +294,8 @@ object Chapter {
                     }
                 }
 
-                val newChapters =
-                    transaction {
-                        ChapterTable.select { ChapterTable.manga eq mangaId }
-                            .orderBy(ChapterTable.sourceOrder to SortOrder.DESC).toList()
-                    }
-
                 if (manga.inLibrary) {
-                    downloadNewChapters(mangaId, numberOfCurrentChapters, newChapters)
+                    downloadNewChapters(mangaId, currentLatestChapterNumber, numberOfCurrentChapters, insertedChapters)
                 }
 
                 chapterList
@@ -301,16 +306,19 @@ object Chapter {
 
     private fun downloadNewChapters(
         mangaId: Int,
+        prevLatestChapterNumber: Float,
         prevNumberOfChapters: Int,
-        updatedChapterList: List<ResultRow>,
+        newChapters: List<ChapterDataClass>,
     ) {
         val log =
             KotlinLogging.logger(
                 "${logger.name}::downloadNewChapters(" +
                     "mangaId= $mangaId, " +
+                    "prevLatestChapterNumber= $prevLatestChapterNumber, " +
                     "prevNumberOfChapters= $prevNumberOfChapters, " +
-                    "updatedChapterList= ${updatedChapterList.size}, " +
+                    "newChapters= ${newChapters.size}, " +
-                    "autoDownloadNewChaptersLimit= ${serverConfig.autoDownloadNewChaptersLimit.value}" +
+                    "autoDownloadNewChaptersLimit= ${serverConfig.autoDownloadNewChaptersLimit.value}, " +
+                    "autoDownloadIgnoreReUploads= ${serverConfig.autoDownloadIgnoreReUploads.value}" +
                     ")",
             )
 
@@ -319,68 +327,22 @@ object Chapter {
             return
         }
 
-        // Only download if there are new chapters, or if this is the first fetch
+        if (newChapters.isEmpty()) {
-        val newNumberOfChapters = updatedChapterList.size
-        val numberOfNewChapters = newNumberOfChapters - prevNumberOfChapters
-
-        val areNewChaptersAvailable = numberOfNewChapters > 0
-        val wasInitialFetch = prevNumberOfChapters == 0
-
-        if (!areNewChaptersAvailable) {
             log.debug { "no new chapters available" }
             return
         }
 
+        val wasInitialFetch = prevNumberOfChapters == 0
         if (wasInitialFetch) {
             log.debug { "skipping download on initial fetch" }
             return
         }
 
-        // Verify the manga is configured to be downloaded based on it's categories.
+        if (!Manga.isInIncludedDownloadCategory(log, mangaId)) {
-        var mangaCategories = CategoryManga.getMangaCategories(mangaId).toSet()
+            return
-        // if the manga has no categories, then it's implicitly in the default category
-        if (mangaCategories.isEmpty()) {
-            val defaultCategory = Category.getCategoryById(Category.DEFAULT_CATEGORY_ID)
-            if (defaultCategory != null) {
-                mangaCategories = setOf(defaultCategory)
-            } else {
-                log.warn { "missing default category" }
-            }
         }
 
-        if (mangaCategories.isNotEmpty()) {
+        val unreadChapters = Manga.getUnreadChapters(mangaId).subtract(newChapters.toSet())
-            val downloadCategoriesMap = Category.getCategoryList().groupBy { it.includeInDownload }
-            val unsetCategories = downloadCategoriesMap[IncludeOrExclude.UNSET].orEmpty()
-            // We only download if it's in the include list, and not in the exclude list.
-            // Use the unset categories as the included categories if the included categories is
-            // empty
-            val includedCategories = downloadCategoriesMap[IncludeOrExclude.INCLUDE].orEmpty().ifEmpty { unsetCategories }
-            val excludedCategories = downloadCategoriesMap[IncludeOrExclude.EXCLUDE].orEmpty()
-            // Only download manga that aren't in any excluded categories
-            val mangaExcludeCategories = mangaCategories.intersect(excludedCategories.toSet())
-            if (mangaExcludeCategories.isNotEmpty()) {
-                log.debug { "download excluded by categories: '${mangaExcludeCategories.joinToString("', '") { it.name }}'" }
-                return
-            }
-            val mangaDownloadCategories = mangaCategories.intersect(includedCategories.toSet())
-            if (mangaDownloadCategories.isNotEmpty()) {
-                log.debug { "download inluded by categories: '${mangaDownloadCategories.joinToString("', '") { it.name }}'" }
-            } else {
-                log.debug { "skipping download due to download categories configuration" }
-                return
-            }
-        } else {
-            log.debug { "no categories configured, skipping check for category download include/excludes" }
-        }
-
-        val newChapters = updatedChapterList.subList(0, numberOfNewChapters)
-
-        // make sure to only consider the latest chapters. e.g. old unread chapters should be ignored
-        val latestReadChapterIndex =
-            updatedChapterList.indexOfFirst { it[ChapterTable.isRead] }.takeIf { it > -1 } ?: (updatedChapterList.size)
-        val unreadChapters =
-            updatedChapterList.subList(numberOfNewChapters, latestReadChapterIndex)
-                .filter { !it[ChapterTable.isRead] }
 
         val skipDueToUnreadChapters = serverConfig.excludeEntryWithUnreadChapters.value && unreadChapters.isNotEmpty()
         if (skipDueToUnreadChapters) {
@@ -388,17 +350,7 @@ object Chapter {
             return
         }
 
-        val firstChapterToDownloadIndex =
+        val chapterIdsToDownload = getNewChapterIdsToDownload(newChapters, prevLatestChapterNumber)
-            if (serverConfig.autoDownloadNewChaptersLimit.value > 0) {
-                (numberOfNewChapters - serverConfig.autoDownloadNewChaptersLimit.value).coerceAtLeast(0)
-            } else {
-                0
-            }
-
-        val chapterIdsToDownload =
-            newChapters.subList(firstChapterToDownloadIndex, numberOfNewChapters)
-                .filter { !it[ChapterTable.isRead] && !it[ChapterTable.isDownloaded] }
-                .map { it[ChapterTable.id].value }
 
         if (chapterIdsToDownload.isEmpty()) {
             log.debug { "no chapters available for download" }
@@ -410,6 +362,37 @@ object Chapter {
         DownloadManager.enqueue(EnqueueInput(chapterIdsToDownload))
     }
 
+    private fun getNewChapterIdsToDownload(
+        newChapters: List<ChapterDataClass>,
+        prevLatestChapterNumber: Float,
+    ): List<Int> {
+        val reUploadedChapters = newChapters.filter { it.chapterNumber < prevLatestChapterNumber }
+        val actualNewChapters = newChapters.subtract(reUploadedChapters.toSet()).toList()
+        val chaptersToConsiderForDownloadLimit =
+            if (serverConfig.autoDownloadIgnoreReUploads.value) {
+                if (actualNewChapters.isNotEmpty()) actualNewChapters.removeDuplicates(actualNewChapters[0]) else emptyList()
+            } else {
+                newChapters.removeDuplicates(newChapters[0])
+            }.sortedBy { it.index }
+
+        val latestChapterToDownloadIndex =
+            if (serverConfig.autoDownloadNewChaptersLimit.value == 0) {
+                chaptersToConsiderForDownloadLimit.size
+            } else {
+                serverConfig.autoDownloadNewChaptersLimit.value.coerceAtMost(chaptersToConsiderForDownloadLimit.size)
+            }
+        val limitedChaptersToDownload = chaptersToConsiderForDownloadLimit.subList(0, latestChapterToDownloadIndex)
+        val limitedChaptersToDownloadWithDuplicates =
+            (
+                limitedChaptersToDownload +
+                    newChapters.filter { newChapter ->
+                        limitedChaptersToDownload.find { it.chapterNumber == newChapter.chapterNumber } != null
+                    }
+            ).toSet()
+
+        return limitedChaptersToDownloadWithDuplicates.map { it.id }
+    }
+
     fun modifyChapter(
         mangaId: Int,
         chapterIndex: Int,

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/Manga.kt

@@ -8,13 +8,18 @@ package suwayomi.tachidesk.manga.impl
  * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
 
 import eu.kanade.tachiyomi.network.GET
+import eu.kanade.tachiyomi.network.HttpException
 import eu.kanade.tachiyomi.network.NetworkHelper
+import eu.kanade.tachiyomi.network.awaitSuccess
 import eu.kanade.tachiyomi.source.local.LocalSource
 import eu.kanade.tachiyomi.source.model.SManga
 import eu.kanade.tachiyomi.source.model.UpdateStrategy
 import eu.kanade.tachiyomi.source.online.HttpSource
+import io.javalin.http.HttpCode
+import mu.KLogger
 import mu.KotlinLogging
 import okhttp3.CacheControl
+import okhttp3.Response
 import org.jetbrains.exposed.sql.ResultRow
 import org.jetbrains.exposed.sql.SortOrder
 import org.jetbrains.exposed.sql.and
@@ -37,6 +42,8 @@ import suwayomi.tachidesk.manga.impl.util.storage.ImageResponse.clearCachedImage
 import suwayomi.tachidesk.manga.impl.util.storage.ImageResponse.getImageResponse
 import suwayomi.tachidesk.manga.impl.util.storage.ImageUtil
 import suwayomi.tachidesk.manga.impl.util.updateMangaDownloadDir
+import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
+import suwayomi.tachidesk.manga.model.dataclass.IncludeOrExclude
 import suwayomi.tachidesk.manga.model.dataclass.MangaDataClass
 import suwayomi.tachidesk.manga.model.dataclass.toGenreList
 import suwayomi.tachidesk.manga.model.table.ChapterTable
@@ -251,9 +258,55 @@ object Manga {
         }
     }
 
+    private suspend fun fetchThumbnailUrl(mangaId: Int): String? {
+        getManga(mangaId, true)
+        return transaction {
+            MangaTable.select { MangaTable.id eq mangaId }.first()
+        }[MangaTable.thumbnail_url]
+    }
+
     private val applicationDirs by DI.global.instance<ApplicationDirs>()
     private val network: NetworkHelper by injectLazy()
 
+    private suspend fun fetchHttpSourceMangaThumbnail(
+        source: HttpSource,
+        mangaEntry: ResultRow,
+        refreshUrl: Boolean = false,
+    ): Response {
+        val mangaId = mangaEntry[MangaTable.id].value
+
+        val requiresInitialization = mangaEntry[MangaTable.thumbnail_url] == null && !mangaEntry[MangaTable.initialized]
+        val refreshThumbnailUrl = refreshUrl || requiresInitialization
+
+        val thumbnailUrl =
+            if (refreshThumbnailUrl) {
+                fetchThumbnailUrl(mangaId)
+            } else {
+                mangaEntry[MangaTable.thumbnail_url]
+            } ?: throw NullPointerException("No thumbnail found")
+
+        return try {
+            source.client.newCall(
+                GET(thumbnailUrl, source.headers, cache = CacheControl.FORCE_NETWORK),
+            ).awaitSuccess()
+        } catch (e: HttpException) {
+            val tryToRefreshUrl =
+                !refreshUrl &&
+                    listOf(
+                        HttpCode.GONE.status,
+                        HttpCode.MOVED_PERMANENTLY.status,
+                        HttpCode.NOT_FOUND.status,
+                        523, // (Cloudflare) Origin Is Unreachable
+                        522, // (Cloudflare) Connection timed out
+                    ).contains(e.code)
+            if (!tryToRefreshUrl) {
+                throw e
+            }
+
+            fetchHttpSourceMangaThumbnail(source, mangaEntry, refreshUrl = true)
+        }
+    }
+
     suspend fun fetchMangaThumbnail(mangaId: Int): Pair<InputStream, String> {
         val cacheSaveDir = applicationDirs.tempThumbnailCacheRoot
         val fileName = mangaId.toString()
@@ -264,22 +317,7 @@ object Manga {
         return when (val source = getCatalogueSourceOrStub(sourceId)) {
             is HttpSource ->
                 getImageResponse(cacheSaveDir, fileName) {
-                    val thumbnailUrl =
+                    fetchHttpSourceMangaThumbnail(source, mangaEntry)
-                        mangaEntry[MangaTable.thumbnail_url]
-                            ?: if (!mangaEntry[MangaTable.initialized]) {
-                                // initialize then try again
-                                getManga(mangaId)
-                                transaction {
-                                    MangaTable.select { MangaTable.id eq mangaId }.first()
-                                }[MangaTable.thumbnail_url]!!
-                            } else {
-                                // source provides no thumbnail url for this manga
-                                throw NullPointerException("No thumbnail found")
-                            }
-
-                    source.client.newCall(
-                        GET(thumbnailUrl, source.headers, cache = CacheControl.FORCE_NETWORK),
-                    ).await()
                 }
 
             is LocalSource -> {
@@ -315,7 +353,7 @@ object Manga {
     suspend fun getMangaThumbnail(mangaId: Int): Pair<InputStream, String> {
         val mangaEntry = transaction { MangaTable.select { MangaTable.id eq mangaId }.first() }
 
-        if (mangaEntry[MangaTable.inLibrary]) {
+        if (mangaEntry[MangaTable.inLibrary] && mangaEntry[MangaTable.sourceReference] != LocalSource.ID) {
             return try {
                 ThumbnailDownloadHelper.getImage(mangaId)
             } catch (_: MissingThumbnailException) {
@@ -333,4 +371,56 @@ object Manga {
         clearCachedImage(applicationDirs.tempThumbnailCacheRoot, fileName)
         clearCachedImage(applicationDirs.thumbnailDownloadsRoot, fileName)
     }
+
+    fun getLatestChapter(mangaId: Int): ChapterDataClass? {
+        return transaction {
+            ChapterTable.select { ChapterTable.manga eq mangaId }.maxByOrNull { it[ChapterTable.sourceOrder] }
+        }?.let { ChapterTable.toDataClass(it) }
+    }
+
+    fun getUnreadChapters(mangaId: Int): List<ChapterDataClass> {
+        return transaction {
+            ChapterTable.select { (ChapterTable.manga eq mangaId) and (ChapterTable.isRead eq false) }
+                .orderBy(ChapterTable.sourceOrder to SortOrder.DESC)
+                .map { ChapterTable.toDataClass(it) }
+        }
+    }
+
+    fun isInIncludedDownloadCategory(
+        logContext: KLogger = logger,
+        mangaId: Int,
+    ): Boolean {
+        val log = KotlinLogging.logger("${logContext.name}::isInExcludedDownloadCategory($mangaId)")
+
+        // Verify the manga is configured to be downloaded based on it's categories.
+        var mangaCategories = CategoryManga.getMangaCategories(mangaId).toSet()
+        // if the manga has no categories, then it's implicitly in the default category
+        if (mangaCategories.isEmpty()) {
+            val defaultCategory = Category.getCategoryById(Category.DEFAULT_CATEGORY_ID)!!
+            mangaCategories = setOf(defaultCategory)
+        }
+
+        val downloadCategoriesMap = Category.getCategoryList().groupBy { it.includeInDownload }
+        val unsetCategories = downloadCategoriesMap[IncludeOrExclude.UNSET].orEmpty()
+        // We only download if it's in the include list, and not in the exclude list.
+        // Use the unset categories as the included categories if the included categories is
+        // empty
+        val includedCategories = downloadCategoriesMap[IncludeOrExclude.INCLUDE].orEmpty().ifEmpty { unsetCategories }
+        val excludedCategories = downloadCategoriesMap[IncludeOrExclude.EXCLUDE].orEmpty()
+        // Only download manga that aren't in any excluded categories
+        val mangaExcludeCategories = mangaCategories.intersect(excludedCategories.toSet())
+        if (mangaExcludeCategories.isNotEmpty()) {
+            log.debug { "download excluded by categories: '${mangaExcludeCategories.joinToString("', '") { it.name }}'" }
+            return false
+        }
+        val mangaDownloadCategories = mangaCategories.intersect(includedCategories.toSet())
+        if (mangaDownloadCategories.isNotEmpty()) {
+            log.debug { "download inluded by categories: '${mangaDownloadCategories.joinToString("', '") { it.name }}'" }
+        } else {
+            log.debug { "skipping download due to download categories configuration" }
+            return false
+        }
+
+        return true
+    }
 }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/MangaList.kt

@@ -8,14 +8,17 @@ package suwayomi.tachidesk.manga.impl
  * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
 
 import eu.kanade.tachiyomi.source.model.MangasPage
+import org.jetbrains.exposed.dao.id.EntityID
 import org.jetbrains.exposed.sql.and
 import org.jetbrains.exposed.sql.batchInsert
 import org.jetbrains.exposed.sql.select
+import org.jetbrains.exposed.sql.statements.BatchUpdateStatement
 import org.jetbrains.exposed.sql.transactions.transaction
 import suwayomi.tachidesk.manga.impl.util.source.GetCatalogueSource.getCatalogueSourceOrStub
 import suwayomi.tachidesk.manga.model.dataclass.PagedMangaListDataClass
 import suwayomi.tachidesk.manga.model.table.MangaTable
 import suwayomi.tachidesk.manga.model.table.toDataClass
+import java.time.Instant
 
 object MangaList {
     fun proxyThumbnailUrl(mangaId: Int): String {
@@ -44,12 +47,12 @@ object MangaList {
         return mangasPage.processEntries(sourceId)
     }
 
-    fun MangasPage.insertOrGet(sourceId: Long): List<Int> {
+    fun MangasPage.insertOrUpdate(sourceId: Long): List<Int> {
         return transaction {
             val existingMangaUrlsToId =
-                MangaTable.slice(MangaTable.url, MangaTable.id).select {
+                MangaTable.select {
                     (MangaTable.sourceReference eq sourceId) and (MangaTable.url inList mangas.map { it.url })
-                }.associate { Pair(it[MangaTable.url], it[MangaTable.id].value) }
+                }.associateBy { it[MangaTable.url] }
             val existingMangaUrls = existingMangaUrlsToId.map { it.key }
 
             val mangasToInsert = mangas.filter { !existingMangaUrls.contains(it.url) }
@@ -73,7 +76,40 @@ object MangaList {
             // delete thumbnail in case cached data still exists
             insertedMangaUrlsToId.forEach { (_, id) -> Manga.clearThumbnail(id) }
 
-            val mangaUrlsToId = existingMangaUrlsToId + insertedMangaUrlsToId
+            val mangaToUpdate =
+                mangas.mapNotNull { sManga ->
+                    existingMangaUrlsToId[sManga.url]?.let { sManga to it }
+                }.filterNot { (_, resultRow) ->
+                    resultRow[MangaTable.inLibrary]
+                }
+
+            if (mangaToUpdate.isNotEmpty()) {
+                BatchUpdateStatement(MangaTable).apply {
+                    mangaToUpdate.forEach { (sManga, manga) ->
+                        addBatch(EntityID(manga[MangaTable.id].value, MangaTable))
+                        this[MangaTable.title] = sManga.title
+                        this[MangaTable.artist] = sManga.artist ?: manga[MangaTable.artist]
+                        this[MangaTable.author] = sManga.author ?: manga[MangaTable.author]
+                        this[MangaTable.description] = sManga.description ?: manga[MangaTable.description]
+                        this[MangaTable.genre] = sManga.genre ?: manga[MangaTable.genre]
+                        this[MangaTable.status] = sManga.status
+                        this[MangaTable.thumbnail_url] = sManga.thumbnail_url ?: manga[MangaTable.thumbnail_url]
+                        this[MangaTable.updateStrategy] = sManga.update_strategy.name
+                        if (!sManga.thumbnail_url.isNullOrEmpty() && manga[MangaTable.thumbnail_url] != sManga.thumbnail_url) {
+                            this[MangaTable.thumbnailUrlLastFetched] = Instant.now().epochSecond
+                            Manga.clearThumbnail(manga[MangaTable.id].value)
+                        } else {
+                            this[MangaTable.thumbnailUrlLastFetched] =
+                                manga[MangaTable.thumbnailUrlLastFetched]
+                        }
+                    }
+                    execute(this@transaction)
+                }
+            }
+
+            val mangaUrlsToId =
+                existingMangaUrlsToId
+                    .mapValues { it.value[MangaTable.id].value } + insertedMangaUrlsToId
 
             mangas.map { manga ->
                 mangaUrlsToId[manga.url]
@@ -86,7 +122,7 @@ object MangaList {
         val mangasPage = this
         val mangaList =
             transaction {
-                val mangaIds = insertOrGet(sourceId)
+                val mangaIds = insertOrUpdate(sourceId)
                 return@transaction MangaTable.select { MangaTable.id inList mangaIds }.map { MangaTable.toDataClass(it) }
             }
         return PagedMangaListDataClass(

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/backup/models/Track.kt

@@ -11,7 +11,7 @@ interface Track : Serializable {
 
     var sync_id: Int
 
-    var media_id: Int
+    var media_id: Long
 
     var library_id: Long?
 

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/backup/models/TrackImpl.kt

@@ -9,7 +9,7 @@ class TrackImpl : Track {
 
     override var sync_id: Int = 0
 
-    override var media_id: Int = 0
+    override var media_id: Long = 0L
 
     override var library_id: Long? = null
 
@@ -43,7 +43,7 @@ class TrackImpl : Track {
     override fun hashCode(): Int {
         var result = (manga_id xor manga_id.ushr(32)).toInt()
         result = 31 * result + sync_id
-        result = 31 * result + media_id
+        result = (31 * result + media_id).toInt()
         return result
     }
 }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/backup/proto/ProtoBackupExport.kt

@@ -31,6 +31,8 @@ import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupChapter
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupManga
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupSerializer
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupSource
+import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupTracking
+import suwayomi.tachidesk.manga.impl.track.Track
 import suwayomi.tachidesk.manga.model.table.CategoryTable
 import suwayomi.tachidesk.manga.model.table.ChapterTable
 import suwayomi.tachidesk.manga.model.table.MangaStatus
@@ -230,9 +232,32 @@ object ProtoBackupExport : ProtoBackupBase() {
                 backupManga.categories = CategoryManga.getMangaCategories(mangaId).map { it.order }
             }
 
-//            if(flags.includeTracking) {
+            if (flags.includeTracking) {
-//                backupManga.tracking = TODO()
+                val tracks =
-//            }
+                    Track.getTrackRecordsByMangaId(mangaRow[MangaTable.id].value).mapNotNull {
+                        if (it.record == null) {
+                            null
+                        } else {
+                            BackupTracking(
+                                syncId = it.record.trackerId,
+                                // forced not null so its compatible with 1.x backup system
+                                libraryId = it.record.libraryId ?: 0,
+                                mediaId = it.record.remoteId,
+                                title = it.record.title,
+                                lastChapterRead = it.record.lastChapterRead.toFloat(),
+                                totalChapters = it.record.totalChapters,
+                                score = it.record.score.toFloat(),
+                                status = it.record.status,
+                                startedReadingDate = it.record.startDate,
+                                finishedReadingDate = it.record.finishDate,
+                                trackingUrl = it.record.remoteUrl,
+                            )
+                        }
+                    }
+                if (tracks.isNotEmpty()) {
+                    backupManga.tracking = tracks
+                }
+            }
 
 //            if (flags.includeHistory) {
 //                backupManga.history = TODO()

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/backup/proto/ProtoBackupImport.kt

@@ -34,22 +34,27 @@ import suwayomi.tachidesk.manga.impl.CategoryManga
 import suwayomi.tachidesk.manga.impl.Manga.clearThumbnail
 import suwayomi.tachidesk.manga.impl.backup.models.Chapter
 import suwayomi.tachidesk.manga.impl.backup.models.Manga
-import suwayomi.tachidesk.manga.impl.backup.models.Track
 import suwayomi.tachidesk.manga.impl.backup.proto.ProtoBackupValidator.ValidationResult
 import suwayomi.tachidesk.manga.impl.backup.proto.ProtoBackupValidator.validate
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupCategory
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupHistory
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupManga
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupSerializer
+import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupTracking
+import suwayomi.tachidesk.manga.impl.track.tracker.TrackerManager
+import suwayomi.tachidesk.manga.impl.track.tracker.model.toTrack
+import suwayomi.tachidesk.manga.impl.track.tracker.model.toTrackRecordDataClass
+import suwayomi.tachidesk.manga.model.dataclass.TrackRecordDataClass
 import suwayomi.tachidesk.manga.model.table.CategoryTable
 import suwayomi.tachidesk.manga.model.table.ChapterTable
 import suwayomi.tachidesk.manga.model.table.MangaTable
 import java.io.InputStream
-import java.lang.Integer.max
 import java.util.Date
 import java.util.Timer
 import java.util.TimerTask
 import java.util.concurrent.TimeUnit
+import kotlin.math.max
+import suwayomi.tachidesk.manga.impl.track.Track as Tracker
 
 object ProtoBackupImport : ProtoBackupBase() {
     private val scope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
@@ -239,10 +244,9 @@ object ProtoBackupImport : ProtoBackupBase() {
         val chapters = backupManga.getChaptersImpl()
         val categories = backupManga.categories
         val history = backupManga.brokenHistory.map { BackupHistory(it.url, it.lastRead) } + backupManga.history
-        val tracks = backupManga.getTrackingImpl()
 
         try {
-            restoreMangaData(manga, chapters, categories, history, tracks, backupCategories, categoryMapping)
+            restoreMangaData(manga, chapters, categories, history, backupManga.tracking, backupCategories, categoryMapping)
         } catch (e: Exception) {
             val sourceName = sourceMapping[manga.source] ?: manga.source.toString()
             errors.add(Date() to "${manga.title} [$sourceName]: ${e.message}")
@@ -255,7 +259,7 @@ object ProtoBackupImport : ProtoBackupBase() {
         chapters: List<Chapter>,
         categories: List<Int>,
         history: List<BackupHistory>,
-        tracks: List<Track>,
+        tracks: List<BackupTracking>,
         backupCategories: List<BackupCategory>,
         categoryMapping: Map<Int, Int>,
     ) {
@@ -265,127 +269,165 @@ object ProtoBackupImport : ProtoBackupBase() {
                     .firstOrNull()
             }
 
-        if (dbManga == null) { // Manga not in database
+        val mangaId =
-            transaction {
+            if (dbManga == null) { // Manga not in database
-                // insert manga to database
+                transaction {
-                val mangaId =
+                    // insert manga to database
-                    MangaTable.insertAndGetId {
+                    val mangaId =
-                        it[url] = manga.url
+                        MangaTable.insertAndGetId {
-                        it[title] = manga.title
+                            it[url] = manga.url
+                            it[title] = manga.title
 
-                        it[artist] = manga.artist
+                            it[artist] = manga.artist
-                        it[author] = manga.author
+                            it[author] = manga.author
-                        it[description] = manga.description
+                            it[description] = manga.description
-                        it[genre] = manga.genre
+                            it[genre] = manga.genre
+                            it[status] = manga.status
+                            it[thumbnail_url] = manga.thumbnail_url
+                            it[updateStrategy] = manga.update_strategy.name
+
+                            it[sourceReference] = manga.source
+
+                            it[initialized] = manga.description != null
+
+                            it[inLibrary] = manga.favorite
+
+                            it[inLibraryAt] = TimeUnit.MILLISECONDS.toSeconds(manga.date_added)
+                        }.value
+
+                    // delete thumbnail in case cached data still exists
+                    clearThumbnail(mangaId)
+
+                    // insert chapter data
+                    val chaptersLength = chapters.size
+                    ChapterTable.batchInsert(chapters) { chapter ->
+                        this[ChapterTable.url] = chapter.url
+                        this[ChapterTable.name] = chapter.name
+                        if (chapter.date_upload == 0L) {
+                            this[ChapterTable.date_upload] = chapter.date_fetch
+                        } else {
+                            this[ChapterTable.date_upload] = chapter.date_upload
+                        }
+                        this[ChapterTable.chapter_number] = chapter.chapter_number
+                        this[ChapterTable.scanlator] = chapter.scanlator
+
+                        this[ChapterTable.sourceOrder] = chaptersLength - chapter.source_order
+                        this[ChapterTable.manga] = mangaId
+
+                        this[ChapterTable.isRead] = chapter.read
+                        this[ChapterTable.lastPageRead] = chapter.last_page_read
+                        this[ChapterTable.isBookmarked] = chapter.bookmark
+
+                        this[ChapterTable.fetchedAt] = TimeUnit.MILLISECONDS.toSeconds(chapter.date_fetch)
+                    }
+
+                    // insert categories
+                    categories.forEach { backupCategoryOrder ->
+                        CategoryManga.addMangaToCategory(mangaId, categoryMapping[backupCategoryOrder]!!)
+                    }
+
+                    mangaId
+                }
+            } else { // Manga in database
+                transaction {
+                    val mangaId = dbManga[MangaTable.id].value
+
+                    // Merge manga data
+                    MangaTable.update({ MangaTable.id eq mangaId }) {
+                        it[artist] = manga.artist ?: dbManga[artist]
+                        it[author] = manga.author ?: dbManga[author]
+                        it[description] = manga.description ?: dbManga[description]
+                        it[genre] = manga.genre ?: dbManga[genre]
                         it[status] = manga.status
-                        it[thumbnail_url] = manga.thumbnail_url
+                        it[thumbnail_url] = manga.thumbnail_url ?: dbManga[thumbnail_url]
                         it[updateStrategy] = manga.update_strategy.name
 
-                        it[sourceReference] = manga.source
+                        it[initialized] = dbManga[initialized] || manga.description != null
 
-                        it[initialized] = manga.description != null
+                        it[inLibrary] = manga.favorite || dbManga[inLibrary]
-
-                        it[inLibrary] = manga.favorite
 
                         it[inLibraryAt] = TimeUnit.MILLISECONDS.toSeconds(manga.date_added)
-                    }.value
-
-                // delete thumbnail in case cached data still exists
-                clearThumbnail(mangaId)
-
-                // insert chapter data
-                val chaptersLength = chapters.size
-                ChapterTable.batchInsert(chapters) { chapter ->
-                    this[ChapterTable.url] = chapter.url
-                    this[ChapterTable.name] = chapter.name
-                    if (chapter.date_upload == 0L) {
-                        this[ChapterTable.date_upload] = chapter.date_fetch
-                    } else {
-                        this[ChapterTable.date_upload] = chapter.date_upload
                     }
-                    this[ChapterTable.chapter_number] = chapter.chapter_number
-                    this[ChapterTable.scanlator] = chapter.scanlator
 
-                    this[ChapterTable.sourceOrder] = chaptersLength - chapter.source_order
+                    // merge chapter data
-                    this[ChapterTable.manga] = mangaId
+                    val chaptersLength = chapters.size
+                    val dbChapters = ChapterTable.select { ChapterTable.manga eq mangaId }
 
-                    this[ChapterTable.isRead] = chapter.read
+                    chapters.forEach { chapter ->
-                    this[ChapterTable.lastPageRead] = chapter.last_page_read
+                        val dbChapter = dbChapters.find { it[ChapterTable.url] == chapter.url }
-                    this[ChapterTable.isBookmarked] = chapter.bookmark
 
-                    this[ChapterTable.fetchedAt] = TimeUnit.MILLISECONDS.toSeconds(chapter.date_fetch)
+                        if (dbChapter == null) {
-                }
+                            ChapterTable.insert {
+                                it[url] = chapter.url
+                                it[name] = chapter.name
+                                if (chapter.date_upload == 0L) {
+                                    it[date_upload] = chapter.date_fetch
+                                } else {
+                                    it[date_upload] = chapter.date_upload
+                                }
+                                it[chapter_number] = chapter.chapter_number
+                                it[scanlator] = chapter.scanlator
 
-                // insert categories
+                                it[sourceOrder] = chaptersLength - chapter.source_order
-                categories.forEach { backupCategoryOrder ->
+                                it[ChapterTable.manga] = mangaId
-                    CategoryManga.addMangaToCategory(mangaId, categoryMapping[backupCategoryOrder]!!)
-                }
-            }
-        } else { // Manga in database
-            transaction {
-                val mangaId = dbManga[MangaTable.id].value
 
-                // Merge manga data
+                                it[isRead] = chapter.read
-                MangaTable.update({ MangaTable.id eq mangaId }) {
+                                it[lastPageRead] = chapter.last_page_read
-                    it[artist] = manga.artist ?: dbManga[artist]
+                                it[isBookmarked] = chapter.bookmark
-                    it[author] = manga.author ?: dbManga[author]
+                            }
-                    it[description] = manga.description ?: dbManga[description]
+                        } else {
-                    it[genre] = manga.genre ?: dbManga[genre]
+                            ChapterTable.update({ (ChapterTable.url eq dbChapter[ChapterTable.url]) and (ChapterTable.manga eq mangaId) }) {
-                    it[status] = manga.status
+                                it[isRead] = chapter.read || dbChapter[isRead]
-                    it[thumbnail_url] = manga.thumbnail_url ?: dbManga[thumbnail_url]
+                                it[lastPageRead] = max(chapter.last_page_read, dbChapter[lastPageRead])
-                    it[updateStrategy] = manga.update_strategy.name
+                                it[isBookmarked] = chapter.bookmark || dbChapter[isBookmarked]
-
-                    it[initialized] = dbManga[initialized] || manga.description != null
-
-                    it[inLibrary] = manga.favorite || dbManga[inLibrary]
-
-                    it[inLibraryAt] = TimeUnit.MILLISECONDS.toSeconds(manga.date_added)
-                }
-
-                // merge chapter data
-                val chaptersLength = chapters.size
-                val dbChapters = ChapterTable.select { ChapterTable.manga eq mangaId }
-
-                chapters.forEach { chapter ->
-                    val dbChapter = dbChapters.find { it[ChapterTable.url] == chapter.url }
-
-                    if (dbChapter == null) {
-                        ChapterTable.insert {
-                            it[url] = chapter.url
-                            it[name] = chapter.name
-                            if (chapter.date_upload == 0L) {
-                                it[date_upload] = chapter.date_fetch
-                            } else {
-                                it[date_upload] = chapter.date_upload
                             }
-                            it[chapter_number] = chapter.chapter_number
-                            it[scanlator] = chapter.scanlator
-
-                            it[sourceOrder] = chaptersLength - chapter.source_order
-                            it[ChapterTable.manga] = mangaId
-
-                            it[isRead] = chapter.read
-                            it[lastPageRead] = chapter.last_page_read
-                            it[isBookmarked] = chapter.bookmark
-                        }
-                    } else {
-                        ChapterTable.update({ (ChapterTable.url eq dbChapter[ChapterTable.url]) and (ChapterTable.manga eq mangaId) }) {
-                            it[isRead] = chapter.read || dbChapter[isRead]
-                            it[lastPageRead] = max(chapter.last_page_read, dbChapter[lastPageRead])
-                            it[isBookmarked] = chapter.bookmark || dbChapter[isBookmarked]
                         }
                     }
-                }
 
-                // merge categories
+                    // merge categories
-                categories.forEach { backupCategoryOrder ->
+                    categories.forEach { backupCategoryOrder ->
-                    CategoryManga.addMangaToCategory(mangaId, categoryMapping[backupCategoryOrder]!!)
+                        CategoryManga.addMangaToCategory(mangaId, categoryMapping[backupCategoryOrder]!!)
+                    }
+
+                    mangaId
                 }
             }
-        }
+
+        val dbTrackRecordsByTrackerId =
+            Tracker.getTrackRecordsByMangaId(mangaId)
+                .mapNotNull { it.record?.toTrack() }
+                .associateBy { it.sync_id }
+
+        val (existingTracks, newTracks) =
+            tracks.mapNotNull { backupTrack ->
+                val track = backupTrack.toTrack(mangaId)
+
+                val isUnsupportedTracker = TrackerManager.getTracker(track.sync_id) == null
+                if (isUnsupportedTracker) {
+                    return@mapNotNull null
+                }
+
+                val dbTrack =
+                    dbTrackRecordsByTrackerId[backupTrack.syncId]
+                        ?: // new track
+                        return@mapNotNull track
+
+                if (track.toTrackRecordDataClass().forComparison() == dbTrack.toTrackRecordDataClass().forComparison()) {
+                    return@mapNotNull null
+                }
+
+                dbTrack.also {
+                    it.media_id = track.media_id
+                    it.library_id = track.library_id
+                    it.last_chapter_read = max(dbTrack.last_chapter_read, track.last_chapter_read)
+                }
+            }.partition { (it.id ?: -1) > 0 }
+
+        existingTracks.forEach(Tracker::updateTrackRecord)
+        newTracks.forEach(Tracker::insertTrackRecord)
 
         // TODO: insert/merge history
-
-        // TODO: insert/merge tracking
     }
+
+    private fun TrackRecordDataClass.forComparison() = this.copy(id = 0, mangaId = 0)
 }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/backup/proto/ProtoBackupValidator.kt

@@ -15,6 +15,7 @@ import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
 import suwayomi.tachidesk.manga.impl.backup.proto.models.Backup
 import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupSerializer
+import suwayomi.tachidesk.manga.impl.track.tracker.TrackerManager
 import suwayomi.tachidesk.manga.model.table.SourceTable
 import java.io.InputStream
 
@@ -39,17 +40,18 @@ object ProtoBackupValidator {
                 sources.filter { SourceTable.select { SourceTable.id eq it.key }.firstOrNull() == null }
             }
 
-//        val trackers = backup.backupManga
+        val trackers =
-//            .flatMap { it.tracking }
+            backup.backupManga
-//            .map { it.syncId }
+                .flatMap { it.tracking }
-//            .distinct()
+                .map { it.syncId }
+                .distinct()
 
-        val missingTrackers = listOf("")
+        val missingTrackers =
-//        val missingTrackers = trackers
+            trackers
-//            .mapNotNull { trackManager.getService(it) }
+                .mapNotNull { TrackerManager.getTracker(it) }
-//            .filter { !it.isLogged }
+                .filter { !it.isLoggedIn }
-//            .map { context.getString(it.nameRes()) }
+                .map { it.name }
-//            .sorted()
+                .sorted()
 
         return ValidationResult(
             missingSources

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/backup/proto/models/BackupTracking.kt

@@ -8,11 +8,12 @@ import suwayomi.tachidesk.manga.impl.backup.models.TrackImpl
 @Serializable
 data class BackupTracking(
     // in 1.x some of these values have different types or names
-    // syncId is called siteId in 1,x
     @ProtoNumber(1) var syncId: Int,
     // LibraryId is not null in 1.x
     @ProtoNumber(2) var libraryId: Long,
-    @ProtoNumber(3) var mediaId: Int = 0,
+    @Deprecated("Use mediaId instead", level = DeprecationLevel.WARNING)
+    @ProtoNumber(3)
+    var mediaIdInt: Int = 0,
     // trackingUrl is called mediaUrl in 1.x
     @ProtoNumber(4) var trackingUrl: String = "",
     @ProtoNumber(5) var title: String = "",
@@ -25,11 +26,17 @@ data class BackupTracking(
     @ProtoNumber(10) var startedReadingDate: Long = 0,
     // finishedReadingDate is called endReadTime in 1.x
     @ProtoNumber(11) var finishedReadingDate: Long = 0,
+    @ProtoNumber(100) var mediaId: Long = 0,
 ) {
     fun getTrackingImpl(): TrackImpl {
         return TrackImpl().apply {
             sync_id = this@BackupTracking.syncId
-            media_id = this@BackupTracking.mediaId
+            media_id =
+                if (this@BackupTracking.mediaIdInt != 0) {
+                    this@BackupTracking.mediaIdInt.toLong()
+                } else {
+                    this@BackupTracking.mediaId
+                }
             library_id = this@BackupTracking.libraryId
             title = this@BackupTracking.title
             // convert from float to int because of 1.x types

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/download/fileProvider/ChaptersFilesProvider.kt

@@ -13,6 +13,8 @@ import suwayomi.tachidesk.manga.impl.Page
 import suwayomi.tachidesk.manga.impl.download.model.DownloadChapter
 import suwayomi.tachidesk.manga.impl.util.createComicInfoFile
 import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
+import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
+import suwayomi.tachidesk.manga.impl.util.storage.ImageResponse
 import suwayomi.tachidesk.manga.model.table.ChapterTable
 import suwayomi.tachidesk.manga.model.table.MangaTable
 import java.io.File
@@ -28,21 +30,40 @@ abstract class ChaptersFilesProvider(val mangaId: Int, val chapterId: Int) : Dow
         return RetrieveFile1Args(::getImageImpl)
     }
 
+    /**
+     * Extract the existing download to the base download folder (see [getChapterDownloadPath])
+     */
+    protected abstract fun extractExistingDownload()
+
+    protected abstract suspend fun handleSuccessfulDownload()
+
     @OptIn(FlowPreview::class)
-    open suspend fun downloadImpl(
+    private suspend fun downloadImpl(
         download: DownloadChapter,
         scope: CoroutineScope,
         step: suspend (DownloadChapter?, Boolean) -> Unit,
     ): Boolean {
-        val pageCount = download.chapter.pageCount
+        extractExistingDownload()
-        val chapterDir = getChapterCachePath(mangaId, chapterId)
-        val folder = File(chapterDir)
-        folder.mkdirs()
 
+        val finalDownloadFolder = getChapterDownloadPath(mangaId, chapterId)
+
+        val cacheChapterDir = getChapterCachePath(mangaId, chapterId)
+        val downloadCacheFolder = File(cacheChapterDir)
+        downloadCacheFolder.mkdirs()
+
+        val pageCount = download.chapter.pageCount
         for (pageNum in 0 until pageCount) {
             var pageProgressJob: Job? = null
             val fileName = Page.getPageName(pageNum) // might have to change this to index stored in database
-            if (File(folder, fileName).exists()) continue
+
+            val pageExistsInFinalDownloadFolder = ImageResponse.findFileNameStartingWith(finalDownloadFolder, fileName) != null
+            val pageExistsInCacheDownloadFolder = ImageResponse.findFileNameStartingWith(cacheChapterDir, fileName) != null
+
+            val doesPageAlreadyExist = pageExistsInFinalDownloadFolder || pageExistsInCacheDownloadFolder
+            if (doesPageAlreadyExist) {
+                continue
+            }
+
             try {
                 Page.getPageImage(
                     mangaId = download.mangaId,
@@ -69,7 +90,7 @@ abstract class ChaptersFilesProvider(val mangaId: Int, val chapterId: Int) : Dow
         }
 
         createComicInfoFile(
-            folder.toPath(),
+            downloadCacheFolder.toPath(),
             transaction {
                 MangaTable.select { MangaTable.id eq mangaId }.first()
             },
@@ -78,6 +99,10 @@ abstract class ChaptersFilesProvider(val mangaId: Int, val chapterId: Int) : Dow
             },
         )
 
+        handleSuccessfulDownload()
+
+        File(cacheChapterDir).deleteRecursively()
+
         return true
     }
 

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/download/fileProvider/impl/ArchiveProvider.kt

@@ -1,6 +1,5 @@
 package suwayomi.tachidesk.manga.impl.download.fileProvider.impl
 
-import kotlinx.coroutines.CoroutineScope
 import kotlinx.coroutines.Dispatchers
 import kotlinx.coroutines.withContext
 import org.apache.commons.compress.archivers.zip.ZipArchiveEntry
@@ -11,7 +10,6 @@ import org.kodein.di.DI
 import org.kodein.di.conf.global
 import org.kodein.di.instance
 import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
-import suwayomi.tachidesk.manga.impl.download.model.DownloadChapter
 import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
 import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
 import suwayomi.tachidesk.manga.impl.util.getMangaDownloadDir
@@ -32,17 +30,21 @@ class ArchiveProvider(mangaId: Int, chapterId: Int) : ChaptersFilesProvider(mang
         return Pair(inputStream.buffered(), "image/$fileType")
     }
 
-    override suspend fun downloadImpl(
+    override fun extractExistingDownload() {
-        download: DownloadChapter,
+        val outputFile = File(getChapterCbzPath(mangaId, chapterId))
-        scope: CoroutineScope,
+        val chapterCacheFolder = File(getChapterCachePath(mangaId, chapterId))
-        step: suspend (DownloadChapter?, Boolean) -> Unit,
+
-    ): Boolean {
+        if (!outputFile.exists()) {
+            return
+        }
+
+        extractCbzFile(outputFile, chapterCacheFolder)
+    }
+
+    override suspend fun handleSuccessfulDownload() {
         val mangaDownloadFolder = File(getMangaDownloadDir(mangaId))
         val outputFile = File(getChapterCbzPath(mangaId, chapterId))
         val chapterCacheFolder = File(getChapterCachePath(mangaId, chapterId))
-        if (outputFile.exists()) handleExistingCbzFile(outputFile, chapterCacheFolder)
-
-        super.downloadImpl(download, scope, step)
 
         withContext(Dispatchers.IO) {
             mangaDownloadFolder.mkdirs()
@@ -68,8 +70,6 @@ class ArchiveProvider(mangaId: Int, chapterId: Int) : ChaptersFilesProvider(mang
         if (chapterCacheFolder.exists() && chapterCacheFolder.isDirectory) {
             chapterCacheFolder.deleteRecursively()
         }
-
-        return true
     }
 
     override fun delete(): Boolean {
@@ -83,7 +83,7 @@ class ArchiveProvider(mangaId: Int, chapterId: Int) : ChaptersFilesProvider(mang
         return cbzDeleted
     }
 
-    private fun handleExistingCbzFile(
+    private fun extractCbzFile(
         cbzFile: File,
         chapterFolder: File,
     ) {

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/download/fileProvider/impl/FolderProvider.kt

@@ -1,11 +1,9 @@
 package suwayomi.tachidesk.manga.impl.download.fileProvider.impl
 
-import kotlinx.coroutines.CoroutineScope
 import org.kodein.di.DI
 import org.kodein.di.conf.global
 import org.kodein.di.instance
 import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
-import suwayomi.tachidesk.manga.impl.download.model.DownloadChapter
 import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
 import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
 import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper
@@ -29,23 +27,16 @@ class FolderProvider(mangaId: Int, chapterId: Int) : ChaptersFilesProvider(manga
         return Pair(FileInputStream(file).buffered(), "image/$fileType")
     }
 
-    override suspend fun downloadImpl(
+    override fun extractExistingDownload() {
-        download: DownloadChapter,
+        // nothing to do
-        scope: CoroutineScope,
+    }
-        step: suspend (DownloadChapter?, Boolean) -> Unit,
+
-    ): Boolean {
+    override suspend fun handleSuccessfulDownload() {
         val chapterDir = getChapterDownloadPath(mangaId, chapterId)
         val folder = File(chapterDir)
 
-        val downloadSucceeded = super.downloadImpl(download, scope, step)
-        if (!downloadSucceeded) {
-            return false
-        }
-
         val cacheChapterDir = getChapterCachePath(mangaId, chapterId)
         File(cacheChapterDir).copyRecursively(folder, true)
-
-        return true
     }
 
     override fun delete(): Boolean {

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/Track.kt

@@ -15,6 +15,7 @@ import org.jetbrains.exposed.sql.insertAndGetId
 import org.jetbrains.exposed.sql.select
 import org.jetbrains.exposed.sql.transactions.transaction
 import org.jetbrains.exposed.sql.update
+import suwayomi.tachidesk.manga.impl.track.tracker.DeletableTrackService
 import suwayomi.tachidesk.manga.impl.track.tracker.TrackerManager
 import suwayomi.tachidesk.manga.impl.track.tracker.model.Track
 import suwayomi.tachidesk.manga.impl.track.tracker.model.toTrack
@@ -73,9 +74,6 @@ object Track {
     }
 
     fun getTrackRecordsByMangaId(mangaId: Int): List<MangaTrackerDataClass> {
-        if (!TrackerManager.hasLoggedTracker()) {
-            return emptyList()
-        }
         val recordMap =
             transaction {
                 TrackRecordTable.select { TrackRecordTable.mangaId eq mangaId }
@@ -83,27 +81,25 @@ object Track {
             }.associateBy { it.trackerId }
 
         val trackers = TrackerManager.services
-        return trackers
+        return trackers.map {
-            .filter { it.isLoggedIn }
+            val record = recordMap[it.id]
-            .map {
+            if (record != null) {
-                val record = recordMap[it.id]
+                val track =
-                if (record != null) {
+                    Track.create(it.id).also { t ->
-                    val track =
+                        t.score = record.score.toFloat()
-                        Track.create(it.id).also { t ->
+                    }
-                            t.score = record.score.toFloat()
+                record.scoreString = it.displayScore(track)
-                        }
-                    record.scoreString = it.displayScore(track)
-                }
-                MangaTrackerDataClass(
-                    id = it.id,
-                    name = it.name,
-                    icon = proxyThumbnailUrl(it.id),
-                    statusList = it.getStatusList(),
-                    statusTextMap = it.getStatusList().associateWith { k -> it.getStatus(k).orEmpty() },
-                    scoreList = it.getScoreList(),
-                    record = record,
-                )
             }
+            MangaTrackerDataClass(
+                id = it.id,
+                name = it.name,
+                icon = proxyThumbnailUrl(it.id),
+                statusList = it.getStatusList(),
+                statusTextMap = it.getStatusList().associateWith { k -> it.getStatus(k).orEmpty() },
+                scoreList = it.getScoreList(),
+                record = record,
+            )
+        }
     }
 
     suspend fun search(input: SearchInput): List<TrackSearchDataClass> {
@@ -158,7 +154,7 @@ object Track {
 
         var lastChapterRead: Double? = null
         var startDate: Long? = null
-        if (chapterNumber != null && chapterNumber > 0) {
+        if (chapterNumber != null && chapterNumber > 0 && chapterNumber > track.last_chapter_read) {
             lastChapterRead = chapterNumber.toDouble()
         }
         if (track.started_reading_date <= 0) {
@@ -186,11 +182,42 @@ object Track {
         }
     }
 
+    suspend fun refresh(recordId: Int) {
+        val recordDb =
+            transaction {
+                TrackRecordTable.select { TrackRecordTable.id eq recordId }.first()
+            }
+
+        val tracker = TrackerManager.getTracker(recordDb[TrackRecordTable.trackerId])!!
+
+        val track = recordDb.toTrack()
+        tracker.refresh(track)
+        upsertTrackRecord(track)
+    }
+
+    suspend fun unbind(
+        recordId: Int,
+        deleteRemoteTrack: Boolean? = false,
+    ) {
+        val recordDb =
+            transaction {
+                TrackRecordTable.select { TrackRecordTable.id eq recordId }.first()
+            }
+
+        val tracker = TrackerManager.getTracker(recordDb[TrackRecordTable.trackerId])
+
+        if (deleteRemoteTrack == true && tracker is DeletableTrackService) {
+            tracker.delete(recordDb.toTrack())
+        }
+
+        transaction {
+            TrackRecordTable.deleteWhere { TrackRecordTable.id eq recordId }
+        }
+    }
+
     suspend fun update(input: UpdateInput) {
         if (input.unbind == true) {
-            transaction {
+            unbind(input.recordId)
-                TrackRecordTable.deleteWhere { TrackRecordTable.id eq input.recordId }
-            }
             return
         }
         val recordDb =
@@ -250,13 +277,14 @@ object Track {
         }
     }
 
-    private suspend fun trackChapter(mangaId: Int) {
+    suspend fun trackChapter(mangaId: Int) {
         val chapter = queryMaxReadChapter(mangaId)
         val chapterNumber = chapter?.get(ChapterTable.chapter_number)
-        logger.debug {
+
-            "[Tracker]mangaId $mangaId chapter:${chapter?.get(ChapterTable.name)} " +
+        logger.info {
-                "chapterNumber:$chapterNumber"
+            "trackChapter(mangaId= $mangaId): maxReadChapter= #$chapterNumber ${chapter?.get(ChapterTable.name)}"
         }
+
         if (chapterNumber != null && chapterNumber > 0) {
             trackChapter(mangaId, chapterNumber.toDouble())
         }
@@ -282,23 +310,55 @@ object Track {
             }
 
         records.forEach {
-            val tracker = TrackerManager.getTracker(it[TrackRecordTable.trackerId])
+            try {
-            val lastChapterRead = it[TrackRecordTable.lastChapterRead]
+                trackChapterForTracker(it, chapterNumber)
-            val isLogin = tracker?.isLoggedIn == true
+            } catch (e: Exception) {
-            logger.debug {
+                KotlinLogging.logger("${logger.name}::trackChapter(mangaId= $mangaId, chapterNumber= $chapterNumber)")
-                "[Tracker]trackChapter id:${tracker?.id} login:$isLogin " +
+                    .error(e) { "failed due to" }
-                    "mangaId:$mangaId dbChapter:$lastChapterRead toChapter:$chapterNumber"
-            }
-            if (isLogin && chapterNumber > lastChapterRead) {
-                it[TrackRecordTable.lastChapterRead] = chapterNumber
-                val track = it.toTrack()
-                tracker?.update(track, true)
-                upsertTrackRecord(track)
             }
         }
     }
 
-    private fun upsertTrackRecord(track: Track): Int {
+    private suspend fun trackChapterForTracker(
+        it: ResultRow,
+        chapterNumber: Double,
+    ) {
+        val tracker = TrackerManager.getTracker(it[TrackRecordTable.trackerId]) ?: return
+        val track = it.toTrack()
+
+        val log =
+            KotlinLogging.logger {
+                "${logger.name}::trackChapterForTracker(chapterNumber= $chapterNumber, tracker= ${tracker.id}, recordId= ${track.id})"
+            }
+        log.debug { "called for $tracker, ${track.title} (recordId= ${track.id}, mangaId= ${track.manga_id})" }
+
+        val localLastReadChapter = it[TrackRecordTable.lastChapterRead]
+
+        if (localLastReadChapter == chapterNumber) {
+            log.debug { "new chapter is the same as the local last read chapter" }
+            return
+        }
+
+        if (!tracker.isLoggedIn) {
+            upsertTrackRecord(track)
+            return
+        }
+
+        tracker.refresh(track)
+        upsertTrackRecord(track)
+
+        val lastChapterRead = track.last_chapter_read
+
+        log.debug { "remoteLastReadChapter= $lastChapterRead" }
+
+        if (chapterNumber > lastChapterRead) {
+            track.last_chapter_read = chapterNumber.toFloat()
+            tracker.update(track, true)
+            upsertTrackRecord(track)
+        }
+    }
+
+    fun upsertTrackRecord(track: Track): Int {
         return transaction {
             val existingRecord =
                 TrackRecordTable.select {
@@ -308,41 +368,53 @@ object Track {
                     .singleOrNull()
 
             if (existingRecord != null) {
-                TrackRecordTable.update({
+                updateTrackRecord(track)
-                    (TrackRecordTable.mangaId eq track.manga_id) and
-                        (TrackRecordTable.trackerId eq track.sync_id)
-                }) {
-                    it[remoteId] = track.media_id
-                    it[libraryId] = track.library_id
-                    it[title] = track.title
-                    it[lastChapterRead] = track.last_chapter_read.toDouble()
-                    it[totalChapters] = track.total_chapters
-                    it[status] = track.status
-                    it[score] = track.score.toDouble()
-                    it[remoteUrl] = track.tracking_url
-                    it[startDate] = track.started_reading_date
-                    it[finishDate] = track.finished_reading_date
-                }
                 existingRecord[TrackRecordTable.id].value
             } else {
-                TrackRecordTable.insertAndGetId {
+                insertTrackRecord(track)
-                    it[mangaId] = track.manga_id
-                    it[trackerId] = track.sync_id
-                    it[remoteId] = track.media_id
-                    it[libraryId] = track.library_id
-                    it[title] = track.title
-                    it[lastChapterRead] = track.last_chapter_read.toDouble()
-                    it[totalChapters] = track.total_chapters
-                    it[status] = track.status
-                    it[score] = track.score.toDouble()
-                    it[remoteUrl] = track.tracking_url
-                    it[startDate] = track.started_reading_date
-                    it[finishDate] = track.finished_reading_date
-                }.value
             }
         }
     }
 
+    fun updateTrackRecord(track: Track): Int =
+        transaction {
+            TrackRecordTable.update(
+                {
+                    (TrackRecordTable.mangaId eq track.manga_id) and
+                        (TrackRecordTable.trackerId eq track.sync_id)
+                },
+            ) {
+                it[remoteId] = track.media_id
+                it[libraryId] = track.library_id
+                it[title] = track.title
+                it[lastChapterRead] = track.last_chapter_read.toDouble()
+                it[totalChapters] = track.total_chapters
+                it[status] = track.status
+                it[score] = track.score.toDouble()
+                it[remoteUrl] = track.tracking_url
+                it[startDate] = track.started_reading_date
+                it[finishDate] = track.finished_reading_date
+            }
+        }
+
+    fun insertTrackRecord(track: Track): Int =
+        transaction {
+            TrackRecordTable.insertAndGetId {
+                it[mangaId] = track.manga_id
+                it[trackerId] = track.sync_id
+                it[remoteId] = track.media_id
+                it[libraryId] = track.library_id
+                it[title] = track.title
+                it[lastChapterRead] = track.last_chapter_read.toDouble()
+                it[totalChapters] = track.total_chapters
+                it[status] = track.status
+                it[score] = track.score.toDouble()
+                it[remoteUrl] = track.tracking_url
+                it[startDate] = track.started_reading_date
+                it[finishDate] = track.finished_reading_date
+            }.value
+        }
+
     @Serializable
     data class LoginInput(
         val trackerId: Int,

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/DeletableTrackService.kt

@@ -6,5 +6,5 @@ import suwayomi.tachidesk.manga.impl.track.tracker.model.Track
  * For track services api that support deleting a manga entry for a user's list
  */
 interface DeletableTrackService {
-    suspend fun delete(track: Track): Track
+    suspend fun delete(track: Track)
 }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/Tracker.kt

@@ -17,6 +17,10 @@ abstract class Tracker(val id: Int, val name: String) {
     // Application and remote support for reading dates
     open val supportsReadingDates: Boolean = false
 
+    abstract val supportsTrackDeletion: Boolean
+
+    override fun toString() = "$name ($id) (isLoggedIn= $isLoggedIn, isAuthExpired= ${getIfAuthExpired()})"
+
     abstract fun getLogo(): String
 
     abstract fun getStatusList(): List<Int>

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/anilist/Anilist.kt

@@ -1,7 +1,6 @@
 package suwayomi.tachidesk.manga.impl.track.tracker.anilist
 
 import android.annotation.StringRes
-import kotlinx.serialization.decodeFromString
 import kotlinx.serialization.encodeToString
 import kotlinx.serialization.json.Json
 import mu.KotlinLogging
@@ -29,9 +28,11 @@ class Anilist(id: Int) : Tracker(id, "AniList"), DeletableTrackService {
         const val POINT_3 = "POINT_3"
     }
 
+    override val supportsTrackDeletion: Boolean = true
+
     private val json: Json by injectLazy()
 
-    private val interceptor by lazy { AnilistInterceptor(this, getPassword()) }
+    private val interceptor by lazy { AnilistInterceptor(this) }
 
     private val api by lazy { AnilistApi(client, interceptor) }
 
@@ -157,13 +158,13 @@ class Anilist(id: Int) : Tracker(id, "AniList"), DeletableTrackService {
         return api.updateLibManga(track)
     }
 
-    override suspend fun delete(track: Track): Track {
+    override suspend fun delete(track: Track) {
         if (track.library_id == null || track.library_id!! == 0L) {
-            val libManga = api.findLibManga(track, getUsername().toInt()) ?: return track
+            val libManga = api.findLibManga(track, getUsername().toInt()) ?: return
             track.library_id = libManga.library_id
         }
 
-        return api.deleteLibManga(track)
+        api.deleteLibManga(track)
     }
 
     override suspend fun bind(

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/anilist/AnilistApi.kt

@@ -115,7 +115,7 @@ class AnilistApi(val client: OkHttpClient, interceptor: AnilistInterceptor) {
         }
     }
 
-    suspend fun deleteLibManga(track: Track): Track {
+    suspend fun deleteLibManga(track: Track) {
         return withIOContext {
             val query =
                 """
@@ -135,7 +135,6 @@ class AnilistApi(val client: OkHttpClient, interceptor: AnilistInterceptor) {
                 }
             authClient.newCall(POST(API_URL, body = payload.toString().toRequestBody(jsonMime)))
                 .awaitSuccess()
-            track
         }
     }
 

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/anilist/AnilistInterceptor.kt

@@ -5,7 +5,7 @@ import okhttp3.Response
 import suwayomi.tachidesk.manga.impl.track.tracker.TokenExpired
 import java.io.IOException
 
-class AnilistInterceptor(val anilist: Anilist, private var token: String?) : Interceptor {
+class AnilistInterceptor(private val anilist: Anilist) : Interceptor {
     /**
      * OAuth object used for authenticated requests.
      *
@@ -17,18 +17,16 @@ class AnilistInterceptor(val anilist: Anilist, private var token: String?) : Int
             field = value?.copy(expires = value.expires * 1000 - 60 * 1000)
         }
 
+    init {
+        oauth = anilist.loadOAuth()
+    }
+
     override fun intercept(chain: Interceptor.Chain): Response {
         if (anilist.getIfAuthExpired()) {
             throw TokenExpired()
         }
         val originalRequest = chain.request()
 
-        if (token.isNullOrEmpty()) {
-            throw Exception("Not authenticated with Anilist")
-        }
-        if (oauth == null) {
-            oauth = anilist.loadOAuth()
-        }
         // Refresh access token if null or expired.
         if (oauth?.isExpired() == true) {
             anilist.setAuthExpired()
@@ -37,7 +35,7 @@ class AnilistInterceptor(val anilist: Anilist, private var token: String?) : Int
 
         // Throw on null auth.
         if (oauth == null) {
-            throw IOException("No authentication token")
+            throw IOException("Anilist: User is not authenticated")
         }
 
         // Add the authorization header to the original request.
@@ -54,7 +52,6 @@ class AnilistInterceptor(val anilist: Anilist, private var token: String?) : Int
      * and the oauth object.
      */
     fun setAuth(oauth: OAuth?) {
-        token = oauth?.access_token
         this.oauth = oauth
         anilist.saveOAuth(oauth)
     }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/mangaupdates/MangaUpdates.kt

@@ -1,5 +1,6 @@
 package suwayomi.tachidesk.manga.impl.track.tracker.mangaupdates
 
+import suwayomi.tachidesk.manga.impl.track.tracker.DeletableTrackService
 import suwayomi.tachidesk.manga.impl.track.tracker.Tracker
 import suwayomi.tachidesk.manga.impl.track.tracker.mangaupdates.dto.ListItem
 import suwayomi.tachidesk.manga.impl.track.tracker.mangaupdates.dto.Rating
@@ -8,8 +9,7 @@ import suwayomi.tachidesk.manga.impl.track.tracker.mangaupdates.dto.toTrackSearc
 import suwayomi.tachidesk.manga.impl.track.tracker.model.Track
 import suwayomi.tachidesk.manga.impl.track.tracker.model.TrackSearch
 
-class MangaUpdates(id: Int) : Tracker(id, "MangaUpdates") {
+class MangaUpdates(id: Int) : Tracker(id, "MangaUpdates"), DeletableTrackService {
-    // , DeletableTracker
     companion object {
         const val READING_LIST = 0
         const val WISH_LIST = 1
@@ -31,6 +31,8 @@ class MangaUpdates(id: Int) : Tracker(id, "MangaUpdates") {
                 }
     }
 
+    override val supportsTrackDeletion: Boolean = true
+
     private val interceptor by lazy { MangaUpdatesInterceptor(this) }
 
     private val api by lazy { MangaUpdatesApi(interceptor, client) }
@@ -74,9 +76,9 @@ class MangaUpdates(id: Int) : Tracker(id, "MangaUpdates") {
         return track
     }
 
-    // override suspend fun delete(track: Track) {
+    override suspend fun delete(track: Track) {
-    //     api.deleteSeriesFromList(track)
+        api.deleteSeriesFromList(track)
-    // }
+    }
 
     override suspend fun bind(
         track: Track,

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/model/TrackConvertor.kt

@@ -1,8 +1,11 @@
 package suwayomi.tachidesk.manga.impl.track.tracker.model
 
 import org.jetbrains.exposed.sql.ResultRow
+import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupTracking
 import suwayomi.tachidesk.manga.model.dataclass.TrackRecordDataClass
 import suwayomi.tachidesk.manga.model.table.TrackRecordTable
+import suwayomi.tachidesk.manga.model.table.TrackRecordTable.lastChapterRead
+import suwayomi.tachidesk.manga.model.table.TrackRecordTable.remoteUrl
 
 fun ResultRow.toTrackRecordDataClass(): TrackRecordDataClass =
     TrackRecordDataClass(
@@ -36,3 +39,52 @@ fun ResultRow.toTrack(): Track =
         it.started_reading_date = this[TrackRecordTable.startDate]
         it.finished_reading_date = this[TrackRecordTable.finishDate]
     }
+
+fun BackupTracking.toTrack(mangaId: Int): Track =
+    Track.create(syncId).also {
+        it.id = -1
+        it.manga_id = mangaId
+        it.media_id = mediaId
+        it.library_id = libraryId
+        it.title = title
+        it.last_chapter_read = lastChapterRead
+        it.total_chapters = totalChapters
+        it.status = status
+        it.score = score
+        it.tracking_url = trackingUrl
+        it.started_reading_date = startedReadingDate
+        it.finished_reading_date = finishedReadingDate
+    }
+
+fun TrackRecordDataClass.toTrack(): Track =
+    Track.create(trackerId).also {
+        it.id = id
+        it.manga_id = mangaId
+        it.media_id = remoteId
+        it.library_id = libraryId
+        it.title = title
+        it.last_chapter_read = lastChapterRead.toFloat()
+        it.total_chapters = totalChapters
+        it.status = status
+        it.score = score.toFloat()
+        it.tracking_url = remoteUrl
+        it.started_reading_date = startDate
+        it.finished_reading_date = finishDate
+    }
+
+fun Track.toTrackRecordDataClass(): TrackRecordDataClass =
+    TrackRecordDataClass(
+        id = id ?: -1,
+        mangaId = manga_id,
+        trackerId = sync_id,
+        remoteId = media_id,
+        libraryId = library_id,
+        title = title,
+        lastChapterRead = last_chapter_read.toDouble(),
+        totalChapters = total_chapters,
+        status = status,
+        score = score.toDouble(),
+        remoteUrl = tracking_url,
+        startDate = started_reading_date,
+        finishDate = finished_reading_date,
+    )

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/myanimelist/MyAnimeList.kt

@@ -1,7 +1,6 @@
 package suwayomi.tachidesk.manga.impl.track.tracker.myanimelist
 
 import android.annotation.StringRes
-import kotlinx.serialization.decodeFromString
 import kotlinx.serialization.encodeToString
 import kotlinx.serialization.json.Json
 import mu.KotlinLogging
@@ -26,9 +25,11 @@ class MyAnimeList(id: Int) : Tracker(id, "MyAnimeList"), DeletableTrackService {
         private const val SEARCH_LIST_PREFIX = "my:"
     }
 
+    override val supportsTrackDeletion: Boolean = true
+
     private val json: Json by injectLazy()
 
-    private val interceptor by lazy { MyAnimeListInterceptor(this, getPassword()) }
+    private val interceptor by lazy { MyAnimeListInterceptor(this) }
     private val api by lazy { MyAnimeListApi(client, interceptor) }
 
     override val supportsReadingDates: Boolean = true
@@ -94,8 +95,8 @@ class MyAnimeList(id: Int) : Tracker(id, "MyAnimeList"), DeletableTrackService {
         return api.updateItem(track)
     }
 
-    override suspend fun delete(track: Track): Track {
+    override suspend fun delete(track: Track) {
-        return api.deleteItem(track)
+        api.deleteItem(track)
     }
 
     override suspend fun bind(

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/myanimelist/MyAnimeListApi.kt

@@ -164,18 +164,15 @@ class MyAnimeListApi(private val client: OkHttpClient, interceptor: MyAnimeListI
         }
     }
 
-    suspend fun deleteItem(track: Track): Track {
+    suspend fun deleteItem(track: Track) {
         return withIOContext {
             val request =
                 Request.Builder()
                     .url(mangaUrl(track.media_id).toString())
                     .delete()
                     .build()
-            with(json) {
+            authClient.newCall(request)
-                authClient.newCall(request)
+                .awaitSuccess()
-                    .awaitSuccess()
-                track
-            }
         }
     }
 

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/track/tracker/myanimelist/MyAnimeListInterceptor.kt

@@ -10,10 +10,10 @@ import suwayomi.tachidesk.manga.impl.track.tracker.TokenRefreshFailed
 import uy.kohesive.injekt.injectLazy
 import java.io.IOException
 
-class MyAnimeListInterceptor(private val myanimelist: MyAnimeList, private var token: String?) : Interceptor {
+class MyAnimeListInterceptor(private val myanimelist: MyAnimeList) : Interceptor {
     private val json: Json by injectLazy()
 
-    private var oauth: OAuth? = null
+    private var oauth: OAuth? = myanimelist.loadOAuth()
 
     override fun intercept(chain: Interceptor.Chain): Response {
         if (myanimelist.getIfAuthExpired()) {
@@ -44,7 +44,6 @@ class MyAnimeListInterceptor(private val myanimelist: MyAnimeList, private var t
      * and the oauth object.
      */
     fun setAuth(oauth: OAuth?) {
-        token = oauth?.access_token
         this.oauth = oauth
         myanimelist.saveOAuth(oauth)
     }
@@ -75,10 +74,7 @@ class MyAnimeListInterceptor(private val myanimelist: MyAnimeList, private var t
                 }
             }
                 .getOrNull()
-                ?.also {
+                ?.also(::setAuth)
-                    this.oauth = it
-                    myanimelist.saveOAuth(it)
-                }
                 ?: throw TokenRefreshFailed()
         }
 }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/update/IUpdater.kt

@@ -1,5 +1,6 @@
 package suwayomi.tachidesk.manga.impl.update
 
+import kotlinx.coroutines.flow.Flow
 import kotlinx.coroutines.flow.StateFlow
 import suwayomi.tachidesk.manga.model.dataclass.CategoryDataClass
 
@@ -12,7 +13,9 @@ interface IUpdater {
         forceAll: Boolean,
     )
 
-    val status: StateFlow<UpdateStatus>
+    val status: Flow<UpdateStatus>
+
+    val statusDeprecated: StateFlow<UpdateStatus>
 
     fun reset()
 }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/update/Updater.kt

@@ -6,9 +6,12 @@ import eu.kanade.tachiyomi.source.model.UpdateStrategy
 import kotlinx.coroutines.CancellationException
 import kotlinx.coroutines.CoroutineScope
 import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.FlowPreview
 import kotlinx.coroutines.SupervisorJob
 import kotlinx.coroutines.cancelChildren
+import kotlinx.coroutines.channels.BufferOverflow
 import kotlinx.coroutines.channels.Channel
+import kotlinx.coroutines.flow.MutableSharedFlow
 import kotlinx.coroutines.flow.MutableStateFlow
 import kotlinx.coroutines.flow.asStateFlow
 import kotlinx.coroutines.flow.catch
@@ -16,6 +19,8 @@ import kotlinx.coroutines.flow.consumeAsFlow
 import kotlinx.coroutines.flow.launchIn
 import kotlinx.coroutines.flow.onCompletion
 import kotlinx.coroutines.flow.onEach
+import kotlinx.coroutines.flow.onStart
+import kotlinx.coroutines.flow.sample
 import kotlinx.coroutines.flow.update
 import kotlinx.coroutines.launch
 import kotlinx.coroutines.sync.Semaphore
@@ -37,14 +42,33 @@ import java.util.Date
 import java.util.concurrent.ConcurrentHashMap
 import kotlin.math.absoluteValue
 import kotlin.time.Duration.Companion.hours
+import kotlin.time.Duration.Companion.seconds
 
+@OptIn(FlowPreview::class)
 class Updater : IUpdater {
     private val logger = KotlinLogging.logger {}
+    private val notifyFlowScope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
     private val scope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
 
-    private val _status = MutableStateFlow(UpdateStatus())
+    private val notifyFlow = MutableSharedFlow<Unit>(extraBufferCapacity = 1, onBufferOverflow = BufferOverflow.DROP_OLDEST)
-    override val status = _status.asStateFlow()
 
+    private val statusFlow = MutableSharedFlow<UpdateStatus>()
+    override val status = statusFlow.onStart { emit(getStatus()) }
+
+    init {
+        // has to be in its own scope (notifyFlowScope), otherwise, the collection gets canceled due to canceling the scopes (scope) children in the reset function
+        notifyFlowScope.launch {
+            notifyFlow.sample(1.seconds).collect {
+                updateStatus(immediate = true)
+            }
+        }
+    }
+
+    private val _status = MutableStateFlow(UpdateStatus())
+    override val statusDeprecated = _status.asStateFlow()
+
+    private var updateStatusCategories: Map<CategoryUpdateStatus, List<CategoryDataClass>> = emptyMap()
+    private var updateStatusSkippedMangas: List<MangaDataClass> = emptyList()
     private val tracker = ConcurrentHashMap<Int, UpdateJob>()
     private val updateChannels = ConcurrentHashMap<String, Channel<UpdateJob>>()
 
@@ -87,7 +111,7 @@ class Updater : IUpdater {
         val lastAutomatedUpdate = preferences.getLong(lastAutomatedUpdateKey, 0)
         preferences.edit().putLong(lastAutomatedUpdateKey, System.currentTimeMillis()).apply()
 
-        if (status.value.running) {
+        if (getStatus().running) {
             logger.debug { "Global update is already in progress" }
             return
         }
@@ -123,23 +147,33 @@ class Updater : IUpdater {
         HAScheduler.schedule(::autoUpdateTask, updateInterval, timeToNextExecution, "global-update")
     }
 
-    /**
+    private fun getStatus(running: Boolean? = null): UpdateStatus {
-     * Updates the status and sustains the "skippedMangas"
+        val jobs = tracker.values.toList()
-     */
-    private fun updateStatus(
-        jobs: List<UpdateJob>,
-        running: Boolean? = null,
-        categories: Map<CategoryUpdateStatus, List<CategoryDataClass>>? = null,
-        skippedMangas: List<MangaDataClass>? = null,
-    ) {
         val isRunning =
             running
                 ?: jobs.any { job ->
                     job.status == JobStatus.PENDING || job.status == JobStatus.RUNNING
                 }
-        val updateStatusCategories = categories ?: _status.value.categoryStatusMap
+        return UpdateStatus(this.updateStatusCategories, jobs, this.updateStatusSkippedMangas, isRunning)
-        val tmpSkippedMangas = skippedMangas ?: _status.value.mangaStatusMap[JobStatus.SKIPPED] ?: emptyList()
+    }
-        _status.update { UpdateStatus(updateStatusCategories, jobs, tmpSkippedMangas, isRunning) }
+
+    /**
+     * Pass "isRunning" to force a specific running state
+     */
+    private suspend fun updateStatus(
+        immediate: Boolean = false,
+        isRunning: Boolean? = null,
+    ) {
+        if (immediate) {
+            val status = getStatus(running = isRunning)
+
+            statusFlow.emit(status)
+            _status.update { status }
+
+            return
+        }
+
+        notifyFlow.emit(Unit)
     }
 
     private fun getOrCreateUpdateChannelFor(source: String): Channel<UpdateJob> {
@@ -166,7 +200,7 @@ class Updater : IUpdater {
         return channel
     }
 
-    private fun handleChannelUpdateFailure(source: String) {
+    private suspend fun handleChannelUpdateFailure(source: String) {
         val isFailedSourceUpdate = { job: UpdateJob ->
             val isForSource = job.manga.sourceId == source
             val hasFailed = job.status == JobStatus.FAILED
@@ -181,17 +215,12 @@ class Updater : IUpdater {
                 tracker[mangaId] = job.copy(status = JobStatus.FAILED)
             }
 
-        updateStatus(
+        updateStatus()
-            tracker.values.toList(),
-            tracker.any { (_, job) ->
-                job.status == JobStatus.PENDING || job.status == JobStatus.RUNNING
-            },
-        )
     }
 
     private suspend fun process(job: UpdateJob) {
         tracker[job.manga.id] = job.copy(status = JobStatus.RUNNING)
-        updateStatus(tracker.values.toList(), true)
+        updateStatus()
 
         tracker[job.manga.id] =
             try {
@@ -207,7 +236,15 @@ class Updater : IUpdater {
                 job.copy(status = JobStatus.FAILED)
             }
 
-        updateStatus(tracker.values.toList())
+        val wasLastJob = tracker.values.none { it.status == JobStatus.PENDING || it.status == JobStatus.RUNNING }
+
+        // in case this is the last update job, the running flag has to be true, before it gets set to false, to be able
+        // to properly clear the dataloader store in UpdateType
+        updateStatus(immediate = wasLastJob, isRunning = true)
+
+        if (wasLastJob) {
+            updateStatus(isRunning = false)
+        }
     }
 
     override fun addCategoriesToUpdateQueue(
@@ -274,10 +311,15 @@ class Updater : IUpdater {
                 .toList()
         val skippedMangas = categoriesToUpdateMangas.subtract(mangasToUpdate.toSet()).toList()
 
-        // In case no manga gets updated and no update job was running before, the client would never receive an info about its update request
+        this.updateStatusCategories = updateStatusCategories
-        updateStatus(emptyList(), mangasToUpdate.isNotEmpty(), updateStatusCategories, skippedMangas)
+        this.updateStatusSkippedMangas = skippedMangas
 
         if (mangasToUpdate.isEmpty()) {
+            // In case no manga gets updated and no update job was running before, the client would never receive an info
+            // about its update request
+            scope.launch {
+                updateStatus(immediate = true)
+            }
             return
         }
 
@@ -288,8 +330,9 @@ class Updater : IUpdater {
     }
 
     private fun addMangasToQueue(mangasToUpdate: List<MangaDataClass>) {
+        // create all manga update jobs before adding them to the queue so that the client is able to calculate the
+        // progress properly right form the start
         mangasToUpdate.forEach { tracker[it.id] = UpdateJob(it) }
-        updateStatus(tracker.values.toList(), mangasToUpdate.isNotEmpty())
         mangasToUpdate.forEach { addMangaToQueue(it) }
     }
 
@@ -303,7 +346,11 @@ class Updater : IUpdater {
     override fun reset() {
         scope.coroutineContext.cancelChildren()
         tracker.clear()
-        updateStatus(emptyList(), false)
+        this.updateStatusCategories = emptyMap()
+        this.updateStatusSkippedMangas = emptyList()
+        scope.launch {
+            updateStatus(immediate = true, isRunning = false)
+        }
         updateChannels.forEach { (_, channel) -> channel.cancel() }
         updateChannels.clear()
     }

server/src/main/kotlin/suwayomi/tachidesk/manga/impl/update/UpdaterSocket.kt

@@ -23,12 +23,12 @@ object UpdaterSocket : Websocket<UpdateStatus>() {
         ctx: WsContext,
         value: UpdateStatus?,
     ) {
-        ctx.send(value ?: updater.status.value)
+        ctx.send(value ?: updater.statusDeprecated.value)
     }
 
     override fun handleRequest(ctx: WsMessageContext) {
         when (ctx.message()) {
-            "STATUS" -> notifyClient(ctx, updater.status.value)
+            "STATUS" -> notifyClient(ctx, updater.statusDeprecated.value)
             else ->
                 ctx.send(
                     """

server/src/main/kotlin/suwayomi/tachidesk/server/ServerConfig.kt

@@ -101,6 +101,7 @@ class ServerConfig(getConfig: () -> Config, val moduleName: String = SERVER_CONF
     val autoDownloadNewChapters: MutableStateFlow<Boolean> by OverrideConfigValue(BooleanConfigAdapter)
     val excludeEntryWithUnreadChapters: MutableStateFlow<Boolean> by OverrideConfigValue(BooleanConfigAdapter)
     val autoDownloadNewChaptersLimit: MutableStateFlow<Int> by OverrideConfigValue(IntConfigAdapter)
+    val autoDownloadIgnoreReUploads: MutableStateFlow<Boolean> by OverrideConfigValue(BooleanConfigAdapter)
 
     // extensions
     val extensionRepos: MutableStateFlow<List<String>> by OverrideConfigValues(StringConfigAdapter)

server/src/main/kotlin/suwayomi/tachidesk/server/ServerSetup.kt

@@ -218,7 +218,7 @@ fun applicationSetup() {
             }
         } catch (e: Throwable) {
             // cover both java.lang.Exception and java.lang.Error
-            e.printStackTrace()
+            logger.error(e) { "Failed to create/remove SystemTray due to" }
         }
     }, ignoreInitialValue = false)
 

server/src/main/kotlin/suwayomi/tachidesk/server/database/migration/M0037_RemoveTrackRecordsOfUnsupportedTrackers.kt

@@ -0,0 +1,19 @@
+package suwayomi.tachidesk.server.database.migration
+
+/*
+ * Copyright (C) Contributors to the Suwayomi project
+ *
+ * This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
+
+import de.neonew.exposed.migrations.helpers.SQLMigration
+import suwayomi.tachidesk.manga.impl.track.tracker.TrackerManager
+
+@Suppress("ClassName", "unused")
+class M0037_RemoveTrackRecordsOfUnsupportedTrackers : SQLMigration() {
+    override val sql: String =
+        """
+        DELETE FROM TRACKRECORD WHERE SYNC_ID NOT IN (${TrackerManager.MYANIMELIST}, ${TrackerManager.ANILIST}, ${TrackerManager.MANGA_UPDATES})
+        """.trimIndent()
+}

server/src/main/kotlin/suwayomi/tachidesk/server/database/migration/M0038_RemoveTrackRecordsOfUnsupportedTrackersII.kt

@@ -0,0 +1,19 @@
+package suwayomi.tachidesk.server.database.migration
+
+/*
+ * Copyright (C) Contributors to the Suwayomi project
+ *
+ * This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
+
+import de.neonew.exposed.migrations.helpers.SQLMigration
+import suwayomi.tachidesk.manga.impl.track.tracker.TrackerManager
+
+@Suppress("ClassName", "unused")
+class M0038_RemoveTrackRecordsOfUnsupportedTrackersII : SQLMigration() {
+    override val sql: String =
+        """
+        DELETE FROM TRACKRECORD WHERE SYNC_ID NOT IN (${TrackerManager.MYANIMELIST}, ${TrackerManager.ANILIST}, ${TrackerManager.MANGA_UPDATES})
+        """.trimIndent()
+}

server/src/main/kotlin/suwayomi/tachidesk/server/util/Browser.kt

@@ -8,9 +8,11 @@ package suwayomi.tachidesk.server.util
  * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
 
 import dorkbox.desktop.Desktop
+import mu.KotlinLogging
 import suwayomi.tachidesk.server.serverConfig
 
 object Browser {
+    private val logger = KotlinLogging.logger { }
     private val electronInstances = mutableListOf<Any>()
 
     private fun getAppBaseUrl(): String {
@@ -28,14 +30,14 @@ object Browser {
                     electronInstances.add(ProcessBuilder(electronPath, appBaseUrl).start())
                 } catch (e: Throwable) {
                     // cover both java.lang.Exception and java.lang.Error
-                    e.printStackTrace()
+                    logger.error(e) { "openInBrowser: failed to launch electron due to" }
                 }
             } else {
                 try {
                     Desktop.browseURL(appBaseUrl)
                 } catch (e: Throwable) {
                     // cover both java.lang.Exception and java.lang.Error
-                    e.printStackTrace()
+                    logger.error(e) { "openInBrowser: failed to launch browser due to" }
                 }
             }
         }

server/src/main/kotlin/suwayomi/tachidesk/server/util/SystemTray.kt

@@ -10,6 +10,7 @@ package suwayomi.tachidesk.server.util
 import dorkbox.systemTray.MenuItem
 import dorkbox.systemTray.SystemTray
 import dorkbox.util.CacheUtil
+import mu.KotlinLogging
 import suwayomi.tachidesk.server.ServerConfig
 import suwayomi.tachidesk.server.generated.BuildConfig
 import suwayomi.tachidesk.server.serverConfig
@@ -17,6 +18,7 @@ import suwayomi.tachidesk.server.util.Browser.openInBrowser
 import suwayomi.tachidesk.server.util.ExitCode.Success
 
 object SystemTray {
+    private val logger = KotlinLogging.logger { }
     private var instance: SystemTray? = null
 
     fun create() {
@@ -60,7 +62,7 @@ object SystemTray {
 
                 systemTray
             } catch (e: Exception) {
-                e.printStackTrace()
+                logger.error(e) { "create: failed to create SystemTray due to" }
                 null
             }
     }

server/src/main/kotlin/suwayomi/tachidesk/server/util/WebInterfaceManager.kt

@@ -136,6 +136,8 @@ enum class WebUIFlavor(
     }
 }
 
+fun WebUIFlavor.isDefault(): Boolean = this == WebUIFlavor.default
+
 object WebInterfaceManager {
     private val logger = KotlinLogging.logger {}
     private val scope = CoroutineScope(SupervisorJob() + Dispatchers.Default)
@@ -314,10 +316,7 @@ object WebInterfaceManager {
             // check if the bundled webUI version is a newer version than the current used version
             // this could be the case in case no compatible webUI version is available and a newer server version was installed
             val shouldUpdateToBundledVersion =
-                flavor.uiName == WebUIFlavor.default.uiName && extractVersion(getLocalVersion()) <
+                flavor.isDefault() && extractVersion(getLocalVersion()) < extractVersion(BuildConfig.WEBUI_TAG)
-                    extractVersion(
-                        BuildConfig.WEBUI_TAG,
-                    )
             if (shouldUpdateToBundledVersion) {
                 log.debug { "update to bundled version \"${BuildConfig.WEBUI_TAG}\"" }
 
@@ -375,7 +374,7 @@ object WebInterfaceManager {
             return
         }
 
-        if (flavor.uiName != WebUIFlavor.default.uiName) {
+        if (!flavor.isDefault()) {
             log.warn { "fallback to default webUI \"${WebUIFlavor.default.uiName}\"" }
 
             serverConfig.webUIFlavor.value = WebUIFlavor.default.uiName
@@ -597,24 +596,25 @@ object WebInterfaceManager {
                     ?: throw Exception("Invalid mappingFile")
             val minServerVersionNumber = extractVersion(minServerVersionString)
 
+            // is a STABLE webUI release, without a specified webUI version, which requires same handling as the PREVIEW release
+            val isUnknownStableVersion = webUIVersion == "STABLEPREVIEW"
+
             if (!WebUIChannel.doesConfigChannelEqual(WebUIChannel.from(webUIVersion))) {
                 // allow only STABLE versions for STABLE channel
-                if (WebUIChannel.doesConfigChannelEqual(WebUIChannel.STABLE)) {
+                if (WebUIChannel.doesConfigChannelEqual(WebUIChannel.STABLE) && !isUnknownStableVersion) {
                     continue
                 }
 
                 // allow all versions for PREVIEW channel
             }
 
-            if (webUIVersion == WebUIChannel.PREVIEW.name) {
+            if (webUIVersion == WebUIChannel.PREVIEW.name || isUnknownStableVersion) {
                 webUIVersion = fetchPreviewVersion(flavor)
             }
 
-            val isCompatibleVersion =
+            val isNewerThanBundled =
-                minServerVersionNumber <= currentServerVersionNumber && minServerVersionNumber >=
+                !flavor.isDefault() || extractVersion(webUIVersion) >= extractVersion(BuildConfig.WEBUI_TAG)
-                    extractVersion(
+            val isCompatibleVersion = minServerVersionNumber <= currentServerVersionNumber && isNewerThanBundled
-                        BuildConfig.WEBUI_TAG,
-                    )
             if (isCompatibleVersion) {
                 return webUIVersion
             }

server/src/main/resources/server-reference.conf

@@ -25,6 +25,7 @@ server.downloadsPath = ""
 server.autoDownloadNewChapters = false # if new chapters that have been retrieved should get automatically downloaded
 server.excludeEntryWithUnreadChapters = true # ignore automatic chapter downloads of entries with unread chapters
 server.autoDownloadNewChaptersLimit = 0 # 0 to disable it - how many unread downloaded chapters should be available - if the limit is reached, new chapters won't be downloaded automatically. this limit will also be applied to the auto download of new chapters on an update
+server.autoDownloadIgnoreReUploads = false # decides if re-uploads should be ignored during auto download of new chapters
 
 # extension repos
 server.extensionRepos = [

server/src/test/resources/server-reference.conf

@@ -10,40 +10,60 @@ server.socksProxyPort = ""
 server.socksProxyUsername = ""
 server.socksProxyPassword = ""
 
+# webUI
+server.webUIEnabled = true
+server.webUIFlavor = "WebUI" # "WebUI", "VUI" or "Custom"
+server.initialOpenInBrowserEnabled = true
+server.webUIInterface = "browser" # "browser" or "electron"
+server.electronPath = ""
+server.webUIChannel = "stable" # "bundled" (the version bundled with the server release), "stable" or "preview" - the webUI version that should be used
+server.webUIUpdateCheckInterval = 23 # time in hours - 0 to disable auto update - range: 1 <= n < 24 - default 23 hours - how often the server should check for webUI updates
+
 # downloader
 server.downloadAsCbz = false
-server.autoDownloadNewChapters = false
+server.downloadsPath = ""
-server.excludeEntryWithUnreadChapters = true
+server.autoDownloadNewChapters = false # if new chapters that have been retrieved should get automatically downloaded
-server.autoDownloadNewChaptersLimit = 0
+server.excludeEntryWithUnreadChapters = true # ignore automatic chapter downloads of entries with unread chapters
+server.autoDownloadNewChaptersLimit = 0 # 0 to disable it - how many unread downloaded chapters should be available - if the limit is reached, new chapters won't be downloaded automatically. this limit will also be applied to the auto download of new chapters on an update
+server.autoDownloadIgnoreReUploads = false # decides if re-uploads should be ignored during auto download of new chapters
+
+# extension repos
+server.extensionRepos = [
+  # an example: https://github.com/MY_ACCOUNT/MY_REPO/tree/repo
+]
 
 # requests
-server.maxSourcesInParallel = 10
+server.maxSourcesInParallel = 6 # range: 1 <= n <= 20 - default: 6 - sets how many sources can do requests (updates, downloads) in parallel. updates/downloads are grouped by source and all mangas of a source are updated/downloaded synchronously
 
 # updater
 server.excludeUnreadChapters = true
 server.excludeNotStarted = true
 server.excludeCompleted = true
-server.globalUpdateInterval = 12
+server.globalUpdateInterval = 12 # time in hours - 0 to disable it - (doesn't have to be full hours e.g. 12.5) - range: 6 <= n < ∞ - default: 12 hours - interval in which the global update will be automatically triggered
-server.updateMangas = false
+server.updateMangas = false # if the mangas should be updated along with the chapter list during a library/category update
+
+# Authentication
+server.basicAuthEnabled = false
+server.basicAuthUsername = ""
+server.basicAuthPassword = ""
 
 # misc
-server.debugLogsEnabled = true
+server.debugLogsEnabled = false
-server.gqlDebugLogsEnabled = false
+server.gqlDebugLogsEnabled = false # this includes logs with non privacy safe information
-server.systemTrayEnabled = false
+server.systemTrayEnabled = true
-
-# webUI
-server.webUIEnabled = true
-server.initialOpenInBrowserEnabled = true
-server.webUIInterface = "browser" # "browser" or "electron"
-server.electronPath = ""
-server.webUIChannel = "stable"
-server.webUIUpdateCheckInterval = 24
 
 # backup
 server.backupPath = ""
-server.backupTime = "00:00"
+server.backupTime = "00:00" # range: hour: 0-23, minute: 0-59 - default: "00:00" - time of day at which the automated backup should be triggered
-server.backupInterval = 1
+server.backupInterval = 1 # time in days - 0 to disable it - range: 1 <= n < ∞ - default: 1 day - interval in which the server will automatically create a backup
-server.backupTTL = 14
+server.backupTTL = 14 # time in days - 0 to disable it - range: 1 <= n < ∞ - default: 14 days - how long backup files will be kept before they will get deleted
 
 # local source
 server.localSourcePath = ""
+
+# Cloudflare bypass
+server.flareSolverrEnabled = false
+server.flareSolverrUrl = "http://localhost:8191"
+server.flareSolverrTimeout = 60 # time in seconds
+server.flareSolverrSessionName = "suwayomi"
+server.flareSolverrSessionTtl = 15 # time in minutes