Control preheating scheduler concurrency (#829)

* Control preheating scheduler concurrency

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Discard stale preheating queue entries

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Add timeouts to preheating tasks

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Inline default preheating jitter

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Prevent preheating queue backpressure leaks

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Cover preheating queue ordering and cleanup

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Add preheating scheduler lifecycle

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Make scheduler shutdown nonblocking

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Order scheduler shutdown before observability

Co-authored-by: Colin <Colin_XKL@outlook.com>

* Document advanced preheating settings

Co-authored-by: Colin <Colin_XKL@outlook.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

Author: Colin-XKL

cmd/main.go

@@ -30,10 +30,10 @@ var (
 func init() {
 	logrus.Info("Preheating scheduler starting...")
 	// Set up the preheating task function to use the new, encapsulated logic.
-	taskFunc := func(recipeName string) error {
+	taskFunc := func(ctx context.Context, recipeName string) error {
 		// The second return value (*feeds.Feed) is ignored as we only care about
 		// the side effect of caching, which happens inside ProcessRecipeByID.
-		_, err := recipe.ProcessRecipeByIDWithTrigger(context.Background(), recipeName, observability.TriggerPreheating)
+		_, err := recipe.ProcessRecipeByIDWithTrigger(ctx, recipeName, observability.TriggerPreheating)
 		return err
 	}
 	shouldRun := func(recipeName string) bool {
@@ -150,6 +150,10 @@ func startServer() {
 	dao.MigrateDatabases()
 	observability.Init(util.GetDatabase())
 	defer observability.Shutdown()
+	defer func() {
+		recipe.Scheduler.Close()
+		recipe.Scheduler.Wait()
+	}()
 	logrus.Info("Database migration done.")
 
 	listenAddr := os.Getenv("LISTEN_ADDR")

doc-site/src/content/docs/en/guides/advanced/customization.md

@@ -75,6 +75,9 @@ You can configure FeedCraft using environment variables in `docker-compose.yml`.
 - **FC_LLM_API_TYPE**: (Optional) `openai` (default) or `ollama`.
 - **FC_LLM_MAX_CONCURRENCY**: (Optional) Global maximum concurrency for LLM requests (default: `3`). Limits concurrent API calls to prevent rate limits.
 - **FC_DOMAIN_MAX_CONCURRENCY**: (Optional) Maximum concurrent requests per target domain during web scraping like fulltext extraction (default: `3`). Prevents overwhelming target servers.
+- **FC_PREHEATING_MAX_CONCURRENCY**: (Optional) Maximum concurrent background preheating tasks (default: `2`).
+- **FC_PREHEATING_QUEUE_SIZE**: (Optional) Maximum pending preheating tasks; defaults to the preheating concurrency.
+- **FC_PREHEATING_TASK_TIMEOUT**: (Optional) Timeout for each preheating task as a Go duration such as `5m` (default: `10m`).
 - **LOG_LEVEL**: (Optional) Log level for the backend application (e.g., `info`, `debug`, `trace`). Overrides the default level set by `ENV`.
 
 ### External Services

doc-site/src/content/docs/zh-tw/guides/advanced/customization.md

@@ -75,6 +75,9 @@ sidebar:
 - **FC_LLM_API_TYPE**: (可選) `openai` (預設) 或 `ollama`.
 - **FC_LLM_MAX_CONCURRENCY**: (可選) 全局最大 LLM 併發請求數（預設: `3`）。用於限制併發請求數量以防止觸發 API 速率限制。
 - **FC_DOMAIN_MAX_CONCURRENCY**: (可選) 網頁抓取（如全文提取）時每個目標域名的最大併發數（預設: `3`）。防止抓取目標伺服器負載過高。
+- **FC_PREHEATING_MAX_CONCURRENCY**: （可選）背景預熱工作的最大併發數（預設：`2`）。
+- **FC_PREHEATING_QUEUE_SIZE**: （可選）預熱等候佇列的最大工作數；預設與預熱併發數相同。
+- **FC_PREHEATING_TASK_TIMEOUT**: （可選）單一預熱工作的逾時時間，使用 `5m` 等 Go duration 格式（預設：`10m`）。
 - **LOG_LEVEL**: (可選) 後端應用的日誌級別 (例如 `info`, `debug`, `trace`)。覆蓋 `ENV` 設定的預設級別。
 
 ### 外部服務

doc-site/src/content/docs/zh/guides/advanced/customization.md

@@ -75,6 +75,9 @@ sidebar:
 - **FC_LLM_API_TYPE**: (可选) `openai` (默认) 或 `ollama`.
 - **FC_LLM_MAX_CONCURRENCY**: (可选) 全局最大 LLM 并发请求数（默认: `3`）。用于限制并发请求数量以防止触发 API 速率限制。
 - **FC_DOMAIN_MAX_CONCURRENCY**: (可选) 网页抓取（如全文提取）时每个目标域名的最大并发数（默认: `3`）。防止抓取目标服务器负载过高。
+- **FC_PREHEATING_MAX_CONCURRENCY**: （可选）后台预热任务的最大并发数（默认：`2`）。
+- **FC_PREHEATING_QUEUE_SIZE**: （可选）预热等待队列的最大任务数；默认与预热并发数相同。
+- **FC_PREHEATING_TASK_TIMEOUT**: （可选）单个预热任务的超时时间，使用 `5m` 等 Go duration 格式（默认：`10m`）。
 - **LOG_LEVEL**: (可选) 后端应用的日志级别 (例如 `info`, `debug`, `trace`)。覆盖 `ENV` 设置的默认级别。
 
 ### 外部服务