From 4e51ac352736afc22868c243320d42351e5dcbec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fabian=20Gr=C3=BCnbichler?= Date: Mon, 2 Dec 2024 14:04:09 +0100 Subject: [PATCH] rest-server: handle failure in worker task setup correctly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit if setting up a new worker fails after it has been inserted into the WORKER_TASK_LIST, we need to clean it up instead of bubbling up the error right away, else we "leak" the worker task and it never finishes.. a worker task that never finishes will indefinitely block shutdown of the rest server process, including the "old" process when reloading the rest server. this issue was found in the wild on a system with lock contention on the file-based lock covering task index updating leading to lock acquiring timeouts. Signed-off-by: Fabian Grünbichler --- proxmox-rest-server/src/worker_task.rs | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/proxmox-rest-server/src/worker_task.rs b/proxmox-rest-server/src/worker_task.rs index 6e76c2ca..3ca93965 100644 --- a/proxmox-rest-server/src/worker_task.rs +++ b/proxmox-rest-server/src/worker_task.rs @@ -923,7 +923,12 @@ impl WorkerTask { set_worker_count(hash.len()); } - setup.update_active_workers(Some(&upid))?; + let res = setup.update_active_workers(Some(&upid)); + if res.is_err() { + // needed to undo the insertion into WORKER_TASK_LIST above + worker.log_result(&res); + res? + } Ok((worker, logger)) }