Update UI, and Update Sitemap tool to get proper thread count

2026-04-09 11:27:13 -07:00
parent 0e410a1f6c
commit 287566716f
5 changed files with 32 additions and 5 deletions
@@ -63,7 +63,7 @@ python3 sitemap_builder.py https://example.com --max-pages 20000 --delay 0.25 --
 - `--timeout`: request timeout in seconds
 - `--include-subdomains`: crawl subdomains of the starting host
 - `--include-documents`: include document links such as PDF, CSV, DOC, DOCX, XLSX, and similar files
- `--workers`: number of worker threads to use. Set `1` to disable multithreading
+- `--workers`: number of worker threads to use. Set `1` to disable multithreading. Default: all CPUs visible to the current machine or container
 - `--save-every`: save progress after every N pages. Default: `25`
 - `--resume`: resume from an existing state file
 - `--fresh`: ignore the existing state file and start over