DATUAN Sitemap Warmer

A CLI script built to help you warm up the website cache by scanning through the sitemap.xml. This utility help to warm up website in major encoding like brotli, gzip, warm up images by various encoding like AVIF, WebP.

---

Tài liệu Tiếng Việt tại đây - Vietnamese documentation here: https://datuan.dev/backlog/datuan-sitemap-warmer/

---

By default, this utility will warm up any URLs that changed in last 5 minutes (300s) by checking the value of <lastmod> tag in sitemap.xml files and any images inside URLs through <image:image> tag. If you want to test this plugin, try to update some posts or pages or using parameter --range with a higher value. See Options section for more details.

Quick start with Docker

docker run tdtgit/sitemap-warmer yourdomain.com

For more options and parameters, please refer to Options section.

Requirements

(In case you don't have Docker installation)

• NodeJS 12/14/16+
• Ubuntu/CentOS/*nix or Windows/MacOS
• A website has a sitemap.xml endpoint. Example: https://datuan.dev/sitemap.xml. Tested and worked well with plugins RankMath SEO, Yoast SEO, Jetpack.

How to use

Install NodeJS and install the utility by command below:

npm install -g datuan-sitemap-warmer

Then you can try to scan and warm up some sites using a simple command. Replace datuan.dev to your site's domain. Protocol (https://) is a default option and only specific if your site are still running http://

warmup datuan.dev
# OR
warmup http://nonhttps.com

You can (should) setup a cronjob to run your command automatically. The recommendation interval is every minute.

* * * * * warmup datuan.dev

For best practice, add another cronjob to warm up all URLs in sitemap in case any URL not warmed up yet. See more at Options section.

0 */12 * * *  warmup datuan.dev -a
* * * * *     warmup datuan.dev

You can also warm up multiple domains of course.

* * * * *     warmup http://domain1.com
*/2 * * * *   warmup domain2.net
*/5 * * * *   warmup domain3.xyz

Options

Usage:

warmup datuan.dev <URL> <parameter>

Parameter	Description	Default
-a, --all	Warm up all URLs in sitemap	False
-r, --range	Only warm up URLs with lastMod newer than X seconds. This parameters is ignored if -a (--all) is provided	300s (5 minutes)
-d, --delay	Delay (in milliseconds) between each warm up call. If you using the low-end hosting, keep this value higher	500
--no-images	Disable images warm up	False
--no-css	Disable CSS warm up	False
--no-js	Disable Javascript warm up	False
--no-brotli	Disable Brotli compression warm up	False
-q, --quite	Suppress the debug log	False
-h, --headers	Add custom headers	None

Advanced options

Custom request headers

warmup datuan.dev --headers.auth "Bearer super_secret" --headers.user-agent "My own crawler"

...

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Robert Michalski 💻

This project follows the all-contributors specification. Contributions of any kind welcome!