Courlan does not load /page/ links

[sbusso]

In reference to the nav filter, courlan will not extract links containing /page/ path. Also, I think page and tag|category should be handled separately. I do need to get all blog posts on my website, which are paginated but I don't want to get tags and categories.

[adbar]

Hi @sbusso, I'm not sure what you mean regarding the /page/ pattern, maybe it's a documentation issue. I added tests, could you please look at the commit above and see if you can make the code work in your case or provide more details?

The separation of pagination from the rest makes sense, I'll think about how if could be implemented.

[sbusso]

URLs containing /page/1, /page/2 won't be extracted with extract_links without making with_nav=True, also this option will also include other index pages like tags and categories. I'd think page and maybe archives could be separated or extended options.

courlan/courlan/filters.py

Lines 49 to 52 in 02e1afe

	NAVIGATION_FILTER = re.compile(
	r"/(archives|auth?or|cat|category|kat|kategorie|page|schlagwort|seite|tags?|topics?|user)/|\?p=[0-9]+",
	re.IGNORECASE,
	)

[adbar]

I see, let's keep an eye on that.