403 error scraping xenforo2 forum

[ipkpjersi]

Here is the syntax I am using via command-line:

python3 '/home/ken/Desktop/xenforo-scrape/xenforo-scraper.py' https://rune-server.org/forums/informative-threads.656/ -c cookies.txt

This is the output I get:

<https://rune-server.org/forums/informative-threads.656/> Request Error: 403 - Forbidden
Getting pages from category.. Current: 1/1
<https://rune-server.org/forums/informative-threads.656/page-1> Request Error: 403 - Forbidden

Done!

What am I doing wrong?

[Jertzukka]

Your command works for me, which would indicate it is more likely cloudflare or similar issue (for example using a VPN which they find suspicious). But even if it worked, I don't think this scraper would work well on this forum as this is mainly a media scraper and your links are mostly text. Also the forum is very customized and quickly checking for example their external images are proxied which wouldn't download properly.

[ipkpjersi]

Ah, I was mostly interested in the text, thanks for letting me know.