Typescript script to scrape the Runescape wiki and get all current quests, along with their requirements from these two pages:
https://oldschool.runescape.wiki/w/Quests/List
https://oldschool.runescape.wiki/w/Quests/Skill_requirements
NodeJS and yarn/npm required.
To run:
- Clone the repo by executing
git clone https://github.com/Evaldash/oswiki-quest-scraper
- Enter the folder you cloned the repo to, and run
yarn install
- Run
yarn run dev
To test, run yarn test
Assuming all went well (and the oswiki layout didn't change...), quest list with requirements should be available in the quests.json
file.
TODO: determine which quests need to be scraped and get their additional (items, etc.) requirements from the individual quest pages, as to not accidentally "DDOS" the wiki by scraping ~150 quest pages every time.