Skip to content

11.0.0
Compare
Choose a tag to compare
@orangecoding orangecoding released this 17 Dec 12:18
· 12 commits to master since this release
11.0.0
3b8279c

The most significant update to Fredy yet 🎉

Until now, Fredy used an XHR-based approach to extract information from various providers. For each run, it sent a GET request to gather the required data. However, this approach had two major drawbacks:

  • It relied on a library that is no longer maintained.
  • With the growing prevalence of SPAs, XHR requests could no longer retrieve the necessary source code, as it is loaded dynamically.

To address these issues, I decided to rewrite Fredy using Puppeteer (https://pptr.dev/). With Puppeteer, Fredy can control an actual browser and behave like a real user, allowing it to wait for dynamically loaded content.

At the same time, I removed ScrapingAnt. While ScrapingAnt was incredibly useful in the early stages, it has become more of a burden lately. It was originally intended to assist with pages like Immoscout that have advanced bot protection, but even ScrapingAnt is now frequently detected as a bot. Additionally, contacting support at ScrapingAnt has proven nearly impossible.

Assets 2
Loading