Twitter-Scraper is a client-side userscript designed for efficiently scraping all replies from a specific Twitter user. The script automatically scrolls through the user's reply history, capturing and storing their posts. Once all replies have been collected, it prompts you to save the data as a text file for further analysis or archival purposes.
- ✅ Automated Scrolling & Capture – Continuously scrolls through replies and extracts all available data.
- ✅ Start/Stop Button – A visible button appears on the Twitter replies page, allowing you to control the script.
- ✅ Accurate Post Detection – Reliable scraping with high accuracy; adjustable speed to match internet/browser performance.
- ✅ Timestamp Option – Choose whether to include dates in the saved data for forensic analysis.
- ✅ Link Extraction – Collect all links posted by the user and save them as a separate file.
- ✅ Integration with included t.co Converter – Easily parse extracted links back to their original URL with the included Python script.
- 🔍 Forensic Data Collection – Gather historical replies for later analysis.
- 📜 Archiving & Research – Store Twitter data before it disappears or gets deleted.
- 🧠 LLM Data Parsing – Format scraped replies for training language models.
- 💾 Personal Storage – Keep a local backup of important discussions.
- Install the userscript in your browser via a userscript manager (e.g., Tampermonkey).
- Navigate to the Twitter replies page of the target user.
- Click the "Start" button to begin capturing replies.
- The script will scroll and extract posts automatically.
- Once all replies are gathered, you'll be prompted to save the data.
💡 Tip: To verify accuracy, test on an account with under 200 posts and compare against the official post count then adjust settings accordingly.
This project is licensed under the MIT License.
Contributions are welcome! If you have improvements or bug fixes, feel free to submit a pull request.
Let me know if you need further tweaks! 🚀