Mercury

A fast and efficient eBay webscraper implemented using Beautiful Soup

Installation

# Clone the repo
$ git clone https://github.com/PB020/Mercury.git

# Change the working directory
$ cd mercury

# Create virtual enviroment
$ python -m venv .env
$ .env/Scripts/activate # Windows
$ .env/bin/activate # Linux/MacOS

# Install requirements
$ pip install -U -r requirements.txt

Usage

usage: mercury.py [-h] [--url URL] [--file FILE] [--output OUTPUT]

A fast and efficient eBay product webscraper

optional arguments:
  -h, --help       show this help message and exit
  --url URL        URL of an eBay product ('https://www.ebay.com/itm/...')
  --file FILE      Path to a file containing ONLY eBay product URLs
  --output OUTPUT  CSV filename where parsed data should be saved

Note: If no URL or file is provided, the script will terminate.

Example

# Single URL
$ mercury.py --url "https://www.ebay.com/itm/..."

# File containing URLs
$ mercury.py --file "input.txt"

# Output to CSV
$ mercury.py --url "https://www.ebay.com/itm/..." --output "output.csv"

Output

ITEM_NUMBER,NAME,CONDITION,PRICE,URL
...,...,...,...,...

Dependencies

Web Scraping:

• lxml
• requests
• Beautiful Soup

CLI:

• argparse
• colorama

Multiprocessing:

• multiprocessing

License