scraper.quiz01.BOT (python)

Python proyect designed to interact with and scrap data out of a specific quizzes web site. Build to be deployed on docker 👍🐳

• scraper.quiz01.BOT-API
• scraper.quiz01.BOT-CLI

Python script is builded with

• Playwright
• Selenium
• Beautiful Soup (bs4) => get it, read the docs.
• Requests (Py) => get it, read the docs.
• Swagger (python-client-generated)
• FastAPI (Run Interface 1)
• Typer (Run Interface 2)

It interacts with its own Serverless API on AWS

• AWS API => https://github.com/moisesJurad0/quiz01-scrapper.aws.serverless
  • AWS SAM
  • AWS Lambda
  • AWS Dynammo
  • AWS API Gateway
  • Boto3

Docker Image

This image is compatible with dotenv files.

Docker Image available on

1. Docker Image - Docker Hub

   1. https://hub.docker.com/repository/docker/squartle/scraper.quiz01.bot-api/general

      docker pull squartle/scraper.quiz01.bot-api:latest

   2. https://hub.docker.com/repository/docker/squartle/scrapper-1/general

      docker pull squartle/scrapper-1:latest

2. Docker Image - GitHub Packages

   1. https://github.com/moisesjurad0/quiz01-scrapper.py.selenium/pkgs/container/scrapper-1

      docker pull ghcr.io/moisesjurad0/scrapper-1:latest

Docker Image - Dependencies

Playwrigth

• playwright/python:v1.34.0-jammy

Selenium

• 3.11-alpine
• chromium
• chromium-chromedriver

Docker Image - Run

To run the Image follow this steps

# build the image
docker build -t <pick-a-name-for-image> .

# run the image
# --env-file => pass a file with environmental variables needed in the config.ini file
docker run -dt --env-file .env --name <pick-a-name-for-container> <pick-the-image-you-build-in-previous-step>

# get inside the container and execute as you need 
docker exec -it <pick-the-container-you-build-in-previous-step> sh

Or use compose

# on the proyect path
docker compose up

# also can rebuild it before running up
docker-compose up --build

# detached mode
docker-compose up --build -d

Docker Build

# docker tag my-image:original my-image:new-tag
docker tag squartle/scraper.quiz01.bot-api:latest squartle/scraper.quiz01.bot-api:selenium
docker build -t squartle/scraper.quiz01.bot-api:playwright .
docker tag squartle/scraper.quiz01.bot-api:playwright squartle/scraper.quiz01.bot-api:latest

# run
docker run --env-file .env -p 80:80  squartle/scraper.quiz01.bot-api