Skip to content

Latest commit

 

History

History
80 lines (57 loc) · 2.76 KB

README.md

File metadata and controls

80 lines (57 loc) · 2.76 KB
Raw

freecycle-twitter-bot

Build Status codecov License: MIT

About

A microservice to periodically scrape content from freecycle.org and broadcast new posts as Twitter tweets. See the service in action here.

Disclaimer: This is a prototype and not considered production ready.

Why

The freecycle.org site is outdated. In advance of any future improvements, it would be nice to provide the community with a more modern interface. A Twitter bot is one way to offer a pub-sub style mechanism for users while at the same time minimising operational overhead.

Architecture

The service architecture is broadly composed of three elements.

  1. A scraper which parses freecycle's public website using cheerio.
  2. A persistence layer which stores post history using a free managed PostgreSQL instance from Scaleway.
  3. A "tweeter" which publishes new posts to Twitter using the twit client.

The above functionality is wrapped up in a long running NodeJS process executed in a Docker container. Travis CI is used for CI tooling.

Development

To run in development:

  1. Clone the repo.
git clone https://github.com/hughmp/freecycle-twitter-bot.git
  1. Build the docker image.
docker build --name freecycle-twitter-bot .
  1. Set your configuration.
export POSTGRES_USERNAME='username'
export POSTGRES_PASSWORD='password'
export POSTGRES_DATABASE='database'
export POSTGRES_HOST='host'
export POSTGRES_PORT=1234
export TWITTER_CONSUMER_KEY='abcdefghijk'
export TWITTER_CONSUMER_SECRET='lmnopqrstuvw'
export TWITTER_ACCESS_TOKEN='abcdefghijk'
export TWITTER_ACCESS_SECRET='lmnopqrstuvw'
  1. Run the container.
docker run -d \
  --env POSTGRES_USERNAME \
  --env POSTGRES_PASSWORD \
  --env POSTGRES_DATABASE \
  --env POSTGRES_HOST \
  --env POSTGRES_PORT \
  --env TWITTER_CONSUMER_KEY \
  --env TWITTER_CONSUMER_SECRET \
  --env TWITTER_ACCESS_TOKEN \
  --env TWITTER_ACCESS_SECRET \
  freecycle-twitter-bot
  1. Profit!

TODO

  • Improve test coverage
  • Integrate continuous deployment
  • Cosider implementing lightweight DB in a Sidecar pattern
  • Remove secrets from build and publish public Docker image