Running PutsReq in production through Docker

[amotl]

Dear Pablo,

first things first: Thank you so much for conceiving and maintaining PutsReq. @thiasB started using it and now we would to ramp up a dedicated instance for our beehive monitoring project, see https://hiveeyes.org/ and https://community.hiveeyes.org/.

We need it to transform HTTP requests originating from TTN to converge properly to our data acquisition backend running Kotori. @thiasB documented this specific telemetry path at TTN/LoRaWAN setup for Terkin, enjoy!

We have been able to get started with it already but humbly ask for further guidance to get things right.

With kind regards,
Andreas.

[amotl]

1. We started by building a Docker image and pushed it to https://hub.docker.com/u/daqzilla.

# Build
git clone https://github.com/phstc/putsreq
cd putsreq
docker build --tag daqzilla/putsreq:latest .

# Upload
docker login
docker push daqzilla/putsreq

2. Then, we amended the docker-compose.yml file like that to use that image.

version: '3'
services:
  db:
    image: mongo:3.6.17
    tty: true
    stdin_open: true
    volumes:
      - ./data:/data
  redis:
    image: redis:alpine
  app:
    image: daqzilla/putsreq
    tty: true
    stdin_open: true
    command: /bin/sh -c "rm -f /app/tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
    ports:
      - '5050:3000'
    env_file:
      - .env.docker
    depends_on:
      - db
      - redis

Please note we had to bump the MongoDB image version as the given thing just crashed silently on our Debian GNU/Linux 10 (buster) machine. Also note that we adjusted the database volume definition because on the updated Docker image for MongoDB 3.6.17, there has apparently been no /var/lib/mongo/data directory.

3. Now, some links on the frontend still point to localhost and we don't see any data from MongoDB under ./data yet. There are just two empty directories:

l data/*
data/configdb:
total 0
drwxr-xr-x 2 root root  6 Apr 29 18:26 ./
drwxr-xr-x 4 root root 32 Apr 29 18:26 ../

data/db:
total 0
drwxr-xr-x 2 root root  6 Apr 29 18:26 ./
drwxr-xr-x 4 root root 32 Apr 29 18:26 ../

[amotl]

Dear Pablo,

we see that "localhost" is referenced at different places within the source tree. Most probably because the Docker-based setup is primarily suited for development setups, right?

While we have seen webpacker.yml already, our knowledge is a bit limited on how to tune that or other files in order to compose an appropriate Docker image suitable to be run in production on a dedicated, full qualified hostname.

So, as mentioned above, we are humbly asking if you could support us to properly saddle up the pony.

Thanks already for taking the time and with kind regards,
Andreas.

[amotl]

We have been able to make it work.

Configuration

We are now using this docker-compose.yml:

version: '3.6'
services:
  db:
    image: mongo:3.6.17
    tty: true
    stdin_open: true
    volumes:
      - data:/data/db
  redis:
    image: redis:alpine
  app:
    image: daqzilla/putsreq
    tty: true
    stdin_open: true
    command: /bin/sh -c "rm -f /app/tmp/pids/server.pid && bundle exec rails server -p 3000 -b '0.0.0.0'"
    ports:
      - '5050:3000'
    env_file:
      - .env.docker
    depends_on:
      - db
      - redis
volumes:
  data:
    external:
      name: putsreq_mongodb

as well as this .env.docker:

RAILS_ENV=production
MONGOLAB_URI=mongodb://db
REDIS_URL=redis://redis
DEVISE_SECRET_KEY=123
SECRET_TOKEN=123

External Docker volume FTW

The referenced external volume has been created by invoking:

docker volume create --name=putsreq_mongodb

The rationale for creating the external volume is https://stackoverflow.com/questions/53870416/data-does-not-persist-to-host-volume-with-docker-compose-yml-for-mongodb. Otherwise, data stored in MongoDB might get lost as already observed by @ddavtian within #51.

Ready-made Docker image on Docker Hub

Additionally, the Docker image on https://hub.docker.com/r/daqzilla/putsreq has been amended using the patch putsreq-production.patch.txt. While we recognize it is not the most performant way to compile and serve assets like that on a production instance (precompiling and serving them from a webserver in a static manner should be preferred), it perfectly fits our bill.

Reverse-proxy configuration for Nginx

Last but not least, the Nginx snippet we use as a reverse proxy configuration is:

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name putsreq.example.org;

    #include snippets/snakeoil.conf;
    include snippets/ssl/putsreq.example.org;

    proxy_buffering off;

    location / {
        proxy_set_header        Host               $http_host;
        proxy_set_header        X-Real-IP          $remote_addr;
        proxy_set_header        X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto  $scheme;
        proxy_pass              http://localhost:5050/;
    }

}

[phstc]

Hi @amotl

Thank you. I'm happy you find it useful.

The production version of PutsReq is deployed to Heroku, which makes it very straightforward to deploy Rails application.

I've used the Docker setup mostly for local development. The localhost references in the webpack.yml, shouldn't affect your production environment, it is used for the local dev server only.

Additionally, the Docker image on https://hub.docker.com/r/daqzilla/putsreq has been amended using this patch: putsreq-production.patch.txt

Nice, is it all set then?

Would you like to add some instructions to the wiki or update the README with some references to share with others?

[amotl]

The localhost references in the webpack.yml shouldn't affect your production environment.

That's true. The reason for that was a wrong Nginx configuration setting: proxy_set_header Host was assigned to $http_x_host - don't know where I got this from.

Nice, is it all set then?

Absolutely. Thanks again!

Would you like to add some instructions to the wiki or update the README with some references to share with others?

That would be cool. I definitively prefer files over the Wiki and added a PR, see #70.

[phstc]

@amotl awesome, thanks for #70. Does that close this issue then?

[amotl]

Sure, let's close it. Thanks again!