Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move from Azure services to self-hosted Docker containers #4

Open
steinbro opened this issue Dec 6, 2023 · 2 comments
Open

Move from Azure services to self-hosted Docker containers #4

steinbro opened this issue Dec 6, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@steinbro
Copy link
Member

steinbro commented Dec 6, 2023

We'd like to take the authoring tool, which historically ran on Azure services at Microsoft, and instead run them on the same Docker-based server infrastructure we use for the Soundscape tile server. This exercise will involve translating the setup instructions in the README file into instructions Docker can understand, as well as replacing any Azure-specific functionality. (Some Azure services, like their Maps API, we probably want to keep -- this is more about managed infrastructure like databases and storage.)

Docker containers are basically lightweight virtual machines. A Dockerfile defines the set of commands to run to go from a fresh OS installation to a running server. A docker-compose.yml file describes a list of Docker containers, and how they should be configured to talk to one another. Both Docker and docker-compose are software you can install and run on your local machine for development.

The Docker files we have for the Soundscape tile server should be a good reference point to start out. For example, I expect the Dockerfile for the backend will look fairly similar to our Dockerfile.tilesrv, since it is taking a base OS image, installing system and Python packages, then running a server. There are other examples online for deploying a generic Node server, which we'll need for the frontend

Our docker-compose.yml file also has some similarities to what we'll need. It's spinning up a Postgres container, a Soundscape tile server container, and a Soundscape ingest server container. For the authoring tool I expect we'll wind up having containers like so:

  1. Database (just a postgres image)
  2. Blob storage (perhaps a Ceph image)
  3. Backend (from a new Dockerfile you create)
  4. Frontend (from a new Dockerfile you create) (edit: @sheldonej clarified the frontend is static, so this would just be an additional command to run when creating the backend image, no need for a separate container)
@steinbro steinbro added the enhancement New feature or request label Dec 6, 2023
@RDMurray
Copy link
Contributor

RDMurray commented Dec 6, 2023

I think Ceph is very heavy wait, and it is a lot more than object storage. other options are:

  • Garage
  • MinIO
  • one of the public clouds, avoiding the need to self-host.
  • Modify the backend to use the filesystem instead of an s3 compatible service.

@steinbro
Copy link
Member Author

steinbro commented Dec 6, 2023

Thanks for pointing that out -- I only know Ceph in the abstract, as having an S3-compatible API you can self-host. But, as Azure Blob Storage is itself not S3-compatible, Ceph wouldn't be a drop-in replacement, anyway. And, if we want to use something like Azure Blob Storage for backups and the like, maybe there's no need to swap it out at all.

I realize this task is mixing two separate goals: containerizing the authoring tool, and removing cloud dependencies. The first step is to just make Dockerfiles for the frontend and backend, and a docker-compose.yml file that ties them together. Once that's in place, we can then go through the individual underlying dependencies (Postgres, Blob Storage, authentication, etc.) and see what we can swap out.

@steinbro steinbro mentioned this issue Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants