Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate Database Migrations #403

Merged
merged 2 commits into from
Mar 13, 2024
Merged

Conversation

nik-dange
Copy link
Member

Info

Closes [#226].

Description

What changes did you make? List all distinct problems that this PR addresses. Explain any relevant
motivation or context.

We need to find a way to automate database migrations when merging in changes that include any schema changes. This used to be done with Heroku, but such functionality doesn't come with Caprover.

Changes

  • Added new yarn release script to migrate and build/run the app
  • Uses the CI/CD changes I made a few months ago

Type of Change

  • Patch (non-breaking change/bugfix)
  • [] Minor (non-breaking change which adds functionality)
  • Major (fix or feature that would cause existing functionality to not work as
    expected)
  • Documentation (A change to a README/description)
  • Continuous Integration/DevOps Change (Related to deployment steps, continuous integration
    workflows, linting, etc.)
  • Other: (Fill In)

If you've selected Patch, Minor, or Major as your change type, make sure to bump the version before merging in package.json!

Testing

I have tested that my changes fully resolve the linked issue ...

  • locally.
  • on the testing API/testing database.
  • with appropriate Postman routes. Screenshots are included below.

Checklist

  • I have performed a self-review of my own code.
  • I have followed the style guidelines of this project.
  • I have appropriately edited the API version in the package.json file.
  • My changes produce no new warnings.

Screenshots

Please include a screenshot of your Postman testing passing successfully.

Copy link

Thanks for contributing!
If you've made changes to the API's functionality, please make sure to bump the package
version—see this guide to semantic versioning for details—and
document those changes as appropriate.

@nik-dange
Copy link
Member Author

nik-dange commented Feb 15, 2024

Summarizing some of the discussion we've had around this issue for visibility + learning:

The most important consideration here is atomicity. The database and API should always be in sync; there should never be a case when a more updated version of the API is pointing to an older version of the database, or vice versa. As such, any solution we come up with should guarantee that both entities are updated when merging to prod.

As a general overview of the deployment process, there are a few steps run on CircleCI (ie building the Docker image, linting, testing, etc.). Once the image is built, we deploy to Caprover. After deploying to Caprover, the migration runs, and the newest version of the app is open to connections + pointing to a now migrated database.

image

I believe this is our best solution–if for any reason tests fail, or the Image isn't built, the deployment process would end BEFORE any deployments to Caprover, which means the database would not unnecessarily migrate–everything needs to pass on CircleCI to move forward. We cannot guarantee the reverse with this solution though–if the migration fails, I'm pretty sure the app would still open connections (so now we'd have an updated API pointing to an older database), but I think this is fine–all deployments should be manually observed anyway, and with the introduction of a staging environment, we can validate migrations succeeding beforehand. A potential alternative would be to migrate on CircleCI, but if anything goes wrong on the Caprover side, our database would be unnecessarily migrated again.

Some more context in this thread: https://discord.com/channels/573028991527550986/1172988012234297384

Copy link
Contributor

@alexzhang1618 alexzhang1618 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm !!!

…into feature/automate-database-migrations

merging from master
@nik-dange nik-dange merged commit 5c1bd1b into master Mar 13, 2024
6 checks passed
@nik-dange nik-dange deleted the feature/automate-database-migrations branch March 31, 2024 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants