From 12d09c50ffc3243549140ec9d365172ee6d44fa6 Mon Sep 17 00:00:00 2001 From: Pysis868 Date: Mon, 30 Apr 2018 00:31:38 -0400 Subject: [PATCH] Adding tool to help us craft migrations easily enough for this (currently :P) non-Rails project. Funny using bundler shows through dependencies that it brings in Rails packages anyway O_o, but I guess that's better than actually running them if you don't need them for the web site. We're going to see how this works without checking in (ignoring) the schema file for now. We already have some structure captured for development, and production data backups for that environment, but that file would be useful to keep to track which migrations have been run. Maybe no need to track in the code repository, and just add it to a backup routine. Other various doc updates with notes that were recently discussed or performed. --- .gitignore | 8 ++++- .standalone_migrations | 6 ++++ DEVELOPMENT.md | 53 ++++++++++++++++++++++++++- Gemfile | 8 +++++ Gemfile.lock | 76 +++++++++++++++++++++++++++++++++++++++ INSTALL.md | 45 ++++++++++++++++------- Rakefile | 2 ++ dev/db/config.yml.example | 14 ++++++++ 8 files changed, 198 insertions(+), 14 deletions(-) create mode 100644 .standalone_migrations create mode 100644 Gemfile create mode 100644 Gemfile.lock create mode 100644 Rakefile create mode 100644 dev/db/config.yml.example diff --git a/.gitignore b/.gitignore index 1bbdba95d..d584bca00 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,8 @@ # User-specific runtime files -# Environment File +# Environment Files .env ZM_nginx.conf +dev/db/config.yml # Cached Files cache/* @@ -17,3 +18,8 @@ vendor # Debug Files php_errors.log phpinfo.php + +# Erroneous Files + +# Part of the database migration framework we're using that could be useful, just not right now. +dev/db/schema.rb diff --git a/.standalone_migrations b/.standalone_migrations new file mode 100644 index 000000000..40ac80be8 --- /dev/null +++ b/.standalone_migrations @@ -0,0 +1,6 @@ +db: + migrate: dev/db/migrate + schema: dev/db/schema.rb + seeds: dev/db/seeds.rb +config: + database: dev/db/config.yml diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 3f27a28fa..c75c35e99 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -12,16 +12,30 @@ You can follow the Feature Work Process, only when making a branch, use this format instead: `issue/#-very-brief-descriptive-summary`, where `#` is the GitHub issue number, and you create your own summary specific to the issue you intend to work on. +# Mainline Re-synchronization + + Hopefully this should not be needed when following the other guidelines, but if you start to experience unique merge troubles for each mainline branch, or code was accidentally merged into a later stream too early, or like the hot-fixing process, you can merge the streams back into latest, but only in the direction. + These should always contain the same code, in that order, and is important to keep the streams' code healthy, and merges easy. + We don't need to pollute the streams with commits doing this often at all. Only when experiencing difficulty working, or when the infrequent operation will get the streams back to a clean state easily. + # Releases Done by Jason, from `master` to `production`, at the end of every 2-week sprint. +# Backup + + Not sure where to put this section for now. + + Live data, probably both the data and schema layout, are already being backed up somehow to somewhere by someone. + + To add to this, we may also want to backup the `dev/db/schema.rb` file so the system knows which migrations are needed to be run, if any. + # Development Guidelines - Follow the format of the code file you are working in. - We may introduce style checking later, and if so, it will only apply to modified/created code as you work in it. - Comment your code so other team members can understand it. - - We want to communicate why code is there, since the 'what' is already represented by the code statements themselves. + - We want to communicate why code is there, since the 'what' is already represented by the code statements themselves, and those are straightforward for the most part, or should try to be. # Etc. @@ -47,6 +61,8 @@ We will store somewhat of the mined, active, production data snapshot in the `dev/db/zeldamaps.sql` file so developers and others can start using the project faster, and at all. + We have a separate method below about migrations for modifying data, including production's, to include new fixes and feature data. + If there is important data you would like to add to this file and check it in, you can issue the following command: `dev/db/exportDatabaseForDev.sh`. Then you can review any changes/updates to commit in place of that file. Now that this has become a decent system, here are the available configuration environmental flags you can utilize: - `QUIET` (Default: `false`) @@ -84,6 +100,41 @@ - ``echo 'DROP DATABASE `zeldamaps`' | mysql --login-path=local`` - `mysql --login-path=local < "dev/db/zeldamaps.sql";` +## Migrations + + Tracking database changes to use in the future for features and bug fixes is important, and we will create migration files that can be easily issued in any environment. + + Thought it would be easy that we use what existing projects use to accomplish this, and I have been using Ruby on Rails lately that seems to do a good job of it. + https://github.com/thuss/standalone-migrations + + Now that only accomplishes half of the responsibility. In order to operate on that actual database content data, we don't have ActiveRecord objects to use as an API, so we just implement raw SQL for these, and make sure that both of the `up` and `down` methods are made as appropriate as possible, if both methods possible for that certain situation. + + Important Note: + Do not edit a migration that has been pushed (to others). + + Common Commands: + Create a new migration: + `rake db:new_migration name=foo_bar_migration` + Then edit it with your features's details. + To apply your newest migration: + `rake db:migrate` + To migrate to a specific version (for example to rollback) + `rake db:migrate VERSION=20081220234130` + To migrate a specific database (for example your "testing" database) + `rake db:migrate RAILS_ENV=test` + To execute a specific up/down of one single migration + `rake db:migrate:up VERSION=20081220234130` + To revert your last migration + `rake db:rollback` + To revert your last 3 migrations + `rake db:rollback STEP=3` + Check which version of the tool you are currently using + `rake db:version` + + More Info: + http://edgeguides.rubyonrails.org/active_record_migrations.html + https://www.ralfebert.de/snippets/ruby-rails/models-tables-migrations-cheat-sheet/ + ## MySQL Workbench (MWB) File Handling **Update:** Since the MWB file in our project only contains the schema information, and we need to give developers at least some populated data to work with, it will only be used to design the structure of the tables, so we will no longer use it to merge data into the database for working, either directly, or by exporting it into SQL. diff --git a/Gemfile b/Gemfile new file mode 100644 index 000000000..e1ca506e0 --- /dev/null +++ b/Gemfile @@ -0,0 +1,8 @@ +# frozen_string_literal: true + +source "https://rubygems.org" + +git_source(:github) {|repo_name| "https://github.com/#{repo_name}" } + +gem 'standalone_migrations' +gem 'mysql2' diff --git a/Gemfile.lock b/Gemfile.lock new file mode 100644 index 000000000..62b2f2e4b --- /dev/null +++ b/Gemfile.lock @@ -0,0 +1,76 @@ +GEM + remote: https://rubygems.org/ + specs: + actionpack (5.2.0) + actionview (= 5.2.0) + activesupport (= 5.2.0) + rack (~> 2.0) + rack-test (>= 0.6.3) + rails-dom-testing (~> 2.0) + rails-html-sanitizer (~> 1.0, >= 1.0.2) + actionview (5.2.0) + activesupport (= 5.2.0) + builder (~> 3.1) + erubi (~> 1.4) + rails-dom-testing (~> 2.0) + rails-html-sanitizer (~> 1.0, >= 1.0.3) + activemodel (5.2.0) + activesupport (= 5.2.0) + activerecord (5.2.0) + activemodel (= 5.2.0) + activesupport (= 5.2.0) + arel (>= 9.0) + activesupport (5.2.0) + concurrent-ruby (~> 1.0, >= 1.0.2) + i18n (>= 0.7, < 2) + minitest (~> 5.1) + tzinfo (~> 1.1) + arel (9.0.0) + builder (3.2.3) + concurrent-ruby (1.0.5) + crass (1.0.4) + erubi (1.7.1) + i18n (1.0.1) + concurrent-ruby (~> 1.0) + loofah (2.2.2) + crass (~> 1.0.2) + nokogiri (>= 1.5.9) + method_source (0.9.0) + mini_portile2 (2.3.0) + minitest (5.11.3) + mysql2 (0.5.1) + nokogiri (1.8.2) + mini_portile2 (~> 2.3.0) + rack (2.0.5) + rack-test (1.0.0) + rack (>= 1.0, < 3) + rails-dom-testing (2.0.3) + activesupport (>= 4.2.0) + nokogiri (>= 1.6) + rails-html-sanitizer (1.0.4) + loofah (~> 2.2, >= 2.2.2) + railties (5.2.0) + actionpack (= 5.2.0) + activesupport (= 5.2.0) + method_source + rake (>= 0.8.7) + thor (>= 0.18.1, < 2.0) + rake (12.3.1) + standalone_migrations (5.2.5) + activerecord (>= 4.2.7, < 5.3.0) + railties (>= 4.2.7, < 5.3.0) + rake (>= 10.0) + thor (0.20.0) + thread_safe (0.3.6) + tzinfo (1.2.5) + thread_safe (~> 0.1) + +PLATFORMS + ruby + +DEPENDENCIES + mysql2 + standalone_migrations + +BUNDLED WITH + 1.16.1 diff --git a/INSTALL.md b/INSTALL.md index 6bf72539b..97fc77638 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -30,15 +30,36 @@ * `php-fpm --prefix /usr/local` * PHP dependencies - Run `composer install`. - * Import this Git Flow configuration - ``` - [gitflow "branch"] - master = production - develop = master - [gitflow "prefix"] - feature = feature/ - release = release/ - hotfix = hotfix/ - support = support/ - versiontag = - ``` + * Coding Workflow + * Import this Git Flow configuration: + * ``` + [gitflow "branch"] + master = production + develop = master + [gitflow "prefix"] + feature = feature/ + release = release/ + hotfix = hotfix/ + support = support/ + versiontag = + ``` + * Install database migration creation framework tool + * Install Ruby + * I was using 2.4.1. + * Install RubyGems + * https://rubygems.org/pages/download + * Used 2.6.11 + * Install the migration tool, and its dependencies + * The individual way + * `gem install mysql2` + * 0.5.1 + * `gem install standalone_migrations` + * 5.2.5 + * You might need my edited version of the repository instead. + * https://github.com/Pysis868/standalone-migrations + * As of commit `685d343752a1b42ff844b5a75677db7e4acf8a36`. + * It fixes a particular issue when running several of the commands, but doesn't pass several tests O_o. + * Some commands might not work, but the main ones I have used so far only print a stack trace for the error, but still work. + * The automatic way + * `gem install bundler` + * `bundle install` diff --git a/Rakefile b/Rakefile new file mode 100644 index 000000000..6822452fb --- /dev/null +++ b/Rakefile @@ -0,0 +1,2 @@ +require 'standalone_migrations' +StandaloneMigrations::Tasks.load_tasks diff --git a/dev/db/config.yml.example b/dev/db/config.yml.example new file mode 100644 index 000000000..ca8535f0b --- /dev/null +++ b/dev/db/config.yml.example @@ -0,0 +1,14 @@ +defaults: &DEFAULTS + adapter: mysql + database: zeldamaps + username: zmaps + encoding: utf8 + +development: + <<: *DEFAULTS + password: ... + +production: + <<: *DEFAULTS + password: ... + socket: /var/run/mysqld/mysqld.sock