Problems with our current codebase

This document outlines the problems seen in our current codebase. When contributing a problem, please provide examples and possible solutions.

Duplication of variables/values between database and application code

Consider a delta that needs to add a table with default values. This may be a special value like the sales tax rate or timezone of Austin, TX. This is a fairly trivial example, but it highlights the issue: namely that the delta script must independently know the value of these special variables.

Possible Solution: String Interpolation

We could interpolate values into the delta script:

DO $$
  declare version             text          := '1.2.12';
  declare default_region      text          := {{config.default_region_name}};
begin
  raise notice '## Running Delta v% ##', version;
  ...
end;

I would suggest that we only allow interpolation inside of the variable declaration block to avoid type errors. I would also suggest that we make this a feature of our upstream pg-delta module and to use that as a dependency in the future.

The number of places you have to specify table names

Creating a new table involves:

• Adding delta schema
• Add definition schema (with table name in 3 different styles)
  • Table name, lowercase, hyphenated in filename
  • Table name, uppercase, with spaces in comments header
  • Table name, lowercase, underscored in definition.name property
• Add to create-tables task
• New model definitions

Possible Solution: Use Dirac

If we use dirac's database syncing features, at least for a fresh setup, then it will take care of schema registration, building the DAG to create the tables in the correct order. That takes care of only 1 of the items (the create-tables task)

Possible Solution: Code Scaffolds

We should add a code generator for schemas at the very least. This could accept a table name and figure out how to properly place it in all the various parts of the (including the filename). The generator could also ask about schema information.

Functional Testing

In order to test, you need a working database state. This is achieved by either grabbing a copy of the latest production data or scaffolding fake data. Then we must start the test server, and finally run the test scripts.

A typical example for me is

node db/setup.js --test 
node db/fake-data.js --test
./bin/start-test
npm test

Because there's four distinct steps that must be run in this order, I might (and often do) mess up running the test suite.

I also tend to forget we have tests, so continuous integration would be even nicer where I don't need to run the tests all the time manually.

Possible Solutions

• Add another npm script for testing against a copy of the production db or fake data. Something along the lines of npm run test-production or npm run test-fake.
• Write a script that can run flags for individual tests
• Add CI testing
• Remember to test and extend tests (for Preston :P )