JTS models POC

[pwalsh]

Description

A POC for JSON Table Schema models: we develop it here in a real code + data environment, and then when we are happy with the API, we extract to https://github.com/frictionlessdata/jsontableschema-models-js and publish as a v0.1 for object models using JTS.

Proposal

# code that belongs to the jsontableschema-models library
/{MODEL_LIB}/models.js // the base model class
/{MODEL_LIB}/query.js // the base query class
/{MODEL_LIB}/define.js // a helper function to define models
/{MODEL_LIB}/index.js // export the library

# code that belongs to the ebola code base
/{SRC}/models/
/{SRC}/models/trials.json // JSON Table Schema descriptor
/{SRC}/models/cases.json // JSON Table Schema descriptor
/{SRC}/models/trials.js // Model defintion
/{SRC}/models/cases.js // Model definition
/{SRC}/models/index.js // export all models, and also the query class from the lib

# models/trials.js
import define from 'jsontableschema-models'

const Trials = define({
  resource: 'http://example.com/trials.csv',  // path to the data source
  descriptor: 'trials.json',  // path to the jts descriptor
  tableName: 'trials',
  instanceMethods: {
    // define custom instance methods here
  },
  classMethods: {
    // define custom class methods here
  }
})

module export Trials

# models/index.js

import { Query, connect } from 'jsontableschema-models'
import Trials from './trials'
import Cases from './cases'

// connect to/init storage

module exports {
  Query,
  Trials,
  Cases
}

# storage object, in memory
{
    "trials": [
      {...},
      {...}
    ],
    "cases": [
      {...},
      {...}
    ]
}

# querying

# there probably is a low-level API to query on the Storage itself
Storage.query('trials', 'find', {'id': 'SOME_ID'})

# a modified version on the models, that calls the lower-level one
Trials.query('find', {'id': 'SOME_ID'})

# and, possibly, a simplified API, that is not Storage dependent, and just implements some commons methods for collections of records
Query('trials').get(id={SOME_ID})
Query('trials').filter(type={SOME_TYPE})

References

• Sequelize ORM
  • Example usage of Sequelize
• lodash API

Tasks

• Implement JSON Table Schema definitions for our tables
• Implement models for our tables
• Implement Storage in jsontableschema-js along the lines of the Python implementation, with a default storage type of memory, being an object in memory
• Implement the low-level query API over the memory storage type, using lodash
• Implement a high-level Query API
• Implement an init/prerun method that loads from the source tables into our data storage (likely, we call this when the server is loaded)
  • Iterates over all sources, based on model definition, and writes (cast) data to storage
• Unit tests

[pwalsh]

@zhenyab based on our conversation today: anything missing?

@roll FYI: any comments or thoughts?

[roll]

Not fully into it for now. But looks goods AFAIK. It will be good to see on this real world example how it works.

[zhenyab]

Seems fine to me.