Merge pull request #2 from Nike-Inc/release-v4.1.0

Release v4.1.0

CHANGELOG.md

@@ -4,6 +4,7 @@ All notable changes to this project will be documented in this file.
 `knockoff-factory` adheres to [Semantic Versioning](http://semver.org/).
 
 #### 4.x Releases
+- `4.1.x` Releases - [4.1.0](#410)
 - `4.0.x` Releases - [4.0.0](#400)
 
 #### 3.x Releases
@@ -36,6 +37,19 @@ All notable changes to this project will be documented in this file.
 
 #### Removed
 
+#### Fixed
+---
+
+## 4.1.0
+
+#### Added
+- CollectionsFactory for providing factory functions that return multiple columns 
+    - Added by [Gregory Yu](https://github.com/gregyu) in Pull Request [#2](https://github.com/Nike-Inc/knockoff-factory/pull/2)
+- Documentation and jupyter notebook for KnockoffTable
+    - Added by [Gregory Yu](https://github.com/gregyu) in Pull Request [#2](https://github.com/Nike-Inc/knockoff-factory/pull/2)
+
+#### Updated
+
 #### Fixed
 
 ---

README.md

@@ -11,6 +11,46 @@ Knockoff Factory
 
 A library for generating mock data and creating database fixtures that can be used for unit testing.
 
+Table of content
+* [Installation](#installation)
+* [Changelog](#changelog)
+* [Documentation](#documentation)
+* [Future Work](#Future-work)
+
+# <a name="installation"></a> Installation
+```shell script
+pip install knockoff
+```
+
+
+# <a name="changelog"></a> Changelog
+
+See the [changelog](CHANGELOG.md) for a history of notable changes to knockoff.
+
+# <a name="documentation"></a> Documentation
+
+We are working on adding more documentation and examples!  
+
+* knockoff sdk
+    * [KnockoffTable](notebook/KnockoffTable.ipynb)
+    * KnockoffDB
+        * KnockoffDatabaseService
+* TempDatabaseService
+* knockoff cli
+
+
+# <a name="future-work"></a> Future work
+* Further documentation and examples for SDK
+* Add yaml based configuration for SDK
+* Make extensible generic output for KnockffDB.insert (csv, parquet, etc)
+* Enable append option for KnockoffDB.insert
+* Autodiscover and populate all tables by using reflection and building dependency graph with foreign key relationships
+* Parallelize execution of dag. (e.g. https://ipython.org/ipython-doc/stable/parallel/dag_dependencies.html)
+
+
+Local development setup and legacy documentation 
+---
+
 # Run poetry install/update with psycopg2
 
 Requirements:
@@ -70,12 +110,6 @@ following environment variable
 export TEST_POSTGRES_ENABLED=0
 ```
 
-### Knockoff Python Based Configuration
-
-We will be providing documentation along with example repo's for
-the python based configuration using the knockoff.sdk modules.
-
-
 ### Knockoff Legacy Yaml Based Configuration
 Note: This yaml based configuration has been moved under the legacy subcommand
 for knockoff and a new yaml based configuration will be introduced that 
@@ -284,11 +318,3 @@ Note: if you are loading data from an s3 bucket you have access to, you can enab
 container access to those credentials by adding `-v ~/.aws:/root/.aws` to the `docker run` command.
 
 
-### Future work
-* Add documentation and examples for SDK
-* Add yaml based configuration for SDK
-* Make extensible generic output for KnockffDB.insert (csv, parquet, etc)
-* Enable append option for KnockoffDB.insert
-* Autodiscover and populate all tables by using reflection and building dependency graph with foreign key relationships
-* Documentation / use-cases
-* Parallelize execution of dag. (e.g. https://ipython.org/ipython-doc/stable/parallel/dag_dependencies.html)

knockoff/sdk/factory/collections.py

@@ -10,6 +10,42 @@
 from .next_strategy.table import sample_table
 
 
+class CollectionsFactory(object):
+    def __init__(self,
+                 callable_,
+                 columns=None,
+                 rename=None,
+                 drop=None,
+                 depends_on=None):
+        self.callable = callable_
+        self.depends_on = depends_on
+        self.columns = columns
+        self.rename = rename or {}
+        self.drop = set(drop or [])
+
+    def __call__(self, *args, **kwargs):
+        record = self.callable(*args, **kwargs)
+        return resolve_columns(record,
+                               columns=self.columns,
+                               rename=self.rename,
+                               drop=self.drop)
+
+
+def resolve_columns(record,
+                    columns=None,
+                    rename=None,
+                    drop=None):
+    rename = rename or {}
+    drop = set(drop or [])
+    out = {}
+    columns = columns or record.keys()
+    for col in columns:
+        if col in drop:
+            continue
+        out[rename.get(col, col)] = record[col]
+    return out
+
+
 class KnockoffFactory(object):
     def __init__(self, obj, columns=None, rename=None, drop=None,
                  next_strategy_callable=None,

knockoff/sdk/table.py

@@ -18,6 +18,7 @@
 
 
 from .factory.column import ColumnFactory
+from .factory.collections import CollectionsFactory
 
 
 logger = logging.getLogger(__name__)
@@ -180,7 +181,7 @@ def build_record(self):
                                "Use ColumnFactory instead")
                 col, factory = factory
                 key_values = {col: factory()}
-            elif (isinstance(factory, (ColumnFactory,)) and
+            elif (isinstance(factory, (ColumnFactory, CollectionsFactory)) and
                    factory.depends_on):
                 kwargs = {key: data[key] for key in factory.depends_on}
                 key_values = factory(**kwargs)