diff --git a/README.rst b/README.rst index b01afebe..6f1464b5 100644 --- a/README.rst +++ b/README.rst @@ -5,3 +5,193 @@ Model synchronization from `dbt`_ to `Metabase`_. .. _`dbt`: https://www.getdbt.com/ .. _`Metabase`: https://www.metabase.com/ + +If dbt is your source of truth for database schemas and you use Metabase as +your analytics tool, dbt-metabase can propagate table relationships, model and +column descriptions and special types (e.g. currency, category, URL) to your +Metabase data model. + +Requirements +============ + +Requires Python 3.6 or above. + +Usage +===== + +You can install dbt-metabase from `PyPI`_: + +.. _`PyPI`: https://pypi.org/project/dbt-metabase/ + +.. code-block:: shell + + pip install dbt-metabase + +To install the dbt package in your project, add the following to your +``packages.yml``: + +.. code-block:: yaml + + packages: + - git: https://github.com/gouline/dbt-metabase.git + revision: vX.Y.Z + +Where ``vX.Y.Z`` corresponds to the `latest release`_. Now run ``dbt deps`` and +you're ready to go. + +.. _`latest release`: https://github.com/gouline/dbt-metabase/releases/latest + +Basic Example +------------- + +Let's start by defining a short sample ``schema.yml`` as below. + +.. code-block:: yaml + + models: + - name: stg_users + description: User records. + columns: + - name: id + description: Primary key. + tests: + - not_null + - unique + - name: email + description: User's email address. + - name: group_id + description: Foreign key to user group. + tests: + - not_null + - relationships: + to: ref('groups') + field: id + - name: stg_groups + description: User groups. + columns: + - name: id + description: Primary key. + tests: + - not_null + - unique + - name: name + description: Group name. + +That's already enough to propagate the primary keys, foreign keys and +descriptions to Metabase by executing the below command. + +.. code-block:: shell + + dbt-metabase export \ + --dbt_path . \ + --mb_host metabase.example.com \ + --mb_user user@example.com \ + --mb_password Password123 \ + --database business \ + --schema public + +Check your Metabase instance by going into Settings > Admin > Data Model, you +will notice that ``ID`` in ``STG_USERS`` is now marked as "Entity Key" and +``GROUP_ID`` is marked as "Foreign Key" pointing to ``ID`` in ``STG_GROUPS``. + +Special Types +------------- + +Now that we have primary and foreign keys, let's tell Metabase that ``email`` +column contains email addresses. + +Change the ``email`` column as follows: + +.. code-block:: yaml + + - name: email + description: User's email address. + tests: + - metabase.column: + special_type: type/Email + +Once you run ``dbt-metabase export`` again, you will notice that ``EMAIL`` is +now marked as "Email". + +Here is the list of special types currently accepted by Metabase: + +* ``type/AvatarURL`` +* ``type/Category`` +* ``type/City`` +* ``type/Country`` +* ``type/Currency`` +* ``type/Description`` +* ``type/Email`` +* ``type/Enum`` +* ``type/ImageURL`` +* ``type/SerializedJSON`` +* ``type/Latitude`` +* ``type/Longitude`` +* ``type/Number`` +* ``type/State`` +* ``type/URL`` +* ``type/ZipCode`` +* ``type/Quantity`` +* ``type/Income`` +* ``type/Discount`` +* ``type/CreationTimestamp`` +* ``type/CreationTime`` +* ``type/CreationDate`` +* ``type/CancelationTimestamp`` +* ``type/CancelationTime`` +* ``type/CancelationDate`` +* ``type/DeletionTimestamp`` +* ``type/DeletionTime`` +* ``type/DeletionDate`` +* ``type/Product`` +* ``type/User`` +* ``type/Source`` +* ``type/Price`` +* ``type/JoinTimestamp`` +* ``type/JoinTime`` +* ``type/JoinDate`` +* ``type/Share`` +* ``type/Owner`` +* ``type/Company`` +* ``type/Subscription`` +* ``type/Score`` +* ``type/Title`` +* ``type/Comment`` +* ``type/Cost`` +* ``type/GrossMargin`` +* ``type/Birthdate`` + +If you notice new ones, please submit a PR to update this readme and +``macros/tests.sql``. + +Database Sync +------------- + +By default, dbt-metabase will tell Metabase to synchronize database fields +and wait for the data model to contain all the tables and columns in your dbt +project. + +You can control this behavior with two arguments: + +* ``--sync`` - boolean to enable or disable pre-synchronization +* ``--sync_timeout`` - number of seconds to wait and re-check data model before + giving up + +Programmatic Invocation +----------------------- + +As you have already seen, you can invoke dbt-metabase from the command +line. But if you prefer to call it from your code, here's how to do it: + +.. code-block:: python + + import dbtmetabase + + dbtmetabase.export(dbt_path, mb_host, mb_user, mb_password, database, schema) + +Code of Conduct +=============== + +All contributors are expected to follow the PyPA `Code of Conduct`_. + +.. _`Code of Conduct`: https://www.pypa.io/en/latest/code-of-conduct/ diff --git a/dbt_project.yml b/dbt_project.yml index 7be7f52d..343eadd4 100644 --- a/dbt_project.yml +++ b/dbt_project.yml @@ -1,5 +1,5 @@ name: 'metabase' -version: '0.1.3' +version: '0.1.4' target-path: "target" clean-targets: ["target", "dbt_modules"] diff --git a/dbtmetabase/__init__.py b/dbtmetabase/__init__.py index 3c326735..2c3c5739 100644 --- a/dbtmetabase/__init__.py +++ b/dbtmetabase/__init__.py @@ -3,7 +3,7 @@ from .dbt import DbtReader from .metabase import MetabaseClient -__version__ = '0.1.3' +__version__ = '0.1.4' def export(dbt_path: str, mb_host: str, mb_user: str, mb_password: str, diff --git a/dbtmetabase/metabase.py b/dbtmetabase/metabase.py index bf98e89e..816b2cc0 100644 --- a/dbtmetabase/metabase.py +++ b/dbtmetabase/metabase.py @@ -174,6 +174,10 @@ def export_column(self, model_name: str, column: dict, field_lookup: dict): fk_target_field_id = field_lookup.get(column['fk_target_table'], {}) \ .get(column['fk_target_column'], {}) \ .get('id') + + self.api('put', f'/api/field/{fk_target_field_id}', json={ + 'special_type': 'type/PK' + }) api_field = self.api('get', f'/api/field/{field_id}')