[Bug Fix] Fix the timestamp casting

CHANGELOG.md

@@ -1,7 +1,36 @@
-# dbt_zuora_source version.version
+# dbt_zuora_source v0.2.2-a1
+This release introduces the following updates:
+
+## Bug Fixes
+- Leveraged the `{{ dbt.type_timestamp() }}` macro within the staging models for all timestamp fields. ([#17](https://github.com/fivetran/dbt_zuora_source/pull/17))
+    - This is needed as certain Redshift warehouses sync these fields as `timestamp with time zone` fields by default, causing compilation errors in downstream models in the `dbt_zuora` package. This macro appropriately removes timezone values from the UTC timestamps and ensures successful compilations of these models.
+
+| **Models**                         | **Field updates cast as timestamp**                                                                 |
+|------------------------------------|------------------------------------------------------------------------------------|
+| `stg_zuora__account`              | `created_date`, `last_invoice_date`, `tax_exempt_effective_date`, `tax_exempt_expiration_date`, `updated_date` |
+| `stg_zuora__amendment`            | `created_date`, `updated_date`                                 |
+| `stg_zuora__contact`            | `created_date`, `updated_date`                                 |
+| `stg_zuora__credit_balance_adjustment` | `cancelled_on`, `created_date`, `updated_date`             |
+| `stg_zuora__invoice_item`         | `charge_date`, `created_date`, `updated_date`                 |
+| `stg_zuora__invoice_payment`      | `created_date`, `updated_date`                                 |
+| `stg_zuora__invoice`              | `created_date`, `last_email_sent_date`, `posted_date`, `updated_date` |
+| `stg_zuora__order`                | `created_date`, `updated_date`                                 |
+| `stg_zuora__payment_method`       | `created_date`, `last_failed_sale_transaction_date`, `last_transaction_date_time`, `updated_date` |
+| `stg_zuora__payment`              | `cancelled_on`, `created_date`, `settled_on`, `submitted_on`, `updated_date` |
+| `stg_zuora__product_rate_plan_charge` | `created_date`, `updated_date`                         |
+| `stg_zuora__product_rate_plan`    | `created_date`, `updated_date`                                 |
+| `stg_zuora__product`    | `created_date`, `updated_date`                                 |
+| `stg_zuora__rate_plan`            | `created_date`, `updated_date`                   |    
+| `stg_zuora__rate_plan_charge`     | `created_date`, `updated_date`                                 |
+| `stg_zuora__refund_invoice_payment`            |     `created_date`, `updated_date`                              |    
+| `stg_zuora__refund`               | `cancelled_on`, `created_date`, `refund_transaction_time`, `settled_on`, `submitted_on`, `updated_date` |
+| `stg_zuora__subscription`         | `cancelled_date`, `contract_acceptance_date`, `contract_effective_date`, `created_date`, `original_created_date`, `service_activation_date`, `term_end_date`, `term_start_date`, `updated_date` |
+| `stg_zuora__taxation_item`        | `created_date`, `exchange_rate_date`, `tax_date`, `updated_date` |
 
 ## Documentation
 - Corrected references to connectors and connections in the README. ([#16](https://github.com/fivetran/dbt_zuora_source/pull/16))
+- Moved badges at top of the README below the H1 header to be consistent with popular README formats. ([#17](https://github.com/fivetran/dbt_zuora_source/pull/17))
+
 
 # dbt_zuora_source v0.2.1
 [PR #11](https://github.com/fivetran/dbt_zuora_source/pull/11) includes the following update:

README.md

@@ -1,4 +1,6 @@
-<p align="center">
+# Zuora Source dbt package ([Docs](https://fivetran.github.io/dbt_zuora_source/))
+
+<p align="left">
     <a alt="License"
         href="https://github.com/fivetran/dbt_zuora_source/blob/main/LICENSE">
         <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" /></a>
@@ -10,8 +12,6 @@
         <img src="https://img.shields.io/badge/Contributions-welcome-blueviolet" /></a>
 </p>
 
-# Zuora Source dbt package ([Docs](https://fivetran.github.io/dbt_zuora_source/))
-
 ## What does this dbt package do?
 - Materializes [Zuora staging tables](https://fivetran.github.io/dbt_zuora_source/#!/overview/github_source/models/?g_v=1), which leverage data in the format described by [this ERD](https://fivetran.com/docs/applications/zuora#schemainformation). These staging tables clean, test, and prepare your Zuora data from [Fivetran's connector](https://fivetran.com/docs/applications/zuora) for analysis by doing the following:
   - Naming columns for consistency across all packages and easier analysis
@@ -42,7 +42,7 @@ Include the following `zuora_source` package version in your `packages.yml` file
 ```yaml
 packages:
   - package: fivetran/zuora_source
-    version: [">=0.2.0", "<0.3.0"] # we recommend using ranges to capture non-breaking changes automatically
+    version: 0.2.2-a1 # we recommend using ranges to capture non-breaking changes automatically
 ```
 ### Step 3: Define database and schema variables
 By default, this package runs using your destination and the `zuora` schema. If this is not where your Zuora data is (for example, if your zuora schema is named `zuora_fivetran`), add the following configuration to your root `dbt_project.yml` file:

dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'zuora_source'
-version: '0.2.1'
+version: '0.2.2'
 config-version: 2
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 

integration_tests/ci/sample.profiles.yml

@@ -16,13 +16,13 @@ integration_tests:
       pass: "{{ env_var('CI_REDSHIFT_DBT_PASS') }}"
       dbname: "{{ env_var('CI_REDSHIFT_DBT_DBNAME') }}"
       port: 5439
-      schema: zuora_source_integration_tests
+      schema: zuora_source_integrations_tests
       threads: 8
     bigquery:
       type: bigquery
       method: service-account-json
       project: 'dbt-package-testing'
-      schema: zuora_source_integration_tests
+      schema: zuora_source_integrations_tests
       threads: 8
       keyfile_json: "{{ env_var('GCLOUD_SERVICE_KEY') | as_native }}"
     snowflake:
@@ -33,7 +33,7 @@ integration_tests:
       role: "{{ env_var('CI_SNOWFLAKE_DBT_ROLE') }}"
       database: "{{ env_var('CI_SNOWFLAKE_DBT_DATABASE') }}"
       warehouse: "{{ env_var('CI_SNOWFLAKE_DBT_WAREHOUSE') }}"
-      schema: zuora_source_integration_tests
+      schema: zuora_source_integrations_tests
       threads: 8
     postgres:
       type: postgres
@@ -42,13 +42,13 @@ integration_tests:
       pass: "{{ env_var('CI_POSTGRES_DBT_PASS') }}"
       dbname: "{{ env_var('CI_POSTGRES_DBT_DBNAME') }}"
       port: 5432
-      schema: zuora_source_integration_tests
+      schema: zuora_source_integrations_tests
       threads: 8
     databricks:
       catalog: "{{ env_var('CI_DATABRICKS_DBT_CATALOG') }}"
       host: "{{ env_var('CI_DATABRICKS_DBT_HOST') }}"
       http_path: "{{ env_var('CI_DATABRICKS_DBT_HTTP_PATH') }}"
-      schema: zuora_source_integration_tests
+      schema: zuora_source_integrations_tests
       threads: 2
       token: "{{ env_var('CI_DATABRICKS_DBT_TOKEN') }}"
       type: databricks

integration_tests/dbt_project.yml

@@ -1,10 +1,10 @@
 name: 'zuora_source_integration_tests'
-version: '0.2.1'
+version: '0.2.2'
 config-version: 2
 profile: 'integration_tests'
 
 vars:
-  zuora_schema: zuora_source_integration_tests
+  zuora_schema: zuora_source_integrations_tests
   zuora_source:
     zuora_account_identifier: "zuora_account_data" 
     zuora_amendment_identifier: "zuora_amendment_data"

models/stg_zuora__account.sql

@@ -28,26 +28,26 @@ final as (
         bill_cycle_day,
         bill_to_contact_id,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         credit_balance,
         crm_id,
         currency,
         default_payment_method_id,
-        last_invoice_date,
+        cast(last_invoice_date as {{ dbt.type_timestamp() }}) as last_invoice_date,
         mrr,
         name,
         notes,
         parent_account_id,
         payment_term,
         sold_to_contact_id,
         status, 
-        tax_exempt_effective_date, 
-        tax_exempt_expiration_date, 
+        cast(tax_exempt_effective_date as {{ dbt.type_timestamp() }}) as tax_exempt_effective_date, 
+        cast(tax_exempt_expiration_date as {{ dbt.type_timestamp() }}) as tax_exempt_expiration_date, 
         tax_exempt_status, 
         total_debit_memo_balance,
         total_invoice_balance,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         vatid,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
 

models/stg_zuora__amendment.sql

@@ -26,7 +26,7 @@ final as (
         code,
         contract_effective_date,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         current_term,
         current_term_period_type,
         customer_acceptance_date,
@@ -48,7 +48,7 @@ final as (
         term_type,
         type,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__contact.sql

@@ -27,14 +27,14 @@ final as (
         city,
         country,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         description, 
         first_name, 
         last_name,  
         postal_code,
         state,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         work_email,
         work_phone,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record,

models/stg_zuora__credit_balance_adjustment.sql

@@ -29,10 +29,10 @@ final as (
         amount, 
         amount_home_currency,
         bill_to_contact_id,
-        cancelled_on,
+        cast(cancelled_on as {{ dbt.type_timestamp() }}) as cancelled_on,
         comment,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         default_payment_method_id,
         exchange_rate,
         exchange_rate_date,
@@ -52,7 +52,7 @@ final as (
         transferred_to_accounting,
         type,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__invoice.sql

@@ -32,7 +32,7 @@ final as (
         bill_to_contact_id,
         comments,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         credit_balance_adjustment_amount,
         credit_memo_amount,
         default_payment_method_id,
@@ -45,12 +45,12 @@ final as (
         includes_usage,
         invoice_date,
         invoice_number,
-        last_email_sent_date,
+        cast(last_email_sent_date as {{ dbt.type_timestamp() }}) as last_email_sent_date,
         parent_account_id,
         payment_amount,
         payment_term,
         posted_by,
-        posted_date, 
+        cast(posted_date as {{ dbt.type_timestamp() }}) as posted_date, 
         refund_amount, 
         sold_to_contact_id,
         source,
@@ -63,7 +63,7 @@ final as (
         transaction_currency,
         transferred_to_accounting,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__invoice_item.sql

@@ -30,10 +30,10 @@ final as (
         bill_to_contact_id, 
         charge_amount, 
         charge_amount_home_currency,
-        charge_date,
+        cast(charge_date as {{ dbt.type_timestamp() }}) as charge_date,
         charge_name,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         deferred_revenue_accounting_code_id,
         exchange_rate,
         exchange_rate_date,
@@ -62,7 +62,7 @@ final as (
         unit_price,
         uom,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__invoice_payment.sql

@@ -30,7 +30,7 @@ final as (
         bill_to_contact_id,
         cash_accounting_code_id,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         default_payment_method_id,
         exchange_rate,
         exchange_rate_date,
@@ -46,7 +46,7 @@ final as (
         sold_to_contact_id,
         transaction_currency,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__order.sql

@@ -25,7 +25,7 @@ final as (
         bill_to_contact_id,
         category,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         default_payment_method_id,
         description,
         error_code,
@@ -38,7 +38,7 @@ final as (
         state,
         status,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__payment.sql

@@ -28,10 +28,10 @@ final as (
         applied_amount,
         applied_credit_balance_amount,
         bill_to_contact_id,
-        cancelled_on,
+        cast(cancelled_on as {{ dbt.type_timestamp() }}) as cancelled_on,
         comment,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         currency,
         effective_date,
         exchange_rate,
@@ -41,16 +41,16 @@ final as (
         payment_method_id,
         payment_number,
         refund_amount,
-        settled_on, 
+        cast(settled_on as {{ dbt.type_timestamp() }}) as settled_on, 
         sold_to_contact_id,
         status,
-        submitted_on,
+        cast(submitted_on as {{ dbt.type_timestamp() }}) as submitted_on,
         transaction_currency,
         transferred_to_accounting,
         type,
         unapplied_amount,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__payment_method.sql

@@ -26,10 +26,10 @@ final as (
         active,
         bank_transfer_account_type,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         credit_card_type,
-        last_failed_sale_transaction_date,
-        last_transaction_date_time,
+        cast(last_failed_sale_transaction_date as {{ dbt.type_timestamp() }}) as last_failed_sale_transaction_date,
+        cast(last_transaction_date_time as {{ dbt.type_timestamp() }}) as last_transaction_date_time,
         last_transaction_status,
         max_consecutive_payment_failures,
         name,
@@ -41,7 +41,7 @@ final as (
         total_number_of_processed_payments,
         type,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__product.sql

@@ -23,14 +23,14 @@ final as (
         id as product_id,
         category,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         description,
         effective_end_date,
         effective_start_date,
         name,
         sku,
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)

models/stg_zuora__product_rate_plan.sql

@@ -22,14 +22,14 @@ final as (
     select 
         id as product_rate_plan_id,
         created_by_id,
-        created_date,
+        cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
         description,
         effective_end_date,
         effective_start_date, 
         name,
         product_id, 
         updated_by_id,
-        updated_date,
+        cast(updated_date as {{ dbt.type_timestamp() }}) as updated_date,
         row_number() over (partition by id order by updated_date desc) = 1 as is_most_recent_record
     from fields
     where not coalesce(_fivetran_deleted, false)