Merge pull request #127 from fivetran/release/v0.13.0

release/v0.13.0

.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md

@@ -0,0 +1,25 @@
+## PR Overview
+**This PR will address the following Issue/Feature:**
+
+**This PR will result in the following new package version:**
+<!--- Please add details around your decision for breaking vs non-breaking version upgrade. If this is a breaking change, were backwards-compatible options explored? -->
+
+**Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:**
+<!--- Copy/paste the CHANGELOG for this version below. -->
+
+## PR Checklist
+### Basic Validation
+Please acknowledge that you have successfully performed the following commands locally:
+- [ ] dbt run –full-refresh && dbt test
+- [ ] dbt run (if incremental models are present)
+
+Before marking this PR as "ready for review" the following have been applied:
+- [ ] The appropriate issue has been linked, tagged, and properly assigned
+- [ ] All necessary documentation and version upgrades have been applied
+- [ ] docs were regenerated (unless this PR does not include any code or yml updates)
+- [ ] BuildKite integration tests are passing
+- [ ] Detailed validation steps have been provided below
+
+### Detailed Validation
+Please share any and all of your validation steps:
+<!--- Provide the steps you took to validate your changes below. -->

CHANGELOG.md

@@ -1,3 +1,22 @@
+# dbt_zendesk v0.13.0
+
+## 🚨 Breaking Change (Snowflake users) 🚨
+- We have changed our identifier logic in the initial Zendesk source package to account for `group` being both a Snowflake reserved word and a source table. Given `dbt_zendesk_source` is a dependency for this package, Snowflake users will want to execute a `dbt run --full-refresh` before using the new version of the package. [PR #42](https://github.com/fivetran/dbt_zendesk_source/pull/42)
+
+## Bug Fixes
+- Updates the `int_zendesk__schedule_spine` model to convert the Holiday schedules into proper UTC values before being used in comparison with the schedule times. This ensures the holidays are properly mapped to schedules regardless of timezones. ([PR #126](https://github.com/fivetran/dbt_zendesk/pull/126))
+
+## 🚀 Feature Updates 🚀
+- Added `solve_time_in_calendar_minutes` and `solve_time_in_business_minutes` to our `zendesk__ticket_metrics` model, which calculates calendar and business minutes for when the ticket was in the 'new', 'open', 'hold', or 'pending' status. ([PR #123](https://github.com/fivetran/dbt_zendesk/pull/123))
+
+## 🔎 Under the Hood 🔎 
+- Updated the seed files and seed file configurations for the package integration tests to align with changes in  dbt_zendesk_source made in [PR #42](https://github.com/fivetran/dbt_zendesk_source/pull/42) for applying the `dbt_utils.star` macro.
+- Corrected the folder structure for the `.github` folder to properly categorize the Community and Maintainer PR templates. ([PR #126](https://github.com/fivetran/dbt_zendesk/pull/126))
+
+## Contributors
+- [@fivetran-jessicacherny](https://github.com/fivetran-jessicacherny) ([Issue #119](https://github.com/fivetran/dbt_zendesk/issues/119))
+- [@cth84](https://github.com/cth84) ([PR #118](https://github.com/fivetran/dbt_zendesk/pull/118))
+
 # dbt_zendesk v0.12.0
 
 This release includes fixes to issues introduced in v0.11.0-v0.11.1 surrounding the incorporation of schedule holidays.

README.md

@@ -59,7 +59,7 @@ Include the following zendesk package version in your `packages.yml` file:
 ```yml
 packages:
   - package: fivetran/zendesk
-    version: [">=0.12.0", "<0.13.0"]
+    version: [">=0.13.0", "<0.14.0"]
 
 ```
 > **Note**: Do not include the Zendesk source package. The Zendesk transform package already has a dependency on the source in its own `packages.yml` file.
@@ -175,7 +175,7 @@ This dbt package is dependent on the following dbt packages. Please be aware tha
 ```yml
 packages:
     - package: fivetran/zendesk_source
-      version: [">=0.9.0", "<0.10.0"]
+      version: [">=0.10.0", "<0.11.0"]
 
     - package: fivetran/fivetran_utils
       version: [">=0.4.0", "<0.5.0"]

dbt_project.yml

@@ -1,5 +1,6 @@
 name: 'zendesk'
-version: '0.12.0'
+version: '0.13.0'
+
 
 config-version: 2
 require-dbt-version: [">=1.3.0", "<2.0.0"]

integration_tests/.gitignore

@@ -3,4 +3,5 @@ target/
 dbt_modules/
 logs/
 .DS_Store
-dbt_packages/
+dbt_packages/
+package-lock.yml

integration_tests/ci/sample.profiles.yml

@@ -16,13 +16,13 @@ integration_tests:
       pass: "{{ env_var('CI_REDSHIFT_DBT_PASS') }}"
       dbname: "{{ env_var('CI_REDSHIFT_DBT_DBNAME') }}"
       port: 5439
-      schema: zendesk_integration_tests_2
+      schema: zendesk_integration_tests_40
       threads: 8
     bigquery:
       type: bigquery
       method: service-account-json
       project: 'dbt-package-testing'
-      schema: zendesk_integration_tests_2
+      schema: zendesk_integration_tests_40
       threads: 8
       keyfile_json: "{{ env_var('GCLOUD_SERVICE_KEY') | as_native }}"
     snowflake:
@@ -33,7 +33,7 @@ integration_tests:
       role: "{{ env_var('CI_SNOWFLAKE_DBT_ROLE') }}"
       database: "{{ env_var('CI_SNOWFLAKE_DBT_DATABASE') }}"
       warehouse: "{{ env_var('CI_SNOWFLAKE_DBT_WAREHOUSE') }}"
-      schema: zendesk_integration_tests_2
+      schema: zendesk_integration_tests_40
       threads: 8
     postgres:
       type: postgres
@@ -42,13 +42,13 @@ integration_tests:
       pass: "{{ env_var('CI_POSTGRES_DBT_PASS') }}"
       dbname: "{{ env_var('CI_POSTGRES_DBT_DBNAME') }}"
       port: 5432
-      schema: zendesk_integration_tests_2
+      schema: zendesk_integration_tests_40
       threads: 8
     databricks:
       catalog: "{{ env_var('CI_DATABRICKS_DBT_CATALOG') }}"
       host: "{{ env_var('CI_DATABRICKS_DBT_HOST') }}"
       http_path: "{{ env_var('CI_DATABRICKS_DBT_HTTP_PATH') }}"
-      schema: zendesk_integration_tests_2
+      schema: zendesk_integration_tests_40
       threads: 8
       token: "{{ env_var('CI_DATABRICKS_DBT_TOKEN') }}"
       type: databricks

integration_tests/dbt_project.yml

@@ -1,12 +1,12 @@
 config-version: 2
 
 name: 'zendesk_integration_tests'
-version: '0.12.0'
+version: '0.13.0'
 
 profile: 'integration_tests'
 
 vars:
-  zendesk_schema: zendesk_integration_tests_2
+  zendesk_schema: zendesk_integration_tests_40
   zendesk_source:
     zendesk_organization_identifier:           "organization_data"
     zendesk_schedule_identifier:               "schedule_data"
@@ -20,15 +20,17 @@ vars:
     zendesk_domain_name_identifier:            "domain_name_data"
     zendesk_time_zone_identifier:              "time_zone_data"
     zendesk_daylight_time_identifier:          "daylight_time_data"
-    group:                                     "{{ ref('group_data') }}" 
-    brand:                                     "{{ fivetran_utils.seed_data_helper('brand_data',['postgres']) }}" 
-    user_tag:                                  "{{ fivetran_utils.snowflake_seed_data('user_tag_data') }}" 
-    organization_tag:                          "{{ fivetran_utils.snowflake_seed_data('organization_tag_data') }}" 
-    user:                                      "{{ fivetran_utils.snowflake_seed_data('user_data') }}" 
+    zendesk_brand_identifier:                  "brand_data"
+    zendesk_group_identifier:                  "group_data" 
+    zendesk_organization_tag_identifier:       "organization_tag_data"
+    zendesk_user_identifier:                   "user_data"
+    zendesk_user_tag_identifier:               "user_tag_data"
 
 seeds:
   +quote_columns: "{{ true if target.type == 'redshift' else false }}"
   zendesk_integration_tests:
+    +column_types:
+      _fivetran_synced: timestamp
     +column_types:
       _fivetran_synced: timestamp
     group_data:
@@ -74,7 +76,6 @@ seeds:
         requester_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         submitter_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         ticket_form_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
-        type: "{{ 'varchar(100)'  if target.type in ('redshift', 'postgres') else 'string'}}"
     ticket_field_history_data:
       +column_types:
         ticket_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
@@ -88,24 +89,33 @@ seeds:
     user_data:
       +column_types:
         id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
-        external_id: "{{ 'varchar(100)'  if target.type in ('redshift', 'postgres') else 'string'}}"
+        external_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         locale_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         organization_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         created_at: timestamp
         last_login_at: timestamp
+        updated_at: timestamp
       +enabled: "{{ true if target.type != 'snowflake' else false }}"
-    user_tag_data:
+    user_data_snowflake:
+      +alias: user_data
       +column_types:
-        user_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
-        _fivetran_synced: timestamp
+        id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
+        external_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
+        locale_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
+        organization_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
+        created_at: timestamp
+        last_login_at: timestamp
+        updated_at: timestamp
+      +enabled: "{{ true if target.type == 'snowflake' else false }}"
     brand_data:
       +column_types:
         id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         _fivetran_synced: timestamp
       +enabled: "{{ true if target.type != 'postgres' else false }}"
     brand_data_postgres:
+      +alias: brand_data
       +column_types:
-        id: bigint
+        id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         _fivetran_synced: timestamp
       +enabled: "{{ true if target.type == 'postgres' else false }}"
     ticket_form_history_data:
@@ -116,25 +126,23 @@ seeds:
       +column_types:
         organization_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         _fivetran_synced: timestamp
-    organization_tag_data:
+    user_tag_data:
       +column_types:
-        organization_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
+        user_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
         _fivetran_synced: timestamp
-    user_data_snowflake:
-      +column_types:
-        id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
-        external_id: "{{ 'varchar(100)'  if target.type in ('redshift', 'postgres') else 'string'}}"
-        locale_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
-        organization_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
-        created_at: timestamp
-        last_login_at: timestamp
-      +enabled: "{{ true if target.type == 'snowflake' else false }}"
+      +enabled: "{{ true if target.type != 'snowflake' else false }}"  
     user_tag_data_snowflake:
+      +alias: user_tag_data
       +enabled: "{{ true if target.type == 'snowflake' else false }}"
+    organization_tag_data:
+      +column_types:
+        organization_id: "{{ 'int64' if target.type == 'bigquery' else 'bigint' }}"
+        _fivetran_synced: timestamp
+      +enabled: "{{ true if target.type != 'snowflake' else false }}"  
     organization_tag_data_snowflake:
+      +alias: organization_tag_data
       +enabled: "{{ true if target.type == 'snowflake' else false }}"
 
-
 dispatch:
   - macro_namespace: dbt_utils
     search_order: ['spark_utils', 'dbt_utils']

integration_tests/requirements.txt

@@ -4,6 +4,4 @@ dbt-redshift>=1.3.0,<2.0.0
 dbt-postgres>=1.3.0,<2.0.0
 dbt-spark>=1.3.0,<2.0.0
 dbt-spark[PyHive]>=1.3.0,<2.0.0
-dbt-databricks>=1.3.0,<2.0.0
-
-oscrypto @ git+https://github.com/wbond/oscrypto.git@d5f3437
+dbt-databricks>=1.6.0,<2.0.0

integration_tests/seeds/ticket_data.csv

@@ -1,5 +1,5 @@
 id,_fivetran_synced,allow_channelback,assignee_id,brand_id,created_at,description,due_at,external_id,forum_topic_id,group_id,has_incidents,is_public,organization_id,priority,problem_id,recipient,requester_id,status,subject,submitter_id,system_client,ticket_form_id,type,updated_at,url,via_channel,via_source_from_id,via_source_from_title,via_source_rel,via_source_to_address,via_source_to_name,merged_ticket_ids,via_source_from_address,followup_ids,via_followup_source_id
-1595,2020-03-20 02:32:49.426,FALSE,,360003529474,2020-02-19 01:54:52,description1,,,,360006965034,FALSE,TRUE,370295712714,,,[email protected],396331237134,deleted,subject1,396331237134,,360002048693,,2020-02-19 01:55:11,https://zendesk.com/api/v2/tickets/1595.json,web,,,,[email protected],,[],,,
+1595,2020-03-20 02:32:49.426,FALSE,,360003529474,2020-02-19 01:54:52,description1,,,,360006965034,FALSE,TRUE,370295712714,,,[email protected],396331237134,deleted,subject1,396331237134,,360002048693,incident,2020-02-19 01:55:11,https://zendesk.com/api/v2/tickets/1595.json,web,,,,[email protected],,[],,,
 16988,2021-01-13 20:09:16.325,FALSE,418284131934,360003529474,2020-12-22 00:19:23,description1,,,,360013366274,FALSE,TRUE,370469077513,,,[email protected],1500656884401,solved,subject1,1500656884401,,360002048693,,2021-01-13 18:42:39,https://zendesk.com/api/v2/tickets/16988.json,email,,,,[email protected],Support,[],,[],
 14173,2020-11-11 20:08:45.130,FALSE,396371699653,360003529474,2020-10-28 12:03:02,description1,,,,360006965034,FALSE,TRUE,370321120273,,,[email protected],424883466453,closed,subject1,424883466453,,360002048693,,2020-11-11 17:01:32,https://zendesk.com/api/v2/tickets/14173.json,email,,,,[email protected],Support,[],,,
 11071,2020-10-02 14:08:33.216,FALSE,,360003529474,2020-08-28 18:06:36,description1,,,,,FALSE,TRUE,,,,[email protected],419755385214,deleted,subject1,419755385214,,360002048693,,2020-09-02 11:01:27,https://zendesk.com/api/v2/tickets/11071.json,email,,,,X,Support,[],,,