Skip to content

Commit

Permalink
Merge pull request #4 from firebolt-db/add_account_name
Browse files Browse the repository at this point in the history
Added ability to include account value in profiles.yml to specify whi…
  • Loading branch information
dataders authored Oct 29, 2021
2 parents a80d6d0 + 1f271bf commit d96b0a6
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 31 deletions.
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@

- removed log statements and an extra `firebolt__get_create_index_sql` macro error via [#2](https://github.com/firebolt-db/dbt-firebolt/pull/2)

- added ability to specify an account for users who have more than one account associated with their credentials


## v.0.21.0

### Features
Expand All @@ -21,4 +24,4 @@

### Contributors

- [@ima-hima](https://github.com/ima-hima) and [@swanderz](https://github.com/swanderz)
- [@ima-hima](https://github.com/ima-hima) and [@swanderz](https://github.com/swanderz)
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ For dbt to function with Firebolt you must have an engine connected to your data
1. The engine must be a general-purpose read-write engine.
1. You must have permissions to access the engine.
1. The engine must be running.
1. *If* you're not using the default engine for the database, the name of the engine must be specified.
1. If you're not using the default engine for the database, you must specify an engine name.
1. If there is more than one account associated with your credentials, you must specify an account.

### YAML configuration file
You'll need to add your project to the `profiles.yml` file. These fields are necessary:
Expand All @@ -35,8 +36,9 @@ You'll need to add your project to the `profiles.yml` file. These fields are nec

These fields are optional:

- `engine_name` (See note above.)
- `host` (The host defaults to `api.app.firebolt.io`. If you want to use a dev account you must include the `host` field and set it to `api.dev.firebolt.io`.)
- `engine` (See note above.)
- `host` (Host defaults to `api.app.firebolt.io`. If you would like to use a dev account you must include the `host` field and set it to `api.dev.firebolt.io`.)
- `account` (Please note that this is the account *name*, not the account ID. If `account` is omitted the default account will be used.)

Note that, although the value of `type` is always `firebolt`, it must be included either in `profiles.yml` or in the dbt_project.yml file for your application.

Expand All @@ -54,8 +56,9 @@ my_project:
schema: <my_schema_name>
jar_path: <path_to_my_local_jdbc_jar>
threads: 1
# The following two fields are optional. Please see the notes above.
engine_name: <my_engine_name>
# The following three fields are optional. Please see the notes above.
account: <my_account_name>
engine: <my_engine>
host: api.app.firebolt.io
```

Expand Down
50 changes: 25 additions & 25 deletions dbt/adapters/firebolt/connections.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from contextlib import contextmanager
from dataclasses import dataclass
from typing import Optional, Dict, List, Any, Iterable
import urllib
from urllib.parse import urlencode, quote
import os
import json
import agate
Expand All @@ -22,7 +22,8 @@ class FireboltCredentials(Credentials):
params: Optional[Dict[str, str]] = None
host: Optional[str] = 'api.app.firebolt.io'
driver: str = 'com.firebolt.FireboltDriver'
engine_name: Optional[str] = None
engine: Optional[str] = None
account: Optional[str] = None

@property
def type(self):
Expand All @@ -33,8 +34,8 @@ def _connection_keys(self):
Return list of keys (i.e. not values) to display
in the `dbt debug` output.
"""
return ("host", "user",
"schema", "database", "engine_name",
return ("host", "account", "user",
"schema", "database", "engine",
"jar_path", "params")

@property
Expand All @@ -47,7 +48,7 @@ def unique_field(self):
# Is this safe, or is it too much information. It should only be
# called by `hashed_unique_field()` as stated in the docstring
# but I'm asking here for noting in the PR of this branch.
return self.engine_name
return self.engine

class FireboltConnectionManager(SQLConnectionManager):
"""Methods to implement:
Expand Down Expand Up @@ -81,16 +82,16 @@ def open(cls, connection):
connection.state = "fail"
# If we get a 502 or 503 error, maybe engine isn't running.
if "50" in f"{e}":
if credentials.engine_name is None:
engine_name = 'default'
if credentials.engine is None:
engine = 'default'
error_msg_append = ('\nTo specify a non-default engine, '
'add an engine_name field into the appropriate target in your '
'add an engine field into the appropriate target in your '
'profiles.yml file.')
else:
engine_name = credentials.engine_name
engine = credentials.engine
error_msg_append = ''
raise EngineOfflineException(
f'Failed to connect via JDBC. Is the {engine_name} engine for '
f'Failed to connect via JDBC. Is the {engine} engine for '
+ f'{credentials.database} running? '
+ error_msg_append
)
Expand All @@ -102,22 +103,21 @@ def make_jdbc_url(self, credentials):
jdbc_url = os.path.join("jdbc:firebolt://",
credentials.host,
credentials.database)
# For both engine name and account, if there's not a value specified
# it uses whatever Firebolt has set as default for this DB. So just
# fill in url variables that are not None.
url_vars = {key:quote(getattr(credentials, key).lower())
for key in ['engine', 'account']
if getattr(credentials, key)
}
# If params, then add them, too.
if credentials.params:
jdbc_url += "".join(
map(
lambda kv: "&"
+ urllib.parse.quote(kv[0])
+ "="
+ urllib.parse.quote(kv[1]),
credentials.params.items(),
)
)
if credentials.engine_name:
# If there's not an engine name specified either as an environment
# variable or as an engine_name value in profiles.yml, it uses the
# engine Firebolt has set as default for this DB.
jdbc_url += f"?engine={urllib.parse.quote(credentials.engine_name)}"

url_vars.update({key:quote(credentials.params.items()[key]).lower()
for key in credentials.params.items()
if credentials.params.items()[key]
})
if url_vars:
jdbc_url += "?" + urlencode(url_vars)
return jdbc_url

@contextmanager
Expand Down

0 comments on commit d96b0a6

Please sign in to comment.