Skip to content

Commit

Permalink
adding draft contributing file for redshift repo (#59)
Browse files Browse the repository at this point in the history
* adding draft contributing file for redshift repo

* updating contributing file

* making updates after mila's initial review

* updating

* adding more defined changes to test.env

* typo fix in dev-requirements -> dev_requirements

* updating test.env.examples based on feedback, changing dev_requiremnts -> dev-requirements to match dbt-core

* rename dev_requirement -> dev-requiremnent

* run pre-commit

* updating test.env.example and contrubiuting.md after feedback from @leah

* change the : to = so that when the values are copied to the test.env for actual use they don't have to be modified

Co-authored-by: Doug Beatty <[email protected]>

* updating to main

* dev_requirements name update

* adding example for running functional tests in contributing.md

* Empty-Commit

Co-authored-by: Doug Beatty <[email protected]>
  • Loading branch information
McKnight-42 and dbeatty10 authored Jul 7, 2022
1 parent 02daa7b commit 86bbe5f
Show file tree
Hide file tree
Showing 7 changed files with 119 additions and 5 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:
- name: Install python dependencies
run: |
python -m pip install --user --upgrade pip
python -m pip install -r dev_requirements.txt
python -m pip install -r dev-requirements.txt
python -m pip --version
pre-commit --version
mypy --version
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/version-bump.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ jobs:
- name: Bumping version
run: |
source env/bin/activate
python -m pip install -r dev_requirements.txt
python -m pip install -r dev-requirements.txt
env/bin/bumpversion --allow-dirty --new-version ${{steps.variables.outputs.VERSION_NUMBER}} major
git status
Expand Down
105 changes: 105 additions & 0 deletions CONTRIBUTING.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Contributing to `dbt-redshift`

1. [About this document](#about-this-document)
3. [Getting the code](#getting-the-code)
5. [Running `dbt-redshift` in development](#running-dbt-redshift-in-development)
6. [Testing](#testing)
7. [Updating Docs](#updating-docs)
7. [Submitting a Pull Request](#submitting-a-pull-request)

## About this document
This document is a guide intended for folks interested in contributing to `dbt-redshift`. Below, we document the process by which members of the community should create issues and submit pull requests (PRs) in this repository. It is not intended as a guide for using `dbt-redshift`, and it assumes a certain level of familiarity with Python concepts such as virtualenvs, `pip`, python modules, filesystems, and so on. This guide assumes you are using macOS or Linux and are comfortable with the command line.

For those wishing to contribute we highly suggest reading the [dbt-core](https://github.com/dbt-labs/dbt-core/blob/main/CONTRIBUTING.md), if you haven't already. Almost all of the information there is applicable to contributing here, too!

### Signing the CLA

Please note that all contributors to `dbt-redshift` must sign the [Contributor License Agreement](https://docs.getdbt.com/docs/contributor-license-agreements) to have their Pull Request merged into an `dbt-redshift` codebase. If you are unable to sign the CLA, then the `dbt-redshift` maintainers will unfortunately be unable to merge your Pull Request. You are, however, welcome to open issues and comment on existing ones.


## Getting the code

You will need `git` in order to download and modify the `dbt-redshift` source code. You can find direction [here](https://github.com/git-guides/install-git) on how to install `git`.

### External contributors

If you are not a member of the `dbt-labs` GitHub organization, you can contribute to `dbt-redshift` by forking the `dbt-redshift` repository. For a detailed overview on forking, check out the [GitHub docs on forking](https://help.github.com/en/articles/fork-a-repo). In short, you will need to:

1. fork the `dbt-redshift` repository
2. clone your fork locally
3. check out a new branch for your proposed changes
4. push changes to your fork
5. open a pull request against `dbt-labs/dbt-redshift` from your forked repository

### dbt Labs contributors

If you are a member of the `dbt Labs` GitHub organization, you will have push access to the `dbt-redshift` repo. Rather than forking `dbt-redshift` to make your changes, just clone the repository, check out a new branch, and push directly to that branch.


## Running `dbt-redshift` in development

### Installation

First make sure that you set up your `virtualenv` as described in [Setting up an environment](https://github.com/dbt-labs/dbt-core/blob/HEAD/CONTRIBUTING.md#setting-up-an-environment). Ensure you have the latest version of pip installed with `pip install --upgrade pip`. Next, install `dbt-redshift` latest dependencies:

```sh
pip install -e . -r ./dev-requirements.txt
```

When `dbt-redshift` is installed this way, any changes you make to the `dbt-redshift` source code will be reflected immediately in your next `dbt run` command that uses `dbt-redshift`.

To confirm you have correct `dbt-core` and adapter versions installed please run `dbt --version` and `which dbt` to check the correct executable path you wish to use for `dbt-core` is in your current virtualenv.


## Testing

### Initial Setup

`dbt-redshift` contains [unit](https://github.com/dbt-labs/dbt-redshift/tree/main/tests/unit) and [integration](https://github.com/dbt-labs/dbt-redshift/tree/main/tests/integration) tests. Integration tests require testing against an actual Redshift warehouse. We have CI set up to test against a Redshift warehouse during PR checks.

In order to run integration tests locally, you will need a `test.env` file in the root of the repository that contains credentials for your Redshift warehouse.

Note: This `test.env` file is git-ignored, but please be extra careful to never check in credentials or other sensitive information when developing. To create your `test.env` file, copy the provided example file, then supply your relevant credentials.

```
cp test.env.example test.env
$EDITOR test.env
```

### Test commands
There are a few methods for running tests locally.

#### `tox`
`tox` takes care of managing Python virtualenvs and installing dependencies in order to run tests. You can also run tests in parallel. For example, you can run unit tests for Python 3.7, Python 3.8, Python 3.9, Python 3.10, and `flake8` checks in parallel with `tox -p`. Also, you can run unit tests for specific python versions with `tox -e py37`. The configuration of these tests are located in `tox.ini`.

#### `pytest`
Finally, you can also run a specific test or group of tests using `pytest` directly. With a Python virtualenv active and dev dependencies installed you can do things like:

```sh
# run specific redshift integration tests
python -m pytest -m profile_redshift tests/integration/adapter_methods_test
# run specific redshift functional tests in a file
python -m pytest tests/functional/adapter/test_basic.py
# run all unit tests in a file
python -m pytest tests/unit/test_redshift_adapter.py
# run a specific unit test
python -m pytest tests/unit/test_redshift_adapter.py::TestRedshiftAdapter::test_convert_date_type
```
## Updating Docs

Many changes will require an update to the `dbt-redshift` docs. If so, here are some useful resources to find where the current behavior is documented.

- Docs are [here](https://docs.getdbt.com/).
- The docs repo for making changes is located [here]( https://github.com/dbt-labs/docs.getdbt.com).
- The changes made are likely to impact one or both of [Redshift Profile](https://docs.getdbt.com/reference/warehouse-profiles/redshift-profile), or [Redshift Configs](https://docs.getdbt.com/reference/resource-configs/redshift-configs).
- We ask every community member who makes a user-facing change to open an issue or PR regarding doc changes.



## Submitting a Pull Request

dbt Labs provides a CI environment to test changes to the `dbt-redshift` adapter and periodic checks against the development version of `dbt-core` through Github Actions.

A `dbt-redshift` maintainer will review your PR. They may suggest code revision for style or clarity, or request that you add unit or integration test(s). These are good things! We believe that, with a little bit of help, anyone can contribute high-quality code.

Once all tests are passing and your PR has been approved, a `dbt-redshift` maintainer will merge your changes into the active development branch. And that's it! Happy developing :tada:
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
.PHONY: dev
dev: ## Installs adapter in develop mode along with development depedencies
@\
pip install -r dev_requirements.txt && pre-commit install
pip install -r dev-requirements.txt && pre-commit install

.PHONY: mypy
mypy: ## Runs mypy against staged changes for static type checking.
Expand Down
File renamed without changes.
9 changes: 9 additions & 0 deletions test.env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Note: Make sure you have a Redshift account that is set up so these fields are easy to complete.

### Test Environment field definitions
# These will all be gathered from account information or created by you.
# Endpoint for Redshift connection
REDSHIFT_TEST_HOST=
# Username on your account
REDSHIFT_TEST_USER=
# Password for Redshift account
REDSHIFT_TEST_PASS=
# Local port to connect on
REDSHIFT_TEST_PORT=
# Name of Redshift database in your account to test against
REDSHIFT_TEST_DBNAME=
4 changes: 2 additions & 2 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ skip_install = true
passenv = DBT_* PYTEST_ADDOPTS
commands = {envpython} -m pytest {posargs} tests/unit
deps =
-rdev_requirements.txt
-rdev-requirements.txt
-e.

[testenv:{integration,py37,py38,py39,py310,py}-{redshift}]
Expand All @@ -19,5 +19,5 @@ commands =
redshift: {envpython} -m pytest {posargs} -m profile_redshift tests/integration
redshift: {envpython} -m pytest {posargs} tests/functional
deps =
-rdev_requirements.txt
-rdev-requirements.txt
-e.

0 comments on commit 86bbe5f

Please sign in to comment.