Release/0.2.0 (#185)

* Update README.md * Improve architecture page description (#135) * change terms * Add steps * add some space * Update all docstring examples to include disable_tls=True (#138) * Docs updates (#133) * add docsearch * update compatibility matrix * remove hardcoded password from overview * dbhost as string * update architecture diagram * add docsearch * update compatibility matrix * remove hardcoded password from overview * update architecture diagram * update roadmpa * trim overview * update next steps * cursor pointer on hover on landing page * update roadmap * update architecture * update apzm icon * add installation page in sidebar, add docker instructions (#139) * Add --version and --dev flags (#141) * add small logo, update home page title (#140) * add small logo, update home page title * half size icon * nitpicks (#145) * Link to Cyral (#149) * include cyral in header links * link to cyral in footer * remove log * Add Support for PyMySQL (#148) * Fix disable_tls=True * Fix Psycopg2 example * refactor * add pymysql support * Update poetry files * Add PyMySql example * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Add check * rename example * Add pymysql test * Add docstring and using default authclient * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * add pymysql to docs site * Update compatibility pages * Fix typo sdk/python/approzium/pymysql/_connect.py Co-authored-by: Becca Petrin <[email protected]> Co-authored-by: UpGado <[email protected]> Co-authored-by: Becca Petrin <[email protected]> * unindent keywords in code blocks (#151) * cyral -> cyralinc (#152) * docs: Add GitHub Security Policy (#156) Adds a doc on reporting security issues to `.github/SECURITY.md`. This allows GitHub to display information on how to report such issues. When creating an Issue or PR, GitHub will present links to this security policy. The policy is also accessible from the Security tab on the main page of the repo. This mostly just moves the existing information in the README to the appropriate location for this GitHub feature. * Fix typo in SECURITY.md * add google analytics (#158) * Add Support for AWS Secrets Manager (#150) * Add AWS Secrets Manager support * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * add comment * Remove unused import * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * standardize credential manager instantiation * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * style * Update hc_vault_test.go * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Add config flag * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * more helpful error messages * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * fix tests * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * refactor * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Add Asm tests * Pass AWS_REGION to authenticator test service * Update test.yml * Update test.yml * more helpful error message * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Pass AWS_REGION to test container * allow tests to run without prior setup * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * improve error messages * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Add separate authenticator for ASM * Add both authenticator hosts to SDK tests * Seed ASM before running python tests * Update docs * fix link * WIP docs * move comment * Rename config_ to c * lowercase error message Co-authored-by: Becca Petrin <[email protected]> * check for username in stored credentials Co-authored-by: UpGado <[email protected]> Co-authored-by: Becca Petrin <[email protected]> * Add Opentelemetry Integration to Python SDK (#123) * Add approzium.opentelemetry * Add authenticator attribute to Psycopg2 connection * remove unnecessary __init__ * clarify exception message * style fix * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Add opentelemtry.ext.psycopg2 to pyproject.toml * add jaeger to docker-compose * store authclient in approziu.asyncpg connection * Add opentelemtry example * add pg2_opentelemetry.py example * style fix * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * sortout poetry files * fix poetry files * Update dockerfile * update examples * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * add opentelemetry integration page * Add docs for Opentelemetry integration * update readthedocs.yml * add authenticator attribute to MySQL connector connections * rename example * Add attribution_info example * WIP add EC2 instance metadata * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * update lock file * add additional EC2 metadata * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * WIP fix path * Include examples in docs * update example * Add max size to lru_cache Co-authored-by: UpGado <[email protected]> * Clarify Docs for Installing Database Drivers Alongside SDK (#165) * update quickstart.mdx * Update userguide * Fix commands for Zsh * Give direct authenticator instructions (#163) * Add convenience script for generating shas (#167) * Add convenience script for generating shas * Add newline * Add Warnings for Postgres MD5 being Insecure (#169) * Add note about MD5 to Compatibility page * pool -> connection pool * Add note in authenticator.go * Add warning about MD5 in Python SDK user guide * Add gosec Security Checks to CI (#172) * Add gosec step to lint.yml * fix typo * Add GO111MODUE=on for gosec * try again * add #nosec to hash computations * #nosec aws.go * Use filepath.Clean * handle errors in hash functions * handle erors in other places * Update lint.yml * Update lint.yml * Update lint.yml * Update lint.yml * Update lint.yml * Update lint.yml * Update lint.yml * Update lint.yml * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Update lint.yml Co-authored-by: UpGado <[email protected]> * Add Releasing to CI (#173) * Add release.yml * add creating authenticator binaries * add goreleaser in authenticator/ * set workdir * WIP .goreleaser.yml * WIP .goreleaser.yml * WIP .goreleaser.yml * WIP .goreleaser.yml * WIP .goreleaser.yml * WIP .goreleaser.yml * WIP .goreleaser.yml * Add poetry publish step * Try to publish via poetry * Add Docker publish * Update release.yml * WIP * Complete release.yml * Update jpeg-js (#175) * Add Terraform module for deploying to AWS (#168) * Add README.md * Add working Terraform for AWS instance * Include Vault and AWS region flags in config * Set AWS Secrets Manager to use AWS region from config * Set Vault backend to use config fields * Update tests * Automatic lint Signed-off-by: GitHub Actions Bot <[email protected]> * Update configuration.mdx * Restore APPROZIUN_VAULT_TOKEN_PATH to config.go * Update config files * Remove SSH and add GRPC * Remove cidr_blocks * Add filepath.Clean back Co-authored-by: UpGado <[email protected]> * Refactor config parsing (#180) * Refactor config parsing * Handle version flag * Make flags take precedence over env * Correct yaml sample and parsing * Update test env vars * Finish updating AWS_REGION * Support env vars without APPROZIUM_ prefix * Update config docs (#181) * Refactor config parsing * Update config doc * Handle version flag * Make flags take precedence over env * Correct yaml sample and parsing * Update test env vars * Finish updating AWS_REGION * Support env vars without APPROZIUM_ prefix * Update docs to match upstream changes * Make supported selections match above * Fix typo * Fix other typo * Remove file name requirement * Update docs for 0.2.0 release (#187) * Update version number in docs * Update docs * Updates from testing AWS Secrets Manager (#188) * Updates from testing Terraform (#189) * Update Terraform from testing * Move TF instructions to Installation * Improve how Installation looks * Update sdk pyproject * Bump prismjs from 1.20.0 to 1.21.0 in /docs (#191) Bumps [prismjs](https://github.com/PrismJS/prism) from 1.20.0 to 1.21.0. - [Release notes](https://github.com/PrismJS/prism/releases) - [Changelog](https://github.com/PrismJS/prism/blob/master/CHANGELOG.md) - [Commits](PrismJS/prism@v1.20.0...v1.21.0) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ipdate jpeg-js (#192) * Improve readme (#190) * Elaborate on features in README * Typo * Improve command-line usage (#193) * Improve command-line usage * Customize help output * Add maintenance note * Update example config file (#195) Co-authored-by: Dio Gado <[email protected]> Co-authored-by: Timothy Nguyen <[email protected]> Co-authored-by: UpGado <[email protected]> Co-authored-by: Tim O'Guin <[email protected]> Co-authored-by: Dio Gado <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
cyralinc · Aug 11, 2020 · a453cbd · a453cbd
1 parent e9a9007
commit a453cbd
Show file tree

Hide file tree

Showing 82 changed files with 2,384 additions and 619 deletions.
diff --git a/.github/SECURITY.md b/.github/SECURITY.md
@@ -0,0 +1,5 @@
+# Reporting Security Issues
+
+We take Approzium's security and our users' trust very seriously. If you believe you
+have found a security issue in Approzium, _please responsibly disclose_ by contacting
+us at [[email protected]](mailto:[email protected]) before filing any public issues.
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -6,11 +6,21 @@ on:
 jobs:
   lint:
     runs-on: ubuntu-latest
+    env:
+      GO111MODULE: on
     steps:
       - uses: actions/checkout@v2
-
+      - uses: actions/setup-go@v2
+        with:
+          go-version: '1.13'
+
       - name: gofmt
         run: gofmt -s -w .
+
+      - name: gosec
+        uses: securego/gosec@master
+        with:
+          args: ./…  
 
       - name: Setup Python
         uses: actions/setup-python@v1

diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -0,0 +1,51 @@
+name: Release
+on:
+  push:
+    tags:
+      - 'v*'
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    env:
+      GO111MODULE: on
+    steps:
+      - name: Checkout Source
+        uses: actions/checkout@v2
+
+      - uses: actions/setup-go@v2
+        with:
+            go-version: '1.13'
+
+      - name : Get release version
+        id: get_version
+        run: echo ::set-env name=RELEASE_VERSION::$(echo ${GITHUB_REF:10})
+
+      - name: Release Authenticator Binaries
+        uses: goreleaser/goreleaser-action@v1
+        with:
+          version: latest
+          workdir: authenticator
+          args: release --rm-dist
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Publish Python SDK
+        uses: abatilo/[email protected]
+        with:
+            python_version: 3.7.8
+            poetry_version: 1.0
+            working_directory: ./sdk/python/
+            args: publish --build
+        env:
+          POETRY_HTTP_BASIC_PYPI_USERNAME: ${{ secrets.PYPI_USERNAME }}
+          POETRY_HTTP_BASIC_PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+      - name: Publish Authenticator Docker Image
+        uses: elgohr/Publish-Docker-Github-Action@master
+        with:
+          name: approzium/authenticator
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+          tags: "latest,${{ env.RELEASE_VERSION }}"
+          buildoptions: "--target authenticator-build"
+        env:
+          COMPOSE_DOCKER_CLI_BUILD: 1
+          DOCKER_BUILDKIT: 1
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -2,7 +2,6 @@ name: test
 on:
   push:
     branches: [ '*' ]
-
 jobs:
   test:
     runs-on: ubuntu-latest
@@ -23,8 +22,10 @@ jobs:
         run: make run-in-docker CMD="make run-gotests"
         env:
           TEST_ASSUMABLE_ARN: ${{ secrets.TEST_ASSUMABLE_ARN }}
+          AWS_REGION: us-east-2  # this is needed for the AWS Secrets Credentials to work.
 
       - name: Run Python tests
         run: make run-in-docker CMD="make run-pythontests"
         env:
           TEST_ASSUMABLE_ARN: ${{ secrets.TEST_ASSUMABLE_ARN }}
+          AWS_REGION: us-east-2  # this is needed for the AWS Secrets Credentials to work.
diff --git a/.gitignore b/.gitignore
@@ -212,6 +212,11 @@ vendor/
 .srl
 
 # Binaries
-
 authenticator/bin
 authenticator/pkg
+
+# Terraform-related
+terraform/*/.tfvars
+terraform/*/.tfvars.json
+terraform/*/.terraform
+terraform/*/terraform.tfstate*
diff --git a/.readthedocs.yml b/.readthedocs.yml
@@ -22,3 +22,4 @@ python:
         path: sdk/python
         extra_requirements:
             - sqllibs
+            - tracing
diff --git a/Dockerfile b/Dockerfile
@@ -19,6 +19,10 @@ RUN mv protoc3/include/* /usr/local/include/
 RUN wget https://releases.hashicorp.com/vault/1.4.2/vault_1.4.2_linux_amd64.zip
 RUN unzip vault_1.4.2_linux_amd64.zip
 RUN mv vault /usr/local/bin/
+# Intall AWS CLI
+RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
+    unzip awscliv2.zip && \
+    ./aws/install
 RUN apt-get install -y \
   build-essential \
   libpq-dev \
@@ -30,7 +34,7 @@ RUN pip3 install poetry tox
 WORKDIR /usr/src/approzium/sdk/python
 COPY sdk/python .
 RUN poetry run pip install -U pip setuptools
-RUN poetry install --extras "sqllibs"
+RUN poetry install --extras "sqllibs tracing"
 # Build Authenticator Go Binary
 WORKDIR /usr/src/approzium/authenticator
 COPY authenticator/ .

diff --git a/Makefile b/Makefile
@@ -59,11 +59,22 @@ seed-vault-all-addrs:
 	for ADDR in $(TEST_DBADDRS); do \
 		make seed-vault-addr ADDR=$$ADDR; \
 	done
+# ASM uses @ to separate host and port, so replace : with @
+seed-asm-addr:
+	AWS_PAGER="" aws secretsmanager create-secret --name approzium/$(shell echo $(ADDR) | sed "s/:/@/g") \
+		--secret-string '{"$(TEST_DBUSER)": $(vault_secret)}' || true
+	AWS_PAGER="" aws secretsmanager put-secret-value --secret-id approzium/$(shell echo $(ADDR) | sed "s/:/@/g") \
+		--secret-string '{"$(TEST_DBUSER)": $(vault_secret)}'
+
+seed-asm-all-addrs:
+	for ADDR in $(TEST_DBADDRS); do \
+		make seed-asm-addr ADDR=$$ADDR; \
+	done
 
 run-testsuite: run-gotests run-pg2tests
 
 run-gotests:
 	cd authenticator && CGO_ENABLED=1 go test -v -race -p 1 ./...
 
-run-pythontests: enable-vault-path seed-vault-all-addrs
+run-pythontests: enable-vault-path seed-vault-all-addrs seed-asm-all-addrs
 	cd sdk/python && poetry run pytest --workers auto
diff --git a/README.md b/README.md
@@ -4,21 +4,27 @@
 ![lint](https://github.com/cyralinc/approzium/workflows/lint/badge.svg)
 [![Documentation Status](https://readthedocs.org/projects/approzium/badge/?version=latest)](http://approzium.readthedocs.io/?badge=latest)
 
-Approzium provides SDKs that allow you to authenticate to a database without ever having access to its password. Your
-identity is provided through the platform on which you're running.
+Approzium is a tool that provides:
+- Password-less database authentication
+- Authentication through your cloud-provider's built-in identity
+- Highly security-oriented logging and metrics
+
+Its aim is to prevent data breaches, and to help you detect them promptly if they do occur or are attempted.
 
 ----
 
 **Please note**: We take Approzium's security and our users' trust very seriously. If you believe you have found a security issue in Approzium, _please responsibly disclose_ by contacting us at [[email protected]](mailto:[email protected]).
 
+See the [SECURITY](.github/SECURITY.md) guide for more details.
+
 ----
 
 We currently support AWS for identity, and have a Python SDK for Postgres drivers. This project is under active development, please
 do stay tuned for more identity platforms, databases, and SDK languages.
 
 ## Docs
 
-See https://approzium.org/ for a Quick Start, or elaboration on the architecture and API.
+See https://approzium.com/ for a Quick Start, or elaboration on the architecture and API.
 
 ## Support
 

diff --git a/authenticator/.goreleaser.yml b/authenticator/.goreleaser.yml
@@ -0,0 +1,26 @@
+archives:
+    -
+        builds:
+        - authenticator
+        name_template: "{{ .Os }}_{{ .Arch }}"
+        format: zip
+        files:
+            - none*
+builds:
+    -
+        id: "authenticator"
+        goarch:
+            - 386
+            - amd64
+            - arm
+            - arm64
+        goos:
+            - linux
+            - darwin
+            - windows
+            - freebsd
+            - netbsd
+            - solaris
+        ignore:
+            - goos: freebsd
+              goarch: arm64
diff --git a/authenticator/Makefile b/authenticator/Makefile
@@ -4,7 +4,7 @@ dev:
 	go build && go install
 
 test:
-	go test -v -race -p 1 ./...
+	go test -v -race ./...
 
 server:
 	go build .

diff --git a/authenticator/README.md b/authenticator/README.md
@@ -11,7 +11,7 @@ the password.
 We love contributions. To easily develop, in the `authenticator` folder, run `$ make dev`. Then, run the authenticator.
 
 ```
-$ authenticator -dev
+$ authenticator --dev
 ```
 
 It will start the authenticator up on your `localhost` without TLS. Check that it's up by hitting its API.

diff --git a/authenticator/approzium.config.yml b/authenticator/approzium.config.yml
@@ -1,6 +1,18 @@
-# host: 127.0.0.0
-# httpport: 6000
-# grpcport: 6001
-# loglevel: info
-# logformat: text
-# lograw: false
+# Example authenticator config file
+---
+listener:
+  grpc_port: 6001
+  host: "127.0.0.1"
+  http_port: 6000
+logging:
+  log_format: text
+  log_level: info
+  log_raw: false
+secrets:
+  secrets_manager: "vault1"
+  vault_addr: "https://somewhere:8200"
+  vault_token_path: "/path/to/tokensink.txt"
+tls:
+  disable_tls: true
+  tls_cert_path: "/path/to/approzium.pem"
+  tls_key_path: "/path/to/approzium.key"
diff --git a/authenticator/go.mod b/authenticator/go.mod
@@ -12,8 +12,6 @@ require (
 	github.com/google/uuid v1.1.1
 	github.com/hashicorp/vault/api v1.0.4
 	github.com/sirupsen/logrus v1.6.0
-	github.com/spf13/pflag v1.0.5
-	github.com/spf13/viper v1.7.0
 	go.opencensus.io v0.22.4
 	golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
 	golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e // indirect
-Original file line number
+Diff line change
@@ Expand Up / @@ -22,3 +22,4 @@ python: @@
             path: sdk/python
             extra_requirements:
                 - sqllibs
+                - tracing