v3 (#164)

* upd GetObject * complete config * Do not add to dependency map same address (#109) * config.yaml add authorizer -> reader dependency * GetObject revert ObjectTypeSelector validation logic * REST API RC1 * upd assets (#112) * OpenAPI support (#113) * add OpenAPI support * upd deps * upd whitespace validation * upd whitespace validation * upd whitespace validation * add model service * config json schema * update deps * savepooint 20230913 * gdrive manifest and ds-load (#139) * Initial commit * gdrive it is * Fixing image paths, and updating emails * Add console experience for topaz * Fix linting * Add console cli cmd * Replace 0.0.0.0 with localhost * Remove Println * Get the latest commit from v3 go-edge-ds branch * Use latest v3 console and fix handler paths * Add model service configuration * Only copy the files of the console build folder * Expose default directory gateway port * Topaz with conosole (`console06`) * clean pkg/app/console before build * Bump to latest self-hosted-console * fix migrate test * fix migrate test * add service cleanup * Revert "fix migrate test" This reverts commit 5b5dfd9. * V3 only (#149) * migrate all builtins to be v3 based, adds handling for v2 and v3 request payloads, v2 request return v2 results, help * is always v3 based * migrate and reorganize tests, remove gomega/ginko dependency * add service cleanup, the cleanup function returned by wire does not cleanup, leaving the database open * Remove 'system' object from citadel data (#150) * add manifest files * updated citadel data in v3 branch * upd v2 builtins & getUserFromIdentity * upd topaz test command * upd ci.yaml * ci.yaml add CONSOLE_VERSION * int automatic schema migration * Latest v3 console (resizable panes) * upd-deps (#156) * Prepare changes for config version 2 (#147) * Prepare changes for config version 2 * Update cli config template and exposed ports * Update config doc and examples * Add cleanups for topaz services * Rename topaz service to authorizer * Update testing configs * Update test engine and configs * Refactor topaz config services * Move config version validation before unmarshal * Defer runtime cleanup and bump service-host * Rename authorizer and topaz structs and bump go-edge-ds --------- Co-authored-by: Gert Drapers <[email protected]> * upd go-edge-ds * upd Dockerfile (s) * topaz configure --stdout * add back LSP header * Assets v3 (#158) * assets v3 formatted * assets v3 formatted * go-directory-cli updates * publish ghcr.io/aserto-dev/topaz:v3-latest-test-<platform> * upd go-directory-cli v3 * upd ci to push v3-latest-test * go-directory v0.30.0 * publish-test-image ci task * add docker_manifests entry * add docker_manifests entry * revert migration changes * push container * excl configure policy msg when --stdout * upd displayName to display_name * fix GetObjects * fix GetRelation(s) * Latest console * topaz test output fix * upd CONSOLE_VERSION in ci.yaml * err cleanup (#161) * upd deps * upd deps * assets directory restructure (#160) * console 0.0.0-20231102163131.0.g76203df3 (#163) * console 0.0.0-20231102163131.0.g76203df3 * console 0.0.0-20231102163131.0.g76203df3 * v0.30.0 deps * upd ci --------- Co-authored-by: carabasdaniel <[email protected]> Co-authored-by: Glenn Block <[email protected]> Co-authored-by: oanatmaria <[email protected]> Co-authored-by: Ronen Hilewicz <[email protected]> Co-authored-by: Oana Tanasoiu <[email protected]> Co-authored-by: Ronen Hilewicz <[email protected]> Co-authored-by: Omri Gazitt <[email protected]>
aserto-dev · Nov 4, 2023 · aaa86b4 · aaa86b4
1 parent a23fd9e
commit aaa86b4
Show file tree

Hide file tree

Showing 113 changed files with 121,011 additions and 2,302 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -19,44 +19,43 @@ on:
 env:
   VAULT_ADDR: https://vault.eng.aserto.com/
   PRE_RELEASE: ${{ github.ref == 'refs/heads/main' && 'main' || '' }}
-  GO_VERSION: "1.19"
+  GO_VERSION: "1.20"
+  GO_RELEASER_VERSION: "v1.20.0"
+  GO_LANGCI_LINT_VERSION: "v1.53.3"
+  GO_TESTSUM_VERSION: "1.10.1"
+  CONSOLE_VERSION: "0.0.0-20231102163131.0.g76203df3"
 
 jobs:
   test:
     runs-on: ubuntu-latest
     steps:
       -
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
       -
         name: Setup Go
         uses: actions/setup-go@v4
         with:
           go-version: ${{ env.GO_VERSION }}
-      -
-        name: Setup caching
-        uses: actions/cache@v3
-        with:
-          path: |
-            ~/.cache/go-build
-            ~/go/pkg/mod
-          key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
-          restore-keys: |
-            ${{ runner.os }}-go-
       -
         name: Build
-        run: |
-          go build -o ./bin/topaz ./cmd/topaz/
-          go build -o ./bin/topazd ./cmd/topazd/
+        uses: goreleaser/goreleaser-action@v4
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          distribution: goreleaser
+          version: ${{ env.GO_RELEASER_VERSION }}
+          args: build --clean --snapshot --single-target
       -
         name: Lint
         uses: golangci/golangci-lint-action@v3
         with:
-          version: v1.52.2
+          version: ${{ env.GO_LANGCI_LINT_VERSION }}
+          args: --timeout=30m
       -
         name: Test Setup
         uses: autero1/[email protected]
         with:
-          gotestsum_version: 1.10.0
+          gotestsum_version: ${{ env.GO_TESTSUM_VERSION }}
       -
         name: Test
         run: |
@@ -67,6 +66,7 @@ jobs:
         continue-on-error: true
         with:
           path-to-profile: cover.out
+
   push:
     runs-on: ubuntu-latest
     # when on a branch only push if the branch is main
@@ -75,7 +75,7 @@ jobs:
     steps:
       -
         name: Read Configuration
-        uses: hashicorp/vault-action@v2.5.0
+        uses: hashicorp/vault-action@v2
         id: vault
         with:
           url: https://vault.eng.aserto.com/
@@ -86,24 +86,14 @@ jobs:
             kv/data/github  "DOCKER_PUSH_TOKEN"   | DOCKER_PASSWORD;
             kv/data/github  "READ_WRITE_TOKEN"    | READ_WRITE_TOKEN;
       -
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
       -
         name: Setup Go
         uses: actions/setup-go@v4
         with:
           go-version: ${{ env.GO_VERSION }}
-      -
-        name: Setup caching
-        uses: actions/cache@v3
-        with:
-          path: |
-            ~/.cache/go-build
-            ~/go/pkg/mod
-          key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
-          restore-keys: |
-            ${{ runner.os }}-go-
       -
         name: Setup QEMU
         uses: docker/setup-qemu-action@v2
@@ -142,7 +132,7 @@ jobs:
         uses: goreleaser/goreleaser-action@v4
         with:
           distribution: goreleaser
-          version: v1.20.0
+          version: ${{ env.GO_RELEASER_VERSION }}
           args: release --clean --snapshot
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -155,7 +145,7 @@ jobs:
     steps:
       -
         name: Read Configuration
-        uses: hashicorp/vault-action@v2.5.0
+        uses: hashicorp/vault-action@v2
         id: vault
         with:
           url: https://vault.eng.aserto.com/
@@ -168,24 +158,14 @@ jobs:
             kv/data/github  "ASERTO_TAP"                              | ASERTO_TAP;
             kv/data/gcp     "SERVICE_ACCOUNT_GITHUB_ACTIONS_RELEASE"  | SERVICE_ACCOUNT_GITHUB_ACTIONS_RELEASE;
       -
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
       -
         name: Setup Go
         uses: actions/setup-go@v4
         with:
           go-version: ${{ env.GO_VERSION }}
-      -
-        name: Setup caching
-        uses: actions/cache@v3
-        with:
-          path: |
-            ~/.cache/go-build
-            ~/go/pkg/mod
-          key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
-          restore-keys: |
-            ${{ runner.os }}-go-
       -
         name: Setup QEMU
         uses: docker/setup-qemu-action@v2
@@ -224,7 +204,7 @@ jobs:
           GOOGLE_APPLICATION_CREDENTIALS: /tmp/gs.json
         with:
           distribution: goreleaser
-          version: v1.20.0
+          version: ${{ env.GO_RELEASER_VERSION }}
           args: release --clean
 
       - name: Archive deployment examples
@@ -245,11 +225,11 @@ jobs:
     runs-on: windows-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
       - name: Read Configuration
-        uses: hashicorp/vault-action@v2.5.0
+        uses: hashicorp/vault-action@v2
         id: vault
         with:
           url: ${{ env.VAULT_ADDR }}

diff --git a/.gitignore b/.gitignore
@@ -35,11 +35,15 @@
 .vscode/launch.json
 
 # test config & data files
-config.y?ml
-test.y?ml
-data?.json
-bundle.tar.gz
-*ds.db
+!config.yaml
+./pkg/testing/assets/config.y?ml
+./pkg/testing/assets/test.y?ml
+./pkg/testing/assets/data?.json
+./pkg/testing/assets/bundle.tar.gz
+./pkg/testing/assets/*ds.db
+
+# console static files that get copied at build
+/pkg/app/console/*
 
 # allow test assets
 !pkg/testing/assets/*.db

diff --git a/.golangci.yaml b/.golangci.yaml
@@ -34,7 +34,7 @@ linters-settings:
       - whyNoLint
       - wrapperFunc
   gocyclo:
-    min-complexity: 18
+    min-complexity: 20
   goimports:
     local-prefixes: github.com/golangci/golangci-lint
   golint:

diff --git a/.goreleaser.yml b/.goreleaser.yml
@@ -8,6 +8,7 @@ env:
 before:
   # https://goreleaser.com/customization/hooks/
   hooks:
+      - ./pre-build.sh {{ .Env.CONSOLE_VERSION }}
 
 builds:
   # https://goreleaser.com/customization/build/
@@ -160,7 +161,7 @@ dockers:
       - "--label=org.opencontainers.image.revision={{.FullCommit}}"
       - "--label=org.opencontainers.image.version={{.Version}}"
       - "--label=org.opencontainers.image.source=https://github.com/{{ .Env.ORG }}/{{ .Env.REPO }}"
-      - "--label=org.opencontainers.image.description=Topaz Authorizer"
+      - "--label=org.opencontainers.image.description=Topaz"
       - "--label=org.opencontainers.image.licenses=Apache-2.0"
 
   - use: buildx
@@ -177,7 +178,7 @@ dockers:
       - "--label=org.opencontainers.image.revision={{.FullCommit}}"
       - "--label=org.opencontainers.image.version={{.Version}}"
       - "--label=org.opencontainers.image.source=https://github.com/{{ .Env.ORG }}/{{ .Env.REPO }}"
-      - "--label=org.opencontainers.image.description=Topaz Authorizer"
+      - "--label=org.opencontainers.image.description=Topaz"
       - "--label=org.opencontainers.image.licenses=Apache-2.0"
 
 docker_manifests:

diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -141,6 +141,7 @@
     "protomsg",
     "protoutil",
     "rakyll",
+    "rapidoc",
     "rbuf",
     "registery",
     "rego",

diff --git a/Depfile b/Depfile
@@ -23,4 +23,4 @@ go:
     version: "v1.3.1"
   grpcurl:
     importPath: "github.com/fullstorydev/grpcurl/cmd/grpcurl"
-    version: "v1.8.7"
+    version: "v1.8.8"
diff --git a/Dockerfile b/Dockerfile
@@ -1,39 +1,20 @@
-ARG GO_VERSION
-FROM golang:$GO_VERSION-alpine AS build-dev
-RUN apk add --no-cache bash build-base git tree curl protobuf openssh
-WORKDIR /src
+FROM alpine
 
-ENV GOBIN=/bin
-ENV ROOT_DIR=/src
+RUN apk add --no-cache bash tzdata
 
-# generate & build
-ARG VERSION
-ARG COMMIT
+EXPOSE 8080
 
-COPY . .
-RUN --mount=type=cache,target=/go/pkg/mod \
-    	--mount=type=cache,target=/root/.cache/go-build \
-    	--mount=type=ssh \
-    	go run mage.go deps build
+EXPOSE 8282
+EXPOSE 8383
 
-FROM alpine
-ARG VERSION
-ARG COMMIT
+EXPOSE 9292
+EXPOSE 9393
 
-LABEL org.opencontainers.image.version=$VERSION
-LABEL org.opencontainers.image.source=https://github.com/aserto-dev/topaz
-LABEL org.opencontainers.image.title="Topaz"
-LABEL org.opencontainers.image.revision=$COMMIT
-LABEL org.opencontainers.image.url=https://aserto.com
+EXPOSE 9494
+EXPOSE 9696
 
-RUN apk add --no-cache bash tzdata
 WORKDIR /app
-COPY --from=build-dev /src/dist/topazd_linux_amd64_v1/topazd /app/
 
-EXPOSE 8282
-EXPOSE 8383
-EXPOSE 8484
-EXPOSE 8585
-EXPOSE 9292
+COPY dist/topaz*_linux_amd64_v1/topaz* /app/
 
 ENTRYPOINT ["./topazd"]
diff --git a/Dockerfile.goreleaser b/Dockerfile.goreleaser
@@ -2,14 +2,19 @@ FROM alpine
 
 RUN apk add --no-cache bash tzdata
 
+EXPOSE 8080
+
 EXPOSE 8282
 EXPOSE 8383
-EXPOSE 8484
-EXPOSE 8585
+
 EXPOSE 9292
+EXPOSE 9393
+
+EXPOSE 9494
+EXPOSE 9696
 
 WORKDIR /app
 
-COPY topazd /app/
+COPY topaz* /app/
 
 ENTRYPOINT ["./topazd"]