Terminus docs

blog/2022-07-26-developer-first-security-sucks-automate-product-security.md

@@ -27,7 +27,7 @@ On the other hand:
 - Almost 60% of devs are releasing code 2x faster, thanks to DevOps.
 - In 2021, only 20% of organizations have fully integrated security into the development
 - Security has low priority. 67% of developers surveyed by Secure Code Warrior admitted that they routinely left known vulnerabilities and exploits in their code
-- Github expects the number of software developers using its platform (56 million in 2020), to grow to 100 million developers in 2025
+- GitHub expects the number of software developers using its platform (56 million in 2020), to grow to 100 million developers in 2025
 
 (Invicti Security, Gitlab, GitHub, VentureBeat)
 

blog/2022-09-27-enterprise-ready-saas-starter-kit.md

@@ -59,7 +59,7 @@ Now, let's take a look at the other conventional features that the SaaS kit prov
 - Sign in with Magic Link
 - Sign in with SAML SSO
 - Sign in with Google [[Setting up Google OAuth](https://support.google.com/cloud/answer/6158849?hl=en)]
-- Sign in with GitHub [[Creating a Github OAuth App](https://docs.github.com/en/developers/apps/building-oauth-apps/creating-an-oauth-app)]
+- Sign in with GitHub [[Creating a GitHub OAuth App](https://docs.github.com/en/developers/apps/building-oauth-apps/creating-an-oauth-app)]
 - Directory Sync (SCIM)
 - Update account
 - Create team

blog/2023-03-13-exploring-the-open-source-business-model-and-how-companies-monetize-it.md

@@ -23,7 +23,7 @@ There are many different ways that open-source companies can monetize - ultimate
 
 ### Donations
 
-One of the most popular models is to offer the source code and documentation completely free and let its users donate at their discretion. This is normally done for smaller projects and donations can be solicited in various ways, such as a button on the website, a link in a newsletter, a Github donation, or one that I like - buymeacoffee.com. The latter allows you to embed an option into your website or interface and donate at the value of a coffee. Although donations are a great way to monetize some projects, this method is not feasible for bigger companies that have complex solutions and large overheads.
+One of the most popular models is to offer the source code and documentation completely free and let its users donate at their discretion. This is normally done for smaller projects and donations can be solicited in various ways, such as a button on the website, a link in a newsletter, a GitHub donation, or one that I like - buymeacoffee.com. The latter allows you to embed an option into your website or interface and donate at the value of a coffee. Although donations are a great way to monetize some projects, this method is not feasible for bigger companies that have complex solutions and large overheads.
 
 ### Support
 

blog/2023-09-15-navigating-the-open-source-landscape-finding-your-first-contribution.md

@@ -34,18 +34,18 @@ Numerous platforms and directories host a plethora of open-source projects, maki
 
 Now that we have a clearer direction on what sparks your interests, let's take a look at how to find open-source projects that you can start contributing to.
 
-![Github dashboard](/img/blog/navigating-open-source-blog/git-dashboard.png)
+![GitHub dashboard](/img/blog/navigating-open-source-blog/git-dashboard.png)
 _Click on the GitHub icon menu in the top left corner._
 
 If we search for open-source we get a long list back but we have the ability to explore trending repositories, search by programming language, and even filter by topics of interest.
 
-![Github explore](/img/blog/navigating-open-source-blog/git-explore.png)
+![GitHub explore](/img/blog/navigating-open-source-blog/git-explore.png)
 
 ## Utilize Keywords and Tags
 
 When searching on platforms like GitHub, utilize relevant keywords and tags. For instance, if you're interested in TypeScript web development, you might use tags like "typescript", “javascript”, "react" and, "nextjs" to narrow down your search.
 
-![Github languages](/img/blog/navigating-open-source-blog/git-languages.png)
+![GitHub languages](/img/blog/navigating-open-source-blog/git-languages.png)
 
 If you would like to narrow your search even further and combine keywords, just wrap them in []. For example, if I want to make the search extremely narrow and I am looking for a project to contribute to with both "nextjs" and "python"?
 Let's wrap [nextjs], [python] in the search and see what comes back.
@@ -84,13 +84,13 @@ Once you've found a project that piques your interest, assess its **health** and
 
 You can also make sure the project is still alive and well by clicking on the **activity** link.
 
-![Github activity](/img/blog/navigating-open-source-blog/github-activity-link.png)
+![GitHub activity](/img/blog/navigating-open-source-blog/github-activity-link.png)
 
 ## Community engagement
 
 A good place to start is to look for project “Discussions” or an external community such as Slack or Discord.
 
-![Github communities](/img/blog/navigating-open-source-blog/git-community.png)
+![GitHub communities](/img/blog/navigating-open-source-blog/git-community.png)
 
 Engage with the project's community by introducing yourself, asking questions, and expressing your interest in contributing. Being an active part of the community enhances your chances of finding a suitable project, not to mention it’s a great way to network and make friends.
 

docs/index.mdx

@@ -74,5 +74,20 @@ Accelerate your Time to Market while enhancing your company's security! BoxyHQ
         </div>
       </Link>
     </div>
+    <div className="col col--6">
+      <Link
+        className="card"
+        to="/docs/terminus/overview"
+        style={{ height: '100%' }}
+      >
+        <div className="card__body">
+          <h4>Terminus (Beta)</h4>
+          <p>
+            Terminus helps organizations comply with data privacy regulations
+            with an easy to integrate no-code API based solution.
+          </p>
+        </div>
+      </Link>
+    </div>
   </div>
 </div>

docs/terminus/architecture/index.md

@@ -0,0 +1,35 @@
+# Components and Architecture
+
+Terminus can be generally represented as the following:
+
+## ![General Diagram](./general.png)
+
+Terminus is by design, a multi-component decoupled system where data is protected from each of the components.
+Nothing travels in the clear unless explicitly set, thus enabling the implementation of the most stringent data privacy and data segregation policies. The data stored in the vault is encrypted as specified in the proxy and has no visibility on it
+
+From an architectural point of view, Terminus is made of the following components having the following responsibilities:
+
+## [Proxy Service](./proxy.md)
+
+- Exposes a public facing API for storage and retrieval of data
+- Via no code UI, allows the setup and configuration of the business data models to be encrypted and masked
+- Implements the access control policies (ACP), including authentication, authorization, role mapping and granular access
+- Performs the relevant encryption/decryption/masking operations
+- Interacts with external KMSs enabling BYOK models
+
+## Proxy Persistence
+
+- Stores and retrieves data related to business models and product/tenant configuration
+
+## [Vault Service](./vaultservice.md)
+
+- Exposes a private API primitive for the proxy to store and retrieve data
+- Enables an additional layer of encryption. Anything that travels to the vault, beyond the business encryption policies gets encrypted at the vault level
+- It has no knowledge or understanding of business models, treating the incoming and outgoing data completely transparently
+
+## [Persistence vault](./vault.md)
+
+- Stores and retrieves data
+- Has no knowledge of whatever payloads are stored internally
+
+The different services can be compiled and run independently. The easier way of getting started is by running them with the Docker Compose file provided using the official published images.

docs/terminus/architecture/proxy.md

@@ -0,0 +1,211 @@
+# Proxy Service
+
+The proxy service is the outermost client facing of Terminus. Amongst it's roles and responsibilities:
+
+- Enables no-code configuration of encryption and masking business data models
+- Supports multi-tenancy and multi-product logical segregation via API URL paths.
+
+> :warning: **IMPORTANT**: Terminus only supports currently one business data model (alpha version), therefore all tenants/products use the same business and encryption models and configurations
+
+- It exposes the public API for clients to interface with the privacy vault
+- Interfaces with the Vault service
+- Implements Access Control Policies
+- Interacts with external KMSs enabling BYOK models
+
+It is written in [Go](https://go.dev/) and can be accessed by default here [http://localhost:3002](http://localhost:3002)
+
+---
+
+## Data Modeling, Encryption and Masking UI
+
+Data modeling is done via a visual interface that allows the composition via puzzle pieces.
+
+> :warning: **IMPORTANT**: changes on encryption policies on fields of stored datasets will most likely render the existing data useless due to the impossibility of interpret and/or decrypt the data. Thread carefully. Adding and removing fields is allowed.
+
+### Model Management
+
+- Terminus supports different business models - one per product. In order to select the relevant model, enter it in the input box below and click on the "Retrieve Model" button.
+
+![Sample Model: Passport](./proxy_assets/ProductModel.png)
+
+- In case the product doesn't exist, a sample model will be loaded as a starting point and once published, the relevant product will have the model associated.
+- When publishing the model, it will be validated on the server-side.
+
+> :warning: **IMPORTANT**: The server will reject the publishing of an invalid model with a 400 Error code. TODO - needs to be handled in the UI
+
+> :warning: **IMPORTANT**: Currently the proxy service keeps a historical of all the published model versions. In future releases it will be possible to manage such historical versions and rollback to previous business models. Currently it is not exposed via the visual interface.
+
+### Sample Model - Product Model
+
+- Below is an illustrated an example of the shallow model of a Passport, with its individual fields, restrictions on the type, type of encryption and decryption to be applied and the masking to apply once data is read.
+
+    **NOTE**: Future versions will include access-role-based masking policies and multitenancy on the proxy side, allowing for different clients to manage their own models.
+
+    ![Sample Model: Passport](./proxy_assets/SampleModel.png)
+
+    **NOTE**: all fields are currently treated as strings. No numerical types are supported yet.
+
+### Field Restrictions
+
+- The below illustration shows the current supported restrictions on the values. Behind the scenes it is implemented via regular expressions.
+
+    ![Field Type Options](./proxy_assets/FieldOptions.png)
+
+#### String
+
+- A special type of field is a string. Terminus configuration is based on [CUE lang](https://cuelang.org). This type of field permits the inclusion of specific patterns that CUE lang supports (enumerations, regular expressions, etc.)
+
+    ![Field Type String](./proxy_assets/String.png)
+
+    For example, the field "PlaceOfBirth":
+
+    `string`: would allow any string to be inputed
+
+    `string | !="" | *"NONE"`: would allow any non empty string with a default value of `NONE`
+
+### Encryption
+
+- Following are the planned encryption types to be supported by Terminus. Currently terminus supports AES Encryption with a 32 bytes key.
+
+![Encryption Options](./proxy_assets/EncryptionOptions.png)
+
+### Masking
+
+- Terminus supports currently generic and redact masking policies.
+
+![Masking Options](./proxy_assets/MaskOptions.png)
+
+---
+
+## Configuration
+
+### Running via Docker Compose
+
+When running on docker, all the environment variables are specified either as environment variables or in the `.env` file.
+
+**When running via `docker compose` the below does not apply.**
+
+### Default vs Runtime
+
+The Proxy Service ships with default configurations including service components configurations as well as a default business model (the Passport) for reference.
+The `default` configuration ships in the `./conf/default` directory and it is copied on startup on the `./conf/runtime` directory IF such directory is empty.
+
+The `./conf/runtime` directory is where the actual configurations are read from. That includes the UI business models.
+
+The configuration files are as follows:
+
+### model.cue and blockly.tmp
+
+Both files are generated by the UI and contain the necessary information about the business models, encryption policies and masking policies. DO NOT EDIT MANUALLY.
+
+### terminusProxyConf.cue
+
+CUE lang based configuration file containing the necessary variables. These can be set both in this file or as ENVIRONMENT variables.
+**If set as environment variables, those will take precedence over the ones defined in the configuration default/runtime files.**
+
+### access.cue
+
+Defines:
+
+- The access control policies to the APIs via CUE lang definition. Multiple ACPs can be defined, but only the one assigned to the `defaultPolicy` will be used.
+
+```
+///////////////////////////////////////////////////////////////////////
+// API Definitions
+///////////////////////////////////////////////////////////////////////
+#apiSet1: acp.#APIsSet & {
+ version: "v1"
+ apis: {
+  vaultStatus: {
+   path: "/status"
+   methods: [ "GET" ]
+   permissions: [ "READ", "ADMIN", "WRITE" ]
+  }
+  publishModel: {
+   path: "/admin/model"
+   methods: [ "POST" ]
+   permissions: [ "ADMIN" ]
+  }
+  registerTenantProductDEK: {
+   path: "/admin/{tenant}/{product}/dek"
+   methods: [ "POST" ]
+   permissions: [ "ADMIN" ]
+  }
+  getModel: {
+   path: "/admin/model"
+   methods: [ "GET" ]
+   permissions: [ "ADMIN" ]
+  }
+  createVaultEntries: {
+   path: "/{tenant}/{product}/data"
+   methods: [ "POST" ]
+   permissions: [ "WRITE" ]
+  }
+  readOneVaultEntry: {
+   path: "/{tenant}/{product}/data/read/{type}/{token}"
+   methods: [ "GET" ]
+   permissions: [ "READ" ]
+  }
+  readVaultEntryMultiple: {
+   path: "/{tenant}/{product}/data/read/{type}"
+   methods: [ "POST" ]
+   permissions: [ "READ" ]
+  }
+ }
+}
+
+// this is the policy the systtem will read
+defaultPolicy: #apiSet1
+
+///////////////////////////////////////////////////////////////////////
+// Policies
+///////////////////////////////////////////////////////////////////////
+
+// The request is filled by the calling system
+apiRequest: acp.#APIRequest & {}
+
+// Sample policy
+access: acp.#AuthResponse & {
+ allowed: acp.#defFalse
+ request: apiRequest
+ for permission in defaultPolicy.apis[request.apiName].permissions {
+  if list.Contains(apiKeysAndPermissions[request.access_token], permission) {
+   allowed: true
+  }
+ }
+}
+
+```
+
+**NOTES**:
+
+- the paths in the API definitions are values accepted by Go's [Gorilla Mux's](https://github.com/gorilla/mux) specification.
+- the names in the API definitions (i.e. `vaultStatus`) are mapped via code to the relevant handling functions.
+- the `version` is used as part of the path. An example of the status endpoint served by a proxy running locally on port 3002 would be:
+    [http://127.0.0.1:3002/v1/status](http://127.0.0.1:3002/v1/status)
+
+---
+---
+
+- The API Keys and their role-based mapping
+
+```
+///////////////////////////////////////////////////////////////////////
+// API Keys
+///////////////////////////////////////////////////////////////////////
+apiKeysAndPermissions: acp.#AuthTokens & {
+ readAPIKey1: [ "READ" ]
+ writeAPIKey1: [ "WRITE" ]
+ adminAPIKey1: [ "ADMIN" ]
+ allAPIKey1: [ "READ", "WRITE", "ADMIN" ]
+ readwriteAPIKey1: [  "READ","WRITE" ]
+}
+```
+
+**NOTE**: this will be removed and implemented either as an external service or fully integrated in an OPA or similar access control framework.
+
+---
+
+## API
+
+[Swagger API](https://github.com/boxyhq/terminus/blob/main/swagger/proxy/swagger.json)

docs/terminus/architecture/vault.md

@@ -0,0 +1,51 @@
+# Persistence Vault
+
+The persistence vault is the persistent layer where data is stored. Amongst it's roles and responsibilities:
+
+- The persistent store implementation is one of the ones supported by [GORM](https://gorm.io/)
+  - Terminus ships by default with a PostgreSQL data store
+- Supports multi-tenancy and multi-product logical segregation via column keys
+- All data is encrypted and there is no indication of what the data stored is
+- It exposes the low-level API for the Proxy to interface with the persistence
+- Can implement additional encryption at rest if supported by the store implementation
+- It is not business model aware
+- It's scalability, partitioning and indexing capabilities is directly depending on the choice of store
+
+It is written in [Go](https://go.dev/).
+
+---
+
+## Interaction with the Persistence Vault
+
+Leverages [GORM](https://gorm.io/) to interface with the database. Currently is configured to use PostgreSQL but supports any persistence store supported by GORM.
+
+Implements AES encryption against the data store.
+
+> :warning: **IMPORTANT**: changes on encryption keys of the logical encryption layer will render the data stored useless.
+
+---
+
+## Configuration
+
+The specific configuration depends on the data store.
+
+By default it is PostgreSQL and gets configured via Docker Compose and environment variables.
+
+---
+
+## Schema
+
+Terminus' schema is transparently deployed by the Vault Service layer via [GORM](https://gorm.io/)
+
+It comprises one data table with the following structure:
+
+![Entries Table](./vault_assets/er.png)
+
+![Entries Table Columns](./vault_assets/store_columns.png)
+
+With:
+
+- `token` being the entry identifier (it's the ID that gets returned on creation and is required for retrieval)
+- `value` being where the data is stored transparently
+- `content_type` being the mime-type of the stored data for retrieval (json, xml, binary, etc.)
+- `tenant`, `product` being the keys for multitenancy indicating Terminus' customer and the configured product