Skip to content

Commit

Permalink
Add docs for v1.2 (#432)
Browse files Browse the repository at this point in the history
Signed-off-by: Daniel Weiße <[email protected]>
  • Loading branch information
daniel-weisse committed Jun 27, 2023
1 parent 47183db commit 313b442
Show file tree
Hide file tree
Showing 40 changed files with 4,007 additions and 0 deletions.
37 changes: 37 additions & 0 deletions docs/versioned_docs/version-1.2/_media/coordinator_deployment.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
53 changes: 53 additions & 0 deletions docs/versioned_docs/version-1.2/_media/marble_deployment.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
41 changes: 41 additions & 0 deletions docs/versioned_docs/version-1.2/_media/overview.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
36 changes: 36 additions & 0 deletions docs/versioned_docs/version-1.2/_media/service_mesh.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
37 changes: 37 additions & 0 deletions docs/versioned_docs/version-1.2/_media/verify_cluster.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 20 additions & 0 deletions docs/versioned_docs/version-1.2/building-services/ego.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Building a service: EGo
To get your Go service ready for MarbleRun, we provide two solutions. With [Transparent TLS (TTLS)](../features/transparent-TLS.md) you are able to use your existing applications without any code changes. Alternatively, you can adapt and recompile your application to manually handle the TLS credentials. Details are given in the following.

## TTLS
Simply follow the steps on [adding a service](../workflows/add-service.md). No code changes and no recompiling needed.

## Manual TLS credentials handling

If your service already uses TLS and gets the credentials from, e.g., a file, you just need to [adapt the manifest](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials). Otherwise, you need to make small code changes.

We provide a convenience package called [`github.com/edgelesssys/ego/marble`](https://pkg.go.dev/github.com/edgelesssys/ego/marble#GetTLSConfig). With it, a service can automatically get and use its MarbleRun TLS credentials. The following gives an example.
```Go
serverCfg, err := marble.GetTLSConfig(false)
if err != nil {
log.Fatalf("Failed to retrieve server TLS config")
}
// use serverCfg, e.g., to create an HTTPS server
```

Finally, you need to re-build your service for the enclave environment with `ego-go` and sign it with `ego sign`. Please follow the build instructions for Go provided in our [Go sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloworld).
164 changes: 164 additions & 0 deletions docs/versioned_docs/version-1.2/building-services/gramine.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
# Building a service: Gramine

Running a Gramine app with MarbleRun requires some changes to its manifest. These are explained in the following. See also the [hello world example](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-hello) for a simple introduction, or the [nginx](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-nginx) and [Redis](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-redis) examples for more detailed applications.

## Requirements

First, get Gramine up and running. Gramine is available [as a Debian package](https://github.com/gramineproject/gramine/releases). Alternatively you can follow either the [Building](https://gramine.readthedocs.io/en/latest/devel/building.html) or [Cloud Deployment](https://gramine.readthedocs.io/en/latest/installation.html) guide to build and install Gramine from source.

Before running your application, make sure you got the prerequisites for ECDSA remote attestation installed on your system. You can collectively install them with the following command:

```sh
sudo apt install libsgx-quote-ex-dev
```

## Configuration

### Entrypoint and argv

We provide the `premain-libos` executable with the [MarbleRun Releases](https://github.com/edgelesssys/marblerun/releases). It will contact the Coordinator, set up the environment, and run the actual application.

Set the premain executable as [the entry point](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#libos-entrypoint) of the Gramine application and place the actual entry point [in argv0](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#command-line-arguments):

```toml
libos.entrypoint = "file:premain-libos"

# argv0 needs to contain the name of your executable
loader.argv = ["hello"]

# add the premain to the list of trusted files
sgx.trusted_files = [
# ...
"file:premain-libos"
]
```

After the premain is done running, it will automatically spawn your application.

### Host environment variables

By default, environment variables from the host won't be passed to the application.
Gramine allows to [pass through whitelisted environment variables from the host](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#environment-variables).
The premain needs access to the following [environment variables for configuration](../workflows/add-service.md#step-3-start-your-service):

```toml
loader.env.EDG_MARBLE_TYPE = { passthrough = true }
loader.env.EDG_MARBLE_COORDINATOR_ADDR = { passthrough = true }
loader.env.EDG_MARBLE_UUID_FILE = { passthrough = true }
loader.env.EDG_MARBLE_DNS_NAMES = { passthrough = true }
```

### UUID file

The Marble must be able to store its UUID:

```toml
sgx.allowed_files = [
# ...
"file:uuid"
]
```

### Remote attestation

The Marble will send an SGX quote to the Coordinator for remote attestation using [DCAP attestation](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#attestation-and-quotes):

```toml
sgx.remote_attestation = "dcap"
```

### Enclave size and threads

The premain process is written in Go. The enclave needs to have enough resources for the Go runtime:

```toml
sgx.enclave_size = "1024M"
sgx.thread_num = 16
```

If your application has high memory demands, you may need to increase the size even further.

### Secret files

A Marble's secrets, e.g. a certificate and private key, can be provisioned as files. You can utilize Gramine's in-memory filesystem [`tmpfs`](https://gramine.readthedocs.io/en/latest/manifest-syntax.html#fs-mount-points), so the secrets will never show up on the host's file system :

```toml
fs.mounts = [
# ...
{ type = "tmpfs", path = "/secrets" },
# ...
]
```

You can specify the files' content in the MarbleRun manifest:

```javascript
...
"Parameters": {
"Files": {
"/secrets/server.crt": "{{ pem .Secrets.serverCert.Cert }}",
"/secrets/server.key": "{{ pem .Secrets.serverCert.Private }}"
}
}
...
```

Gramine also allows to store files [encrypted on the host's file system](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#encrypted-files).

```toml
fs.mounts = [
# ...
{ type = "encrypted", path = "/secrets", uri = "file:/path/to/local/directory", key_name = "[KEY_NAME]" },
# ...
]
```

Gramine provides access to a pseudo filesystem for [setting the encryption key](https://gramine.readthedocs.io/en/v1.3/attestation.html#low-level-dev-attestation-interface).
MarbleRun can set up your enclave with keys at runtime by specifying them in the MarbleRun manifest:

```javascript
...
"Parameters": {
"Files": {
"/dev/attestation/[KEY_NAME]": "{{ raw .Secrets.encryptedFilesKey }}"
}
}
...
```

You can see how this is done in the [nginx example](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-nginx).

## Troubleshooting

### aesm_service returned error: 30

If you receive the following error message on launch:

```sh
aesm_service returned error: 30
load_enclave() failed with error -1
```

Make sure you installed the Intel AESM ECDSA plugins on your machine. You can do this by installing the `libsgx-quote-dev` package mentioned in the requirements above.

If you are running your application in a container, you will need to mount the aesm socket. The socket is located at `/var/run/aesmd/`.

If you are deploying your application on Kubernetes with the [Intel SGX device plugin](https://intel.github.io/intel-device-plugins-for-kubernetes/cmd/sgx_plugin/README.html) installed, the socket is automatically mounted by setting the `sgx.intel.com/quote-provider: aesmd` annotation for your deployment:

```yaml
apiVersion: v1
kind: Pod
metadata:
name: gramine-marble
labels:
marblerun/marbletype: gramine-marble
annotations:
sgx.intel.com/quote-provider: aesmd
spec:
container:
- name: gramine-marble
image: localhost/gramine-marble
resources:
limits:
sgx.intel.com/epc: 10Mi
```
81 changes: 81 additions & 0 deletions docs/versioned_docs/version-1.2/building-services/occlum.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Building a service: Occlum
Running an Occlum app with MarbleRun requires some changes to its manifest.

## Requirements
Set up an environment to create Occlum images. For an easy start, we recommend that you use either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or use [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our cloud deployment guide](../deployment/cloud.md).

To build your service, you can start with [Occlum's Introduction](https://github.com/occlum/occlum#introduction) to get your application up and running, and then come back here to adapt it for use with MarbleRun.

## Configuration
### Premain executable
Add our pre-built [premain-libos](https://github.com/edgelesssys/marblerun/releases/latest/download/premain-libos) executable to your Occlum image, e.g., by copying it to `image/bin/premain-libos`. By default, Occlum restricts executable files to the `/bin` directory. If you placed the `premain-libos` binary to a different path, you need to adjust this setting accordingly.

Finally, define the original entry point for your Occlum instance as the first `Argv` parameter for your Marble in MarbleRun's `manifest.json`. See [Defining a manifest](../workflows/define-manifest.md) for more information on how to define the `Argv` parameters. This lets MarbleRun launch your application after it succeeded in authenticating with the Coordinator and provides entrypoint pinning similar to the one offered in `Occlum.json`.

### Environment variables
The Marble needs to retrieve the MarbleRun specific configuration parameters via environment variables, as [described under Step 3 in "Adding a service."](../workflows/add-service.md)

To pass environment variables to the enclave, Occlum requires them to be specified in the `env` section in `Occlum.json`.

You can provide default (hardcoded) values under `default`, and you may also define them additionally as `untrusted` in case you want to allow changes to the Marble configuration after build time.

For example, consider this configuration:
```javascript
"env": {
"default": [
"OCCLUM=yes",
"EDG_MARBLE_COORDINATOR_ADDR=localhost:2001",
"EDG_MARBLE_TYPE=hello",
"EDG_MARBLE_UUID_FILE=uuid",
"EDG_MARBLE_DNS_NAMES=localhost"
],
"untrusted": [
"EDG_MARBLE_COORDINATOR_ADDR",
"EDG_MARBLE_TYPE",
"EDG_MARBLE_UUID_FILE",
"EDG_MARBLE_DNS_NAMES"
]
},
```

This will allow you to embed the expected default values during build time. It also lets the user/host system change them during run time when a non-default Coordinator configuration is used.

### Resource limits
The premain process is written in Go. The enclave needs to have enough resources for the Go runtime, plus additional memory to launch your application.

We recommend starting with the following values which should work fine for light-weight to medium memory demanding applications:
```javascript
"user_space_size": "2048MB",
"default_mmap_size": "900MB"
"max_num_of_threads": 64
```

In case you are running into issues with memory demands, check out the [Resource Configuration Guide](https://github.com/occlum/occlum/blob/master/docs/resource_config_guide.md) provided by the Occlum team to debug and resolve issues related to resource limits.

## Troubleshooting

### Error message: `fatal error: failed to reserve page summary memory`

If you receive this error during the launch of your Occlum image, make sure you allocated enough memory in `Occlum.json` [as described above](#resource-limits). The most important parameters are `user_space_size` and `default_mmap_size`.

### Error message: `Error returned from the p_sgx_get_quote_config API`

If Occlum crashes during the quote generation with the following error message:
```
[get_platform_quote_cert_data ../qe_logic.cpp:346] Error returned from the p_sgx_get_quote_config API. 0xe019
thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
left: `SGX_QL_SUCCESS`,
right: `SGX_QL_NETWORK_ERROR`: fail to launch QE', src/util/sgx/dcap/quote_generator.rs:22:13
```

You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration needs to be correctly set from **inside the container.**

If you use an Azure Confidential Computing machine, you can use our [provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). It patches the official Occlum image to use the Azure DCAP client, which handles the configuration automatically.

For other DCAP setups, please consult the documentation of your Intel Provisioning Certificate Caching Service (PCCS) service running locally or remotely.

### Other issues
If you are running into other issues, Occlum's error logging might help:
```bash
OCCLUM_LOG_LEVEL=error occlum run /bin/premain-libos
```
86 changes: 86 additions & 0 deletions docs/versioned_docs/version-1.2/deployment/cloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Cloud deployment

This guide walks you through setting up MarbleRun on different CSP offerings individually.

## Azure confidential computing VMs

[Azure confidential computing services](https://azure.microsoft.com/en-us/solutions/confidential-compute/) provide access to VMs with Intel SGX enabled in [DCsv2 VM instances](https://docs.microsoft.com/en-us/azure/virtual-machines/dcv2-series).
The description below uses a VM running Ubuntu 18.04.

### Prerequisites

* [Update and install EGo](https://github.com/edgelesssys/ego#install)
* [Update and install the Azure DCAP client](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#3-install-the-intel-and-open-enclave-packages-and-dependencies)

### Deploy MarbleRun

You can run MarbleRun standalone on your Azure DCsv2 VM, see our [standalone guide](../deployment/standalone.md).
Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube.

## Alibaba Cloud Elastic Compute Service

With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/doc-detail/207734.htm) users can try out and use Intel SGX on Alibaba Cloud.
Currently, security-enhanced instances are only available as part of an invitational preview.

The description below uses a VM running Ubuntu 18.04.

### Prerequisites

1. Install Intel DCAP Quote Provider Library

Add the Intel SGX APT repository:
```bash
wget -qO - https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add -
echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list
```

Download and install the QPL:
```bash
sudo apt update
sudo apt install libsgx-dcap-default-qpl
```

1. Set configuration for Alibaba Cloud SGX remote attestation service

Alibaba Cloud provides a PCCS for remote attestation, deployed on a per-region basis. For optimal stability it's recommended to access the service in your instance's region.
The configuration is set in `/etc/sgx_default_qcnl.conf`.
* If your instance is assigned a public IP address, change the configuration to the following, where `[Region-ID]` is the ID of your instance's region:
```
PCCS_URL=https://sgx-dcap-server.[Region-ID].aliyuncs.com/sgx/certification/v3/
USE_SECURE_CERT=TRUE
```

* If your instance is in a virtual private cloud and has only internal IP addresses, change the configuration to the following, where `[Region-ID]` is the ID of your instance's region:
```
PCCS_URL=https://sgx-dcap-server-vpc.[Region-ID].aliyuncs.com/sgx/certification/v3/
USE_SECURE_CERT=TRUE
```
:::note
Currently, the Alibaba Cloud SGX remote attestation service is only supported within [mainland China regions](https://www.alibabacloud.com/help/doc-detail/40654.htm#concept-2459516)
:::
1. [Update and install EGo](https://github.com/edgelesssys/ego#install)
### Deploy MarbleRun
You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../deployment/standalone.md).
Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube.
## Azure Kubernetes Services (AKS)
Azure Kubernetes Service (AKS) offers a popular deployment technique relying on
Azure's cloud resources. AKS hosts Kubernetes pods in Azure confidential compute
VMs and exposes the underlying confidential compute hardware.

### Prerequisites

Follow the instructions on the [AKS Confidential Computing Quick Start guide](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-enclave-nodes-aks-get-started)
to provision an AKS cluster with Intel SGX enabled worker nodes.

### Deploy MarbleRun

See our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in your AKS cluster.
Loading

0 comments on commit 313b442

Please sign in to comment.