-
Notifications
You must be signed in to change notification settings - Fork 34
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Daniel Weiße <[email protected]>
- Loading branch information
1 parent
47183db
commit 9dfb045
Showing
40 changed files
with
4,007 additions
and
0 deletions.
There are no files selected for viewing
37 changes: 37 additions & 0 deletions
37
docs/versioned_docs/version-1.2/_media/coordinator_deployment.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
53 changes: 53 additions & 0 deletions
53
docs/versioned_docs/version-1.2/_media/marble_deployment.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Building a service: EGo | ||
To get your Go service ready for MarbleRun, we provide two solutions. With [Transparent TLS (TTLS)](../features/transparent-TLS.md) you are able to use your existing applications without any code changes. Alternatively, you can adapt and recompile your application to manually handle the TLS credentials. Details are given in the following. | ||
|
||
## TTLS | ||
Simply follow the steps on [adding a service](../workflows/add-service.md). No code changes and no recompiling needed. | ||
|
||
## Manual TLS credentials handling | ||
|
||
If your service already uses TLS and gets the credentials from, e.g., a file, you just need to [adapt the manifest](../workflows/add-service.md#make-your-service-use-the-provided-tls-credentials). Otherwise, you need to make small code changes. | ||
|
||
We provide a convenience package called [`github.com/edgelesssys/ego/marble`](https://pkg.go.dev/github.com/edgelesssys/ego/marble#GetTLSConfig). With it, a service can automatically get and use its MarbleRun TLS credentials. The following gives an example. | ||
```Go | ||
serverCfg, err := marble.GetTLSConfig(false) | ||
if err != nil { | ||
log.Fatalf("Failed to retrieve server TLS config") | ||
} | ||
// use serverCfg, e.g., to create an HTTPS server | ||
``` | ||
|
||
Finally, you need to re-build your service for the enclave environment with `ego-go` and sign it with `ego sign`. Please follow the build instructions for Go provided in our [Go sample](https://github.com/edgelesssys/marblerun/blob/master/samples/helloworld). |
164 changes: 164 additions & 0 deletions
164
docs/versioned_docs/version-1.2/building-services/gramine.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,164 @@ | ||
# Building a service: Gramine | ||
|
||
Running a Gramine app with MarbleRun requires some changes to its manifest. These are explained in the following. See also the [hello world example](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-hello) for a simple introduction, or the [nginx](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-nginx) and [Redis](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-redis) examples for more detailed applications. | ||
|
||
## Requirements | ||
|
||
First, get Gramine up and running. Gramine is available [as a Debian package](https://github.com/gramineproject/gramine/releases). Alternatively you can follow either the [Building](https://gramine.readthedocs.io/en/latest/devel/building.html) or [Cloud Deployment](https://gramine.readthedocs.io/en/latest/installation.html) guide to build and install Gramine from source. | ||
|
||
Before running your application, make sure you got the prerequisites for ECDSA remote attestation installed on your system. You can collectively install them with the following command: | ||
|
||
```sh | ||
sudo apt install libsgx-quote-ex-dev | ||
``` | ||
|
||
## Configuration | ||
|
||
### Entrypoint and argv | ||
|
||
We provide the `premain-libos` executable with the [MarbleRun Releases](https://github.com/edgelesssys/marblerun/releases). It will contact the Coordinator, set up the environment, and run the actual application. | ||
|
||
Set the premain executable as [the entry point](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#libos-entrypoint) of the Gramine application and place the actual entry point [in argv0](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#command-line-arguments): | ||
|
||
```toml | ||
libos.entrypoint = "file:premain-libos" | ||
|
||
# argv0 needs to contain the name of your executable | ||
loader.argv = ["hello"] | ||
|
||
# add the premain to the list of trusted files | ||
sgx.trusted_files = [ | ||
# ... | ||
"file:premain-libos" | ||
] | ||
``` | ||
|
||
After the premain is done running, it will automatically spawn your application. | ||
|
||
### Host environment variables | ||
|
||
By default, environment variables from the host won't be passed to the application. | ||
Gramine allows to [pass through whitelisted environment variables from the host](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#environment-variables). | ||
The premain needs access to the following [environment variables for configuration](../workflows/add-service.md#step-3-start-your-service): | ||
|
||
```toml | ||
loader.env.EDG_MARBLE_TYPE = { passthrough = true } | ||
loader.env.EDG_MARBLE_COORDINATOR_ADDR = { passthrough = true } | ||
loader.env.EDG_MARBLE_UUID_FILE = { passthrough = true } | ||
loader.env.EDG_MARBLE_DNS_NAMES = { passthrough = true } | ||
``` | ||
|
||
### UUID file | ||
|
||
The Marble must be able to store its UUID: | ||
|
||
```toml | ||
sgx.allowed_files = [ | ||
# ... | ||
"file:uuid" | ||
] | ||
``` | ||
|
||
### Remote attestation | ||
|
||
The Marble will send an SGX quote to the Coordinator for remote attestation using [DCAP attestation](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#attestation-and-quotes): | ||
|
||
```toml | ||
sgx.remote_attestation = "dcap" | ||
``` | ||
|
||
### Enclave size and threads | ||
|
||
The premain process is written in Go. The enclave needs to have enough resources for the Go runtime: | ||
|
||
```toml | ||
sgx.enclave_size = "1024M" | ||
sgx.thread_num = 16 | ||
``` | ||
|
||
If your application has high memory demands, you may need to increase the size even further. | ||
|
||
### Secret files | ||
|
||
A Marble's secrets, e.g. a certificate and private key, can be provisioned as files. You can utilize Gramine's in-memory filesystem [`tmpfs`](https://gramine.readthedocs.io/en/latest/manifest-syntax.html#fs-mount-points), so the secrets will never show up on the host's file system : | ||
|
||
```toml | ||
fs.mounts = [ | ||
# ... | ||
{ type = "tmpfs", path = "/secrets" }, | ||
# ... | ||
] | ||
``` | ||
|
||
You can specify the files' content in the MarbleRun manifest: | ||
|
||
```javascript | ||
... | ||
"Parameters": { | ||
"Files": { | ||
"/secrets/server.crt": "{{ pem .Secrets.serverCert.Cert }}", | ||
"/secrets/server.key": "{{ pem .Secrets.serverCert.Private }}" | ||
} | ||
} | ||
... | ||
``` | ||
|
||
Gramine also allows to store files [encrypted on the host's file system](https://gramine.readthedocs.io/en/v1.3/manifest-syntax.html#encrypted-files). | ||
|
||
```toml | ||
fs.mounts = [ | ||
# ... | ||
{ type = "encrypted", path = "/secrets", uri = "file:/path/to/local/directory", key_name = "[KEY_NAME]" }, | ||
# ... | ||
] | ||
``` | ||
|
||
Gramine provides access to a pseudo filesystem for [setting the encryption key](https://gramine.readthedocs.io/en/v1.3/attestation.html#low-level-dev-attestation-interface). | ||
MarbleRun can set up your enclave with keys at runtime by specifying them in the MarbleRun manifest: | ||
|
||
```javascript | ||
... | ||
"Parameters": { | ||
"Files": { | ||
"/dev/attestation/[KEY_NAME]": "{{ raw .Secrets.encryptedFilesKey }}" | ||
} | ||
} | ||
... | ||
``` | ||
|
||
You can see how this is done in the [nginx example](https://github.com/edgelesssys/marblerun/tree/master/samples/gramine-nginx). | ||
|
||
## Troubleshooting | ||
|
||
### aesm_service returned error: 30 | ||
|
||
If you receive the following error message on launch: | ||
|
||
```sh | ||
aesm_service returned error: 30 | ||
load_enclave() failed with error -1 | ||
``` | ||
|
||
Make sure you installed the Intel AESM ECDSA plugins on your machine. You can do this by installing the `libsgx-quote-dev` package mentioned in the requirements above. | ||
|
||
If you are running your application in a container, you will need to mount the aesm socket. The socket is located at `/var/run/aesmd/`. | ||
|
||
If you are deploying your application on Kubernetes with the [Intel SGX device plugin](https://intel.github.io/intel-device-plugins-for-kubernetes/cmd/sgx_plugin/README.html) installed, the socket is automatically mounted by setting the `sgx.intel.com/quote-provider: aesmd` annotation for your deployment: | ||
|
||
```yaml | ||
apiVersion: v1 | ||
kind: Pod | ||
metadata: | ||
name: gramine-marble | ||
labels: | ||
marblerun/marbletype: gramine-marble | ||
annotations: | ||
sgx.intel.com/quote-provider: aesmd | ||
spec: | ||
container: | ||
- name: gramine-marble | ||
image: localhost/gramine-marble | ||
resources: | ||
limits: | ||
sgx.intel.com/epc: 10Mi | ||
``` |
81 changes: 81 additions & 0 deletions
81
docs/versioned_docs/version-1.2/building-services/occlum.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
# Building a service: Occlum | ||
Running an Occlum app with MarbleRun requires some changes to its manifest. | ||
|
||
## Requirements | ||
Set up an environment to create Occlum images. For an easy start, we recommend that you use either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or use [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our cloud deployment guide](../deployment/cloud.md). | ||
|
||
To build your service, you can start with [Occlum's Introduction](https://github.com/occlum/occlum#introduction) to get your application up and running, and then come back here to adapt it for use with MarbleRun. | ||
|
||
## Configuration | ||
### Premain executable | ||
Add our pre-built [premain-libos](https://github.com/edgelesssys/marblerun/releases/latest/download/premain-libos) executable to your Occlum image, e.g., by copying it to `image/bin/premain-libos`. By default, Occlum restricts executable files to the `/bin` directory. If you placed the `premain-libos` binary to a different path, you need to adjust this setting accordingly. | ||
|
||
Finally, define the original entry point for your Occlum instance as the first `Argv` parameter for your Marble in MarbleRun's `manifest.json`. See [Defining a manifest](../workflows/define-manifest.md) for more information on how to define the `Argv` parameters. This lets MarbleRun launch your application after it succeeded in authenticating with the Coordinator and provides entrypoint pinning similar to the one offered in `Occlum.json`. | ||
|
||
### Environment variables | ||
The Marble needs to retrieve the MarbleRun specific configuration parameters via environment variables, as [described under Step 3 in "Adding a service."](../workflows/add-service.md) | ||
|
||
To pass environment variables to the enclave, Occlum requires them to be specified in the `env` section in `Occlum.json`. | ||
|
||
You can provide default (hardcoded) values under `default`, and you may also define them additionally as `untrusted` in case you want to allow changes to the Marble configuration after build time. | ||
|
||
For example, consider this configuration: | ||
```javascript | ||
"env": { | ||
"default": [ | ||
"OCCLUM=yes", | ||
"EDG_MARBLE_COORDINATOR_ADDR=localhost:2001", | ||
"EDG_MARBLE_TYPE=hello", | ||
"EDG_MARBLE_UUID_FILE=uuid", | ||
"EDG_MARBLE_DNS_NAMES=localhost" | ||
], | ||
"untrusted": [ | ||
"EDG_MARBLE_COORDINATOR_ADDR", | ||
"EDG_MARBLE_TYPE", | ||
"EDG_MARBLE_UUID_FILE", | ||
"EDG_MARBLE_DNS_NAMES" | ||
] | ||
}, | ||
``` | ||
|
||
This will allow you to embed the expected default values during build time. It also lets the user/host system change them during run time when a non-default Coordinator configuration is used. | ||
|
||
### Resource limits | ||
The premain process is written in Go. The enclave needs to have enough resources for the Go runtime, plus additional memory to launch your application. | ||
|
||
We recommend starting with the following values which should work fine for light-weight to medium memory demanding applications: | ||
```javascript | ||
"user_space_size": "2048MB", | ||
"default_mmap_size": "900MB" | ||
"max_num_of_threads": 64 | ||
``` | ||
|
||
In case you are running into issues with memory demands, check out the [Resource Configuration Guide](https://github.com/occlum/occlum/blob/master/docs/resource_config_guide.md) provided by the Occlum team to debug and resolve issues related to resource limits. | ||
|
||
## Troubleshooting | ||
|
||
### Error message: `fatal error: failed to reserve page summary memory` | ||
|
||
If you receive this error during the launch of your Occlum image, make sure you allocated enough memory in `Occlum.json` [as described above](#resource-limits). The most important parameters are `user_space_size` and `default_mmap_size`. | ||
|
||
### Error message: `Error returned from the p_sgx_get_quote_config API` | ||
|
||
If Occlum crashes during the quote generation with the following error message: | ||
``` | ||
[get_platform_quote_cert_data ../qe_logic.cpp:346] Error returned from the p_sgx_get_quote_config API. 0xe019 | ||
thread '<unnamed>' panicked at 'assertion failed: `(left == right)` | ||
left: `SGX_QL_SUCCESS`, | ||
right: `SGX_QL_NETWORK_ERROR`: fail to launch QE', src/util/sgx/dcap/quote_generator.rs:22:13 | ||
``` | ||
|
||
You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration needs to be correctly set from **inside the container.** | ||
|
||
If you use an Azure Confidential Computing machine, you can use our [provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). It patches the official Occlum image to use the Azure DCAP client, which handles the configuration automatically. | ||
|
||
For other DCAP setups, please consult the documentation of your Intel Provisioning Certificate Caching Service (PCCS) service running locally or remotely. | ||
|
||
### Other issues | ||
If you are running into other issues, Occlum's error logging might help: | ||
```bash | ||
OCCLUM_LOG_LEVEL=error occlum run /bin/premain-libos | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
# Cloud deployment | ||
|
||
This guide walks you through setting up MarbleRun on different CSP offerings individually. | ||
|
||
## Azure confidential computing VMs | ||
|
||
[Azure confidential computing services](https://azure.microsoft.com/en-us/solutions/confidential-compute/) provide access to VMs with Intel SGX enabled in [DCsv2 VM instances](https://docs.microsoft.com/en-us/azure/virtual-machines/dcv2-series). | ||
The description below uses a VM running Ubuntu 18.04. | ||
|
||
### Prerequisites | ||
|
||
* [Update and install EGo](https://github.com/edgelesssys/ego#install) | ||
* [Update and install the Azure DCAP client](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#3-install-the-intel-and-open-enclave-packages-and-dependencies) | ||
|
||
### Deploy MarbleRun | ||
|
||
You can run MarbleRun standalone on your Azure DCsv2 VM, see our [standalone guide](../deployment/standalone.md). | ||
Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube. | ||
|
||
## Alibaba Cloud Elastic Compute Service | ||
|
||
With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/doc-detail/207734.htm) users can try out and use Intel SGX on Alibaba Cloud. | ||
Currently, security-enhanced instances are only available as part of an invitational preview. | ||
|
||
The description below uses a VM running Ubuntu 18.04. | ||
|
||
### Prerequisites | ||
|
||
1. Install Intel DCAP Quote Provider Library | ||
|
||
Add the Intel SGX APT repository: | ||
```bash | ||
wget -qO - https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add - | ||
echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list | ||
``` | ||
|
||
Download and install the QPL: | ||
```bash | ||
sudo apt update | ||
sudo apt install libsgx-dcap-default-qpl | ||
``` | ||
|
||
1. Set configuration for Alibaba Cloud SGX remote attestation service | ||
|
||
Alibaba Cloud provides a PCCS for remote attestation, deployed on a per-region basis. For optimal stability it's recommended to access the service in your instance's region. | ||
The configuration is set in `/etc/sgx_default_qcnl.conf`. | ||
* If your instance is assigned a public IP address, change the configuration to the following, where `[Region-ID]` is the ID of your instance's region: | ||
``` | ||
PCCS_URL=https://sgx-dcap-server.[Region-ID].aliyuncs.com/sgx/certification/v3/ | ||
USE_SECURE_CERT=TRUE | ||
``` | ||
|
||
* If your instance is in a virtual private cloud and has only internal IP addresses, change the configuration to the following, where `[Region-ID]` is the ID of your instance's region: | ||
``` | ||
PCCS_URL=https://sgx-dcap-server-vpc.[Region-ID].aliyuncs.com/sgx/certification/v3/ | ||
USE_SECURE_CERT=TRUE | ||
``` | ||
:::note | ||
Currently, the Alibaba Cloud SGX remote attestation service is only supported within [mainland China regions](https://www.alibabacloud.com/help/doc-detail/40654.htm#concept-2459516) | ||
::: | ||
1. [Update and install EGo](https://github.com/edgelesssys/ego#install) | ||
### Deploy MarbleRun | ||
You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../deployment/standalone.md). | ||
Alternatively, you can install a Kubernetes cluster, probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/), see our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in minikube. | ||
## Azure Kubernetes Services (AKS) | ||
Azure Kubernetes Service (AKS) offers a popular deployment technique relying on | ||
Azure's cloud resources. AKS hosts Kubernetes pods in Azure confidential compute | ||
VMs and exposes the underlying confidential compute hardware. | ||
|
||
### Prerequisites | ||
|
||
Follow the instructions on the [AKS Confidential Computing Quick Start guide](https://docs.microsoft.com/en-us/azure/confidential-computing/confidential-enclave-nodes-aks-get-started) | ||
to provision an AKS cluster with Intel SGX enabled worker nodes. | ||
|
||
### Deploy MarbleRun | ||
|
||
See our [Kubernetes guide](../deployment/kubernetes.md) on how to install MarbleRun in your AKS cluster. |
Oops, something went wrong.