Skip to content

Commit

Permalink
feat: update PoC container
Browse files Browse the repository at this point in the history
  • Loading branch information
larry-neil-1206 committed Dec 8, 2024
1 parent de97da8 commit dd24813
Show file tree
Hide file tree
Showing 5 changed files with 19 additions and 159 deletions.
4 changes: 0 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
FROM python:3.12-slim

# Install any Python dependencies your application needs, e.g.:
RUN pip install --no-cache-dir requests

RUN mkdir /sealed && chmod 777 /sealed

WORKDIR /app

Expand Down
81 changes: 18 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This poc provides a basic structure for building proof tasks that:

```json
{
"dlp_id": 10,
"dlp_id": 12,
"valid": true,
"score": 0.7614457831325301,
"time_minimums": 1.0,
Expand All @@ -26,99 +26,55 @@ This poc provides a basic structure for building proof tasks that:
}
```

The project is designed to work with [Gramine](https://gramine.readthedocs.io/en/latest/), a lightweight library OS that enables running unmodified applications in secure enclaves, such as Intel SGX (Software Guard Extensions). This allows the code to run in a trusted execution environment, ensuring confidentiality and integrity of the computation.
The project is designed to work with Intel TDX (Trust Domain Extensions), providing hardware-level isolation and security guarantees for confidential computing workloads.

## Project Structure

- `my_proof/`: Contains the main proof logic
- `proof.py`: Implements the proof generation logic
- `__main__.py`: Entry point for the proof execution
- `models/`: Data models for the proof system
- `demo/`: Contains sample input and output for testing
- `.github/workflows/`: CI/CD pipeline for building and releasing
- `Dockerfile`: Defines the container image for the proof task
- `my-proof.manifest.template`: Gramine manifest template for running securely in an Intel SGX enclave
- `config.yaml`: Configuration file for Gramine Shielded Containers (GSC)
- `requirements.txt`: Python package dependencies

## Customizing the Proof Logic

The main proof logic is implemented in `my_proof/proof.py`. To customize it, update the `Proof.generate()` function to change how input files are processed.

The proof can be configured using environment variables. When running in an enclave, the environment variables must be defined in the `my-proof.manifest.template` file as well. The following environment variables are used for the VanaTensor DLP dataset proof:
The proof can be configured using environment variables:

- `USER_EMAIL`: The email address of the data contributor, to verify data ownership

## Local Development

To setup venv and install dependencies:

```
```bash
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
export PYTHONPATH=.
```

To run the proof locally, without Gramine, you can use Docker:
To run the proof locally for testing, you can use Docker:

```
```bash
docker build -t my-proof .
```

```bash
docker run --rm --volume $(pwd)/input:/input --volume $(pwd)/output:/output -e AWS_ACCESS_KEY_ID=<your-access-key-id> -e AWS_SECRET_ACCESS_KEY=<your-secret-access-key> my-proof
```
docker run --rm --volume $(pwd)/demo/sealed:/sealed --volume $(pwd)/demo/input:/input --volume $(pwd)/demo/output:/output -e AWS_ACCESS_KEY_ID=<your-access-key-id> -e AWS_SECRET_ACCESS_KEY=<your-secret-access-key> my-proof
```

## Building and Releasing

This includes a GitHub Actions workflow that automatically:

1. Builds a Docker image with your code
2. Creates a Gramine-shielded container (GSC) image
3. Publishes the GSC image as a GitHub release

**Important:** To use this workflow, you must generate a signing key and add it to your GitHub secrets. Follow these steps:

1. Generate a signing key (see instructions below)
2. Add the key as a GitHub secret named `SIGNING_KEY`
3. Push your changes to the `main` branch or create a pull request

### Generating the Gramine Signing Key (Required)

Before building and signing your graminized Docker image, you must generate a signing key. This key is crucial for creating secure SGX enclaves. Here's how to generate it:

1. If you have Gramine installed:

```
gramine-sgx-gen-private-key enclave-key.pem
```
## Running with Intel TDX

2. If you don't have Gramine, use OpenSSL:
Intel TDX (Trust Domain Extensions) provides hardware-based memory encryption and integrity protection for virtual machines. To run this container in a TDX-enabled environment, follow your infrastructure provider's specific instructions for deploying confidential containers.

```
openssl genrsa -3 -out enclave-key.pem 3072
```
Common volume mounts and environment variables:

After generating the key:

1. Keep this key secure, as it will be used to sign your enclaves.
2. Add the contents of `enclave-key.pem` as a GitHub secret named `SIGNING_KEY`.

This key is essential for the `gsc sign-image` step in the GSC workflow.

## Running with SGX

Intel SGX (Software Guard Extensions) is a set of security-related instruction codes built into modern Intel CPUs. It allows parts of a program to be executed in a secure enclave, isolated from the rest of the system.

To load a released image with docker, copy the URL from the release and run:

```
curl -L https://address/of/gsc-my-proof.tar.gz | docker load
```

To run the image:

```
docker run --rm --volume /gsc-my-proof/input:/input --volume /gsc-my-proof/output:/output --device /dev/sgx_enclave:/dev/sgx_enclave --volume /var/run/aesmd:/var/run/aesmd --volume /mnt/gsc-my-proof/sealed:/sealed gsc-my-proof
```bash
docker run --rm --volume /path/to/input:/input --volume /path/to/output:/output -e AWS_ACCESS_KEY_ID=<your-access-key-id> -e AWS_SECRET_ACCESS_KEY=<your-secret-access-key> my-proof
```

Remember to populate the `/input` directory with the files you want to process.
Expand All @@ -127,10 +83,9 @@ Remember to populate the `/input` directory with the files you want to process.

This proof leverages several security features:

1. **Secure Enclaves**: The proof runs inside an SGX enclave, isolating it from the rest of the system.
2. **Encrypted Storage**: The `/sealed` directory is automatically encrypted/decrypted by Gramine, providing secure storage for sensitive data.
3. **Input/Output Isolation**: Input and output directories are mounted separately, ensuring clear data flow boundaries.
4. **Minimal Attack Surface**: The Gramine manifest limits the files and resources accessible to the enclave, reducing potential vulnerabilities.
1. **Hardware-based Isolation**: The proof runs inside a TDX-protected environment, isolating it from the rest of the system
2. **Input/Output Isolation**: Input and output directories are mounted separately, ensuring clear data flow boundaries
3. **Minimal Container**: Uses a minimal Python base image to reduce attack surface

## Contributing

Expand Down
55 changes: 0 additions & 55 deletions config.yaml

This file was deleted.

36 changes: 0 additions & 36 deletions my-proof.manifest.template

This file was deleted.

2 changes: 1 addition & 1 deletion my_proof/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
def load_config() -> Dict[str, Any]:
"""Load proof configuration from environment variables."""
config = {
'dlp_id': 10,
'dlp_id': 12,
'input_dir': INPUT_DIR,
'aws_access_key_id': os.environ.get('AWS_ACCESS_KEY_ID', None),
'aws_secret_access_key': os.environ.get('AWS_SECRET_ACCESS_KEY', None),
Expand Down

0 comments on commit dd24813

Please sign in to comment.