Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: calculate SGX mr_enclave for notary server in gramine docker #701

Open
wants to merge 9 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 124 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ on:
- "[v]?[0-9]+.[0-9]+.[0-9]+*"
pull_request:

permissions:
id-token: write
contents: read
attestations: write

env:
CARGO_TERM_COLOR: always
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
Expand All @@ -17,7 +22,7 @@ env:
# - https://github.com/privacy-scaling-explorations/mpz/issues/178
# 32 seems to be big enough for the foreseeable future
RAYON_NUM_THREADS: 32

GIT_COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }}
jobs:
fmt:
name: Check formatting
Expand All @@ -38,6 +43,7 @@ jobs:

- name: Check formatting
run: cargo +nightly fmt --check --all

build-and-test:
name: Build and test
runs-on: ubuntu-latest
Expand All @@ -62,6 +68,7 @@ jobs:

- name: Test
run: cargo test

wasm:
name: Build and Test wasm
runs-on: ubuntu-latest
Expand Down Expand Up @@ -136,6 +143,7 @@ jobs:

- name: Run integration tests
run: cargo test --profile tests-integration --workspace --exclude tlsn-tls-client --exclude tlsn-tls-core -- --include-ignored

coverage:
runs-on: ubuntu-latest
env:
Expand All @@ -155,4 +163,118 @@ jobs:
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: lcov.info
fail_ci_if_error: true
fail_ci_if_error: true

build-sgx:
runs-on: ubuntu-latest
needs: build-and-test
container:
image: rust:latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Install Clang
run: |
apt update
apt install -y clang

- name: Use caching
uses: Swatinem/[email protected]

- name: Build Rust Binary
run: |
cargo build --bin notary-server --release --features tee_quote
cp --verbose target/release/notary-server $GITHUB_WORKSPACE

- name: Upload Binary for use in the Gramine Job
uses: actions/upload-artifact@v4
with:
name: notary-server
path: notary-server
if-no-files-found: error

gramine-sgx:
runs-on: ubuntu-latest
needs: build-sgx
environment: tee
container:
image: gramineproject/gramine:latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Restore SGX signing key from secrets
run: |
mkdir -p "${HOME}/.config/gramine/"
echo "${{ secrets.SGX_SIGNING_KEY }}" > "${HOME}/.config/gramine/enclave-key.pem"
# verify key
openssl rsa -in "${HOME}/.config/gramine/enclave-key.pem" -check -noout

- name: Download notary-server binary from build job
uses: actions/download-artifact@v4
with:
name: notary-server
path: crates/notary/server/tee

- name: Install jq
run: |
apt update
apt install -y jq

- name: Run notary-server in Gramine
run: |
cd crates/notary/server/tee

chmod +x notary-server

gramine-manifest \
-Dlog_level=debug \
-Darch_libdir=/lib/x86_64-linux-gnu \
-Dself_exe=notary-server \
notary-server.manifest.template \
notary-server.manifest

gramine-sgx-sign \
--manifest notary-server.manifest \
--output notary-server.manifest.sgx

gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig | tee >> notary-server-sigstruct.json

cat notary-server-sigstruct.json

mr_enclave=$(jq -r '.mr_enclave' notary-server-sigstruct.json)
mr_signer=$(jq -r '.mr_signer' notary-server-sigstruct.json)

echo "mrenclave=$mr_enclave" >>"$GITHUB_OUTPUT"
echo "#### sgx mrenclave" | tee >>$GITHUB_STEP_SUMMARY
echo "\`\`\`mr_enclave: ${mr_enclave}\`\`\`" | tee >>$GITHUB_STEP_SUMMARY
echo "\`\`\`mr_signer: ${mr_signer}\`\`\`" | tee >>$GITHUB_STEP_SUMMARY

- name: Upload notary-server and signatures
id: upload-notary-server-sgx
uses: actions/upload-artifact@v4
with:
name: notary-server-sgx.zip
path: |
crates/notary/server/tee/notary-server
crates/notary/server/tee/notary-server-sigstruct.json
crates/notary/server/tee/notary-server.sig
crates/notary/server/tee/notary-server.manifest
crates/notary/server/tee/notary-server.manifest.sgx
crates/notary/server/tee/config
crates/notary/server/tee/notary-server-sgx.md
if-no-files-found: error

- name: Attest Build Provenance
if: startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/dev'
uses: actions/attest-build-provenance@v2
with:
subject-name: notary-server-sgx.zip
subject-digest: sha256:${{ steps.upload-notary-server-sgx.outputs.artifact-digest }}

- uses: geekyeggo/delete-artifact@v5 # Delete notary-server from the build job, It is part of the zipfile with the signature
with:
name: notary-server
39 changes: 0 additions & 39 deletions .github/workflows/tee-ci.yml

This file was deleted.

68 changes: 68 additions & 0 deletions crates/notary/server/tee/notary-server-sgx.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# TLSNotary Notary Server for Intel SGX

This package contains the **SGX-enabled version of the TLSNotary Notary Server**, built with **Gramine** to run securely on Intel SGX hardware. Follow the instructions below to install dependencies, verify the integrity of the enclave, and launch the server.

## Install Intel SGX Drivers & Runtime Dependencies

Ensure your system has the required Intel SGX components installed.

```sh
wget https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key
cat intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null

# Add the repository to your sources:
echo 'deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu noble main' | sudo tee /etc/apt/sources.list.d/intel-sgx.list

sudo apt-get update
sudo apt-get install libsgx-epid libsgx-quote-ex libsgx-dcap-ql -y
```

For more details, refer to the official **[Intel SGX Installation Guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf).**

## Verify the Enclave Signature (`MR_ENCLAVE`, `MR_SIGNER`)

Before running the Notary Server, verify that the provided **enclave signature (`.sig`)** matches the expected **`MR_ENCLAVE` and `MR_SIGNER`** values.

Run the following command inside a **Gramine Docker container** to inspect the enclave's signature:

```sh
docker run --rm -v "$(pwd):/work" -w /work gramineproject/gramine:latest \
"gramine-sgx-sigstruct-view --verbose --output-format=json notary-server.sig"
```

The output should be the same as `notary-server-sigstruct.json`

## Run the Notary Server in SGX

Once verification is complete, you can start the Notary Server inside a **Gramine-protected SGX enclave**.

```sh
docker run --detach \
--restart=unless-stopped \
--device=/dev/sgx_enclave \
--device=/dev/sgx_provision \
--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
--publish=7047:7047 \
--volume="$(pwd):/work" \
--workdir=/work \
gramineproject/gramine:latest \
"gramine-sgx notary-server"
```

Notes:
- `--device /dev/sgx_enclave --device /dev/sgx_provision` → Exposes SGX devices.
- `--volume=/var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket` → Enables access to **Intel's AESM daemon** (required for remote attestation).
- `-p 7047:7047` → Exposes the notary server on port **7047**.

## Attestation & Verification

The Notary Server runs inside an **Intel SGX enclave**, which supports **remote attestation**. When connecting to it, clients should request an **SGX quote** to verify:

- **MR_ENCLAVE** (ensures the correct enclave binary is running).
- **MR_SIGNER** (ensures the enclave was signed by the expected key).
- **Quote Freshness** (prevents replay attacks).

To retrieve the SGX attestation quote, navigate to `<your notary server>:7047/info`:
```
curl localhost:7047/info | jq
```
2 changes: 1 addition & 1 deletion crates/notary/server/tee/notary-server.manifest.template
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,4 @@ sys.disallow_subprocesses = true
#### tlsn rev
sgx.isvprodid = 7
#### F
sgx.isvsvn = 46
sgx.isvsvn = 1