Skip to content

Commit

Permalink
Custom Digest-md5 implementation (#84)
Browse files Browse the repository at this point in the history
  • Loading branch information
Kimahriman authored Mar 25, 2024
1 parent 18aa506 commit 3562554
Show file tree
Hide file tree
Showing 20 changed files with 493 additions and 5,798 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/python-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
container: quay.io/pypa/manylinux2014_${{ matrix.target }}:latest
docker-options: -e LD_LIBRARY_PATH=/opt/rh/llvm-toolset-7.0/root/usr/lib64 -e LLVM_CONFIG_PATH=/opt/rh/llvm-toolset-7.0/root/usr/bin/llvm-config
before-script-linux: |
yum install -y epel-release && yum install -y krb5-devel libgsasl-devel llvm-toolset-7.0-clang llvm-toolset-7.0-llvm-devel
yum install -y epel-release && yum install -y krb5-devel llvm-toolset-7.0-clang llvm-toolset-7.0-llvm-devel
- name: Upload wheels
uses: actions/upload-artifact@v3
with:
Expand All @@ -56,7 +56,7 @@ jobs:
- uses: actions/checkout@v4
- name: Install native libs
run:
brew install krb5 gsasl
brew install krb5
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/python-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
java-version: "17"

- name: Install native libs
run: sudo apt-get install -y libkrb5-dev libgsasl-dev krb5-user
run: sudo apt-get install -y libkrb5-dev krb5-user

- name: Download Hadoop
run: |
Expand Down Expand Up @@ -87,4 +87,4 @@ jobs:
manylinux: '2014'
docker-options: -e LD_LIBRARY_PATH=/opt/rh/llvm-toolset-7.0/root/usr/lib64 -e LLVM_CONFIG_PATH=/opt/rh/llvm-toolset-7.0/root/usr/bin/llvm-config
before-script-linux: |
yum install -y epel-release && yum install -y krb5-devel libgsasl-devel llvm-toolset-7.0-clang llvm-toolset-7.0-llvm-devel
yum install -y epel-release && yum install -y krb5-devel llvm-toolset-7.0-clang llvm-toolset-7.0-llvm-devel
17 changes: 7 additions & 10 deletions .github/workflows/rust-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,25 +41,22 @@ jobs:
- uses: Swatinem/rust-cache@v2

- name: Install native libs
run: sudo apt-get install -y libkrb5-dev libgsasl-dev
run: sudo apt-get install -y libkrb5-dev

- name: build and lint with clippy
run: cargo clippy --tests --features kerberos,token,integration-test
run: cargo clippy --all-targets --features kerberos,integration-test,benchmark -- -D warnings

- name: Check docs
run: cargo doc

- name: Check no features
run: cargo check
run: cargo check --tests

- name: Check kerberos
run: cargo check --features kerberos

- name: Check token
run: cargo check --features token
run: cargo check --tests --features kerberos

- name: Check all features
run: cargo check --features kerberos,token,integration-test
run: cargo check --all-targets --features kerberos,integration-test,benchmark

test:
strategy:
Expand Down Expand Up @@ -93,7 +90,7 @@ jobs:
java-version: "17"

- name: Install native libs
run: sudo apt-get install -y libkrb5-dev libgsasl-dev krb5-user
run: sudo apt-get install -y libkrb5-dev krb5-user

- name: Download Hadoop
run: |
Expand All @@ -102,4 +99,4 @@ jobs:
echo "$GITHUB_WORKSPACE/hadoop-3.3.6/bin" >> $GITHUB_PATH
- name: Run tests
run: cargo test --features kerberos,token,integration-test
run: cargo test --features kerberos,integration-test
16 changes: 9 additions & 7 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,21 +43,20 @@ All other settings are generally assumed to be the defaults currently. For insta

### Mac
```
brew install gsasl krb5
brew install krb5
# You might need these env vars on newer Macs
export BINDGEN_EXTRA_CLANG_ARGS="-I/opt/homebrew/include"
export LIBRARY_PATH=/opt/homebrew/lib
cargo build --features token,kerberos
cargo build --features kerberos
```

### Ubuntu
```
apt-get install clang libkrb5-dev libgsasl-dev
cargo build --features token,kerberos
apt-get install clang libkrb5-dev
cargo build --features kerberos
```

## Crate features
- `token` - enables token based DIGEST-MD5 authentication support. This uses the `gsasl` native library and only supports authentication, not integrity or confidentiality
- `kerberos` - enables kerberos GSSAPI authentication support. This uses the `libgssapi` crate and supports integrity as well as confidentiality

## Object store implementation
Expand All @@ -67,7 +66,7 @@ An object_store implementation for HDFS is provided in the [hdfs-native-object-s
The tests are mostly integration tests that utilize a small Java application in `rust/mindifs/` that runs a custom `MiniDFSCluster`. To run the tests, you need to have Java, Maven, Hadoop binaries, and Kerberos tools available and on your path. Any Java version between 8 and 17 should work.

```bash
cargo test -p hdfs-native --features token,kerberos,intergation-test
cargo test -p hdfs-native --features kerberos,intergation-test
```

### Python tests
Expand Down
Loading

0 comments on commit 3562554

Please sign in to comment.