Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Leveldb RocksDb Runtime Binaries #3665

Open
wants to merge 30 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
9e52cbb
Fixed leveldb on release
cschuchardt88 Jan 9, 2025
91b1424
fixed release.yml
cschuchardt88 Jan 9, 2025
5b1fa73
update test and everything
cschuchardt88 Jan 9, 2025
663b901
fixed bug
cschuchardt88 Jan 9, 2025
18a9528
fix bug
cschuchardt88 Jan 9, 2025
2eb726a
Merge from master
cschuchardt88 Jan 9, 2025
2c5f070
fixed bug
cschuchardt88 Jan 9, 2025
aeb5b59
fixed bug
cschuchardt88 Jan 9, 2025
02ab1a6
Merge branch 'master' into fix/leveldb-deps
shargon Jan 9, 2025
26dd38f
Added support for linux-arm64
cschuchardt88 Jan 9, 2025
2da130d
Merge branch 'fix/leveldb-deps' of https://github.com/cschuchardt88/n…
cschuchardt88 Jan 9, 2025
18af18b
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Jan 9, 2025
bfedd0f
Merge branch 'master' into fix/leveldb-deps
Jim8y Jan 9, 2025
e3fe725
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Jan 11, 2025
b95c675
Merger and fixes
cschuchardt88 Jan 17, 2025
5fad90f
fixed `main.yml`
cschuchardt88 Jan 17, 2025
1f2a22e
Merge branch 'fix/leveldb-deps' of https://github.com/cschuchardt88/n…
cschuchardt88 Jan 17, 2025
0d950a7
remove `arm64` and `x64`
cschuchardt88 Jan 17, 2025
905db21
Merge branch 'master' into fix/leveldb-deps
Jim8y Jan 20, 2025
447cd5f
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Jan 22, 2025
b8ae4e3
Merge branch 'master' into fix/leveldb-deps
Jim8y Jan 23, 2025
b09b346
Update release.yml
cschuchardt88 Jan 24, 2025
5476b2f
Fix move command
cschuchardt88 Jan 24, 2025
e599950
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Jan 24, 2025
4e87a5c
Merge branch 'master' into fix/leveldb-deps
Jim8y Jan 27, 2025
ce1efd2
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Jan 31, 2025
9eee9fe
Merge branch 'master' into fix/leveldb-deps
Jim8y Jan 31, 2025
f26882d
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Jan 31, 2025
39c5b26
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Feb 3, 2025
ccc999d
Merge branch 'master' into fix/leveldb-deps
cschuchardt88 Feb 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,13 @@ jobs:
-p:DebugType=embedded \
-p:ServerGarbageCollection=true

# Copy 'libleveldb' library file
- name: Native Content (neo-cli)
working-directory: ${{ env.OUTPUT_PATH }}/${{ matrix.runtime }}
run: |
mv -v ./runtimes/${{ matrix.runtime }}/native/* .
rm -v -R runtimes

- name: Build (LevelDbStore)
run: |
dotnet build ./src/Plugins/LevelDBStore \
Expand All @@ -57,6 +64,7 @@ jobs:

- name: Remove (junk)
run: |
rm -v -R ${{ env.DIST_DIR }}/Plugins/LevelDBStore/runtimes
rm -v ${{ env.DIST_DIR }}/Plugins/LevelDBStore/Neo*
rm -v ${{ env.DIST_DIR }}/Plugins/LevelDBStore/*.pdb
rm -v ${{ env.DIST_DIR }}/Plugins/LevelDBStore/*.xml
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,13 @@ jobs:
if: matrix.os == 'macos-latest'
run: |
brew install leveldb
brew install gperftools
dotnet build
cp -vp /opt/homebrew/Cellar/leveldb/1.23_2/lib/libleveldb.dylib ./tests/Neo.Plugins.Storage.Tests/bin/Debug/net9.0/
dotnet test --no-build

- name: Test (windows)
if: matrix.os == 'windows-latest'
run: |
dotnet sln neo.sln remove ./tests/Neo.Plugins.Storage.Tests/Neo.Plugins.Storage.Tests.csproj
dotnet build
dotnet test --no-build

Expand Down
95 changes: 8 additions & 87 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,69 +13,7 @@ env:
OUTPUT_PATH: /tmp/out

jobs:
build-leveldb:
name: Build leveldb win-${{ matrix.arch }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [windows-latest]
arch: [x64, arm64]

steps:
# Step to lookup cache for the LevelDB build distribution
- name: Lookup Cache Distribution
id: cache-leveldb
uses: actions/cache@v4
with:
path: ./leveldb/build/Release/*
key: leveldb-${{ matrix.os }}-${{ matrix.arch }}
enableCrossOsArchive: true
lookup-only: true

# Conditionally checkout LevelDB repository if cache is not found
- if: ${{ steps.cache-leveldb.outputs.cache-hit != 'true' }}
name: Checkout Repository Code (leveldb)
uses: actions/checkout@v4
with:
repository: google/leveldb
path: leveldb
submodules: true
fetch-depth: 0

# Conditionally setup MSBuild if cache is not found
- if: ${{ matrix.os == 'windows-latest' && steps.cache-leveldb.outputs.cache-hit != 'true' }}
name: Setup MSBuild
uses: microsoft/setup-msbuild@v2

# Conditionally setup LevelDB build directory if cache is not found
- if: ${{ steps.cache-leveldb.outputs.cache-hit != 'true' }}
name: Setup LevelDb
working-directory: ./leveldb
run: mkdir -p ./build/Release

# Conditionally create build files for LevelDB if cache is not found
- if: ${{ steps.cache-leveldb.outputs.cache-hit != 'true' }}
name: Create Build Files (win-${{ matrix.arch }})
working-directory: ./leveldb/build
run: cmake -DBUILD_SHARED_LIBS=ON -A ${{ matrix.arch }} ..

# Conditionally build LevelDB using MSBuild if cache is not found
- if: ${{ matrix.os == 'windows-latest' && steps.cache-leveldb.outputs.cache-hit != 'true' }}
name: Build (MSBuild)
working-directory: ./leveldb/build
run: msbuild ./leveldb.sln /p:Configuration=Release

# Conditionally cache the LevelDB distribution if it was built
- if: ${{ steps.cache-leveldb.outputs.cache-hit != 'true' }}
name: Cache Distribution
uses: actions/cache/save@v4
with:
path: ./leveldb/build/Release/*
key: leveldb-${{ matrix.os }}-${{ matrix.arch }}
enableCrossOsArchive: true

build-neo-cli:
needs: [build-leveldb]
name: ${{ matrix.runtime }}
runs-on: ubuntu-latest
strategy:
Expand Down Expand Up @@ -123,6 +61,13 @@ jobs:
-p:DebugType=embedded \
-p:ServerGarbageCollection=true

# Copy 'libleveldb' library file
- name: Native Content (neo-cli)
working-directory: ${{ env.OUTPUT_PATH }}/${{ matrix.runtime }}
run: |
mv -v ./runtimes/${{ matrix.runtime }}/native/* .
rm -v -R runtimes

# Build the LevelDBStore plugin
- name: .NET Build (LevelDBStore)
run: |
Expand All @@ -138,6 +83,7 @@ jobs:
- name: Remove files (junk)
working-directory: ${{ env.OUTPUT_PATH }}/${{ matrix.runtime }}/Plugins/LevelDBStore
run: |
rm -v -R runtimes
rm -v Neo*
rm -v *.pdb
rm -v *.xml
Expand All @@ -147,31 +93,6 @@ jobs:
working-directory: ${{ env.OUTPUT_PATH }}/${{ matrix.runtime }}
run: rm -v *.xml

# Get cached LevelDB distribution for Windows x64 if applicable
- if: ${{ startsWith(matrix.runtime, 'win-x64') }}
name: Get Distribution Caches (win-x64)
uses: actions/cache@v4
with:
path: ./leveldb/build/Release/*
key: leveldb-windows-latest-x64
enableCrossOsArchive: true
fail-on-cache-miss: true

# Get cached LevelDB distribution for Windows arm64 if applicable
- if: ${{ startsWith(matrix.runtime, 'win-arm64') }}
name: Get Distribution Caches (win-arm64)
uses: actions/cache@v4
with:
path: ./leveldb/build/Release/*
key: leveldb-windows-latest-arm64
enableCrossOsArchive: true
fail-on-cache-miss: true

# Copy LevelDB files to the output directory for Windows
- if: ${{ startsWith(matrix.runtime, 'win') }}
name: Copy Files (leveldb) (win)
run: cp -v ./leveldb/build/Release/leveldb.dll ${{ env.OUTPUT_PATH }}/${{ matrix.runtime }}/libleveldb.dll

# Create the distribution directory
- name: Create Distribution Directory
run: mkdir -p ${{ env.DIST_PATH }}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
<Project Sdk="Microsoft.NET.Sdk">
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<RootNamespace>Neo.Extensions</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<RootNamespace>Neo.Extensions</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>

<ItemGroup>
<ProjectReference Include="..\..\src\Neo.Extensions\Neo.Extensions.csproj" />
Expand Down
7 changes: 7 additions & 0 deletions src/Neo.CLI/Neo.CLI.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,18 @@
<ProjectReference Include="..\Neo.ConsoleService\Neo.ConsoleService.csproj" />
<ProjectReference Include="..\Neo.Extensions\Neo.Extensions.csproj" />
<ProjectReference Include="..\Neo\Neo.csproj" />
<ProjectReference Include="..\Plugins\LevelDBStore\LevelDBStore.csproj" />
<ProjectReference Include="..\Plugins\RocksDBStore\RocksDBStore.csproj" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="System.CommandLine" Version="2.0.0-beta4.22272.1" />
<PackageReference Include="System.CommandLine.NamingConventionBinder" Version="2.0.0-beta4.22272.1" />
</ItemGroup>

<Target Name="MoveLevelDbForBuild" BeforeTargets="PostBuildEvent">
<Move SourceFiles="$(OutputPath)/LevelDBStore.dll" DestinationFolder="$(OutDir)/Plugins/LevelDBStore" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just this part is good for now, @cschuchardt88 .
If so we close the other PR I opened.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to open this separately, because the rest of the PR, for now, I am not in agreement.

Copy link
Member Author

@cschuchardt88 cschuchardt88 Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you specify what you do not like with this PR? So maybe we can work on fixing it or answering any concerns or reasons you have.

<Move SourceFiles="$(OutputPath)/RocksDBStore.dll" DestinationFolder="$(OutDir)/Plugins/RocksDBStore" />
</Target>

</Project>
21 changes: 21 additions & 0 deletions src/Plugins/LevelDBStore/LevelDBStore.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,25 @@
<OutputPath>../../../bin/$(PackageId)</OutputPath>
</PropertyGroup>

<Target Name="DownloadNativeLibs" BeforeTargets="PreBuildEvent">
<DownloadFile Condition="!Exists('runtimes/%(NativeLib.RuntimeId)/native/%(NativeLib.Lib)')" SourceUrl="https://github.com/neo-ngd/leveldb/releases/download/v1.23/libleveldb-%(NativeLib.RuntimeId).zip" DestinationFolder="$(BaseIntermediateOutputPath)" />
<Unzip Condition="Exists('$(BaseIntermediateOutputPath)libleveldb-%(NativeLib.RuntimeId).zip') And !Exists('runtimes/%(NativeLib.RuntimeId)/native/%(NativeLib.Lib)')" SourceFiles="$(BaseIntermediateOutputPath)libleveldb-%(NativeLib.RuntimeId).zip" DestinationFolder="$(BaseIntermediateOutputPath)" />
<Move SourceFiles="$(BaseIntermediateOutputPath)libleveldb-%(NativeLib.RuntimeId)/%(NativeLib.Lib)" DestinationFolder="$(BaseIntermediateOutputPath)libleveldb-%(NativeLib.RuntimeId)/runtimes/%(NativeLib.RuntimeId)/native/" />
</Target>

<ItemGroup>
<NativeLib Include="%(RuntimeId)-%(Lib)" RuntimeId="win-x64" Lib="libleveldb.dll" />
shargon marked this conversation as resolved.
Show resolved Hide resolved
<NativeLib Include="%(RuntimeId)-%(Lib)" RuntimeId="win-arm64" Lib="libleveldb.dll" />
<NativeLib Include="%(RuntimeId)-%(Lib)" RuntimeId="linux-x64" Lib="libleveldb.so" />
<NativeLib Include="%(RuntimeId)-%(Lib)" RuntimeId="linux-arm64" Lib="libleveldb.so" />
<NativeLib Include="%(RuntimeId)-%(Lib)" RuntimeId="osx-x64" Lib="libleveldb.dylib" />
<NativeLib Include="%(RuntimeId)-%(Lib)" RuntimeId="osx-arm64" Lib="libleveldb.dylib" />

<Content Include="@(NativeLib -> '$(BaseIntermediateOutputPath)libleveldb-%(RuntimeId)/runtimes/%(RuntimeId)/native/%(Lib)')" PackagePath="runtimes/%(RuntimeId)/native/%(Lib)" Link="runtimes/%(RuntimeId)/native/%(Lib)">
<Pack>true</Pack>
<Visible>false</Visible>
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</Content>
</ItemGroup>

</Project>