Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK: Implement Cesium MSBuild Project SDK #561

Merged
merged 45 commits into from
Apr 21, 2024
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
c0ea89e
SDK: Initial progress
seclerp Jul 9, 2023
8081153
Add dry-run option to CesiumCompile
seclerp Jul 22, 2023
4a6f203
Add CommandArgumentsBuilder for argument escaping
seclerp Jul 23, 2023
bce3441
SDK: Start writing tests
seclerp Jul 23, 2023
8a93fee
SDK: Propagate solution metadata from MSBuild, add test base
seclerp Dec 17, 2023
ea83c61
SDK: WIP
seclerp Dec 18, 2023
fb17800
Use .NET 8 across the solution & toolset
seclerp Dec 20, 2023
15c3cdd
SDK: WIP
seclerp Dec 21, 2023
6ce1bb8
SDK: WIP
seclerp Dec 22, 2023
5638a1f
SDK: Work on compiler cross-platform packing
seclerp Dec 23, 2023
45d1a69
SDK: Introduce Nuke for unified compiler packs packaging
seclerp Dec 24, 2023
c71201d
SDK: Progress with packing and testing
seclerp Mar 16, 2024
e956f29
SDK: Introduce option to publish & pack only specific RID
seclerp Mar 17, 2024
cd3c49a
SDK: Fix compilation after packages consolidation
seclerp Mar 17, 2024
0fc5d1e
SDK: Add missing build schema update
seclerp Mar 17, 2024
d097f93
SDK: Add missing Self-contained option to compiler publishing
seclerp Mar 17, 2024
d802518
SDK: Make enum-related compiler properties validation more self-descr…
seclerp Mar 17, 2024
98a2cf4
SDK: Fix issues with proper properties propagation from MSBuild targe…
seclerp Mar 17, 2024
c17076c
Compiler: Fix AOT-related exception from CommandLineParser
seclerp Mar 17, 2024
04215a3
SDK: Fix compilation task
seclerp Mar 17, 2024
63953e8
CodeGen: Fix System.Runtime usage instead of System.Private.CoreLib
seclerp Apr 7, 2024
80e98f0
SDK: Add ability to publish non-AOT compiler pack
seclerp Apr 7, 2024
fe87deb
SDK: Auto-publish Cesium.Runtime
seclerp Apr 7, 2024
77202ea
SDK: Add missing configuration & fixes
seclerp Apr 7, 2024
e478946
SDK: Fix build for Exe and WinExe output types in netfx
seclerp Apr 7, 2024
f20d990
SDK: Add more tests
seclerp Apr 7, 2024
b6ec978
Tests: Fix failing tests after local to global artifacts folder migra…
seclerp Apr 14, 2024
32b6cf4
SDK: Include full path information for artifacts coming from build
seclerp Apr 14, 2024
c3310f8
CI: Include SDK tests into build workflow
seclerp Apr 15, 2024
e7aeacd
CI: Fix different issues on CI because of SDK implementation
seclerp Apr 15, 2024
b609346
SDK: Refactor compiler bundle producing, add docs
seclerp Apr 17, 2024
40cb7f6
SDK: Fix missing reference about MSBuild SDK docs in README
seclerp Apr 18, 2024
2f5c859
SDK: Fix line endings in NUKE build.sh script
seclerp Apr 18, 2024
af3c690
SDK: Move SDK helpers into TestFramework
seclerp Apr 19, 2024
df2583e
(#80) Docs: document how to run the SDK tests
ForNeVeR Apr 20, 2024
681920c
(#80) Add documentation on the solution metadata attribute
ForNeVeR Apr 20, 2024
5012948
(#80) Docs: more info on running tests
ForNeVeR Apr 20, 2024
65e6b99
(#80) SDK: self-written code for process argument passing
ForNeVeR Apr 20, 2024
21e9391
(#80) SDK: add a TODO note
ForNeVeR Apr 20, 2024
c020d67
SDK: Fix minor issues in documentation
seclerp Apr 21, 2024
06ccca0
SDK: Remove unnecessary lines
seclerp Apr 21, 2024
3ae514f
SDK: Fix naming of methods those are referencing the old "pack" term …
seclerp Apr 21, 2024
f9745f6
SDK: Use ArgumentsList instead of Arguments when running MSBuild process
seclerp Apr 21, 2024
6463a89
(#80) Docs: improve the test explanation
ForNeVeR Apr 21, 2024
38cd4b7
(#80, #196, #571) TODO processing
ForNeVeR Apr 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .config/dotnet-tools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"version": 1,
"isRoot": true,
"tools": {
"nuke.globaltool": {
"version": "7.0.6",
"commands": [
"nuke"
]
}
}
}
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
build.sh eol=lf
10 changes: 7 additions & 3 deletions .github/workflows/perform-common-steps/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,20 @@ runs:
using: "composite"
steps:
- name: ⚙ Setup .NET SDK ⚙
uses: actions/setup-dotnet@v1
uses: actions/setup-dotnet@v3
with:
dotnet-version: '8.0.x'
global-json-file: global.json

- name: ♻ NuGet Cache ♻
uses: actions/cache@v2
with:
path: ${{ env.NUGET_PACKAGES }}
key: ${{ runner.os }}.nuget.${{ hashFiles('**/*.csproj') }}

- name: 🛠️ Restore local .NET tools 🛠️
shell: bash
run: dotnet tool restore

- name: 🔄 Restore Nuget Packages 🔄
shell: bash
run: dotnet restore
run: dotnet nuke RestoreAll
12 changes: 10 additions & 2 deletions .github/workflows/run-build-and-unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,16 @@ jobs:

- name: 🛠 Build Solution 🛠
shell: bash
run: dotnet build
run: dotnet nuke CompileAll

- name: 🚚 Publish Compiler Bundle 🚚
shell: bash
run: dotnet nuke PublishCompilerBundle

- name: 📦 Pack Compiler Bundle 📦
shell: bash
run: dotnet nuke PackCompilerBundle

- name: ✅ Run Unit Tests ✅
shell: bash
run: dotnet test
run: dotnet nuke TestAll
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,13 @@

bin/
obj/
artifacts/
/.nuke/temp

*.dll
*.exe
*.runtimeconfig.json
*.received.txt
*.user


148 changes: 148 additions & 0 deletions .nuke/build.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"$ref": "#/definitions/build",
"title": "Build Schema",
"definitions": {
"build": {
"type": "object",
"properties": {
"Configuration": {
"type": "string",
"description": "Configuration to build - Default is 'Debug' or 'Release'",
"enum": [
"Debug",
"Release"
]
},
"Continue": {
"type": "boolean",
"description": "Indicates to continue a previously failed build attempt"
},
"Help": {
"type": "boolean",
"description": "Shows the help text for this build assembly"
},
"Host": {
"type": "string",
"description": "Host for execution. Default is 'automatic'",
"enum": [
"AppVeyor",
"AzurePipelines",
"Bamboo",
"Bitbucket",
"Bitrise",
"GitHubActions",
"GitLab",
"Jenkins",
"Rider",
"SpaceAutomation",
"TeamCity",
"Terminal",
"TravisCI",
"VisualStudio",
"VSCode"
]
},
"NoLogo": {
"type": "boolean",
"description": "Disables displaying the NUKE logo"
},
"Partition": {
"type": "string",
"description": "Partition to use on CI"
},
"Plan": {
"type": "boolean",
"description": "Shows the execution plan (HTML)"
},
"Profile": {
"type": "array",
"description": "Defines the profiles to load",
"items": {
"type": "string"
}
},
"PublishAot": {
"type": "boolean",
"description": "If set to true, publishes compiler packs in AOT mode"
},
"Root": {
"type": "string",
"description": "Root directory during build execution"
},
"RuntimeId": {
"type": "string",
"description": "If set, only executes targets for a specified runtime identifier. Provided RID must be included in <RuntimeIdentifiers> property of Cesium.Compiler project"
},
"Skip": {
"type": "array",
"description": "List of targets to be skipped. Empty list skips all dependencies",
"items": {
"type": "string",
"enum": [
"Clean",
"CompileAll",
"ForceClear",
"PackAllCompilerBundles",
"PackCompilerBundle",
"PackSdk",
"PublishAllCompilerBundles",
"PublishCompilerBundle",
"RestoreAll",
"TestAll",
"TestCodeGen",
"TestCompiler",
"TestIntegration",
"TestParser",
"TestRuntime",
"TestSdk"
]
}
},
"SkipCaches": {
"type": "boolean",
"description": "If set to true, ignores all cached build results. Default: false"
},
"Solution": {
"type": "string",
"description": "Path to a solution file that is automatically loaded"
},
"Target": {
"type": "array",
"description": "List of targets to be invoked. Default is '{default_target}'",
"items": {
"type": "string",
"enum": [
"Clean",
"CompileAll",
"ForceClear",
"PackAllCompilerBundles",
"PackCompilerBundle",
"PackSdk",
"PublishAllCompilerBundles",
"PublishCompilerBundle",
"RestoreAll",
"TestAll",
"TestCodeGen",
"TestCompiler",
"TestIntegration",
"TestParser",
"TestRuntime",
"TestSdk"
]
}
},
"Verbosity": {
"type": "string",
"description": "Logging verbosity during build execution. Default is 'Normal'",
"enum": [
"Minimal",
"Normal",
"Quiet",
"Verbose"
]
}
}
}
}
}
4 changes: 4 additions & 0 deletions .nuke/parameters.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"$schema": "./build.schema.json",
"Solution": "Cesium.sln"
}
3 changes: 2 additions & 1 deletion Cesium.Ast/Cesium.Ast.csproj
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
</PropertyGroup>

<ItemGroup>
Expand Down
2 changes: 1 addition & 1 deletion Cesium.CodeGen/Contexts/AssemblyContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ private TypeReference GetStubType(int size)
"",
stubStructTypeName,
TypeAttributes.Sealed | TypeAttributes.ExplicitLayout | TypeAttributes.NestedPrivate,
Module.ImportReference(MscorlibAssembly.GetType("System.ValueType")))
Module.ImportReference(new TypeReference("System", "ValueType", MscorlibAssembly.MainModule, MscorlibAssembly.MainModule.TypeSystem.CoreLibrary)))
{
PackingSize = 1,
ClassSize = size
Expand Down
10 changes: 5 additions & 5 deletions Cesium.CodeGen/Ir/Types/InPlaceArrayType.cs
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ public FieldDefinition CreateFieldOfType(TranslationUnitContext context, TypeDef

CustomAttribute GenerateCustomFieldAttribute()
{
var typeType = context.Module.ImportReference(context.AssemblyContext.MscorlibAssembly.GetType("System.Type"));
var fixedBufferAttributeType = context.AssemblyContext.MscorlibAssembly.GetType("System.Runtime.CompilerServices.FixedBufferAttribute") ?? throw new AssertException(
var typeType = context.Module.ImportReference(new TypeReference("System", "Type", context.AssemblyContext.MscorlibAssembly.MainModule, context.AssemblyContext.MscorlibAssembly.MainModule.TypeSystem.CoreLibrary));
var fixedBufferAttributeType = new TypeReference("System.Runtime.CompilerServices", "FixedBufferAttribute", context.AssemblyContext.MscorlibAssembly.MainModule, context.AssemblyContext.MscorlibAssembly.MainModule.TypeSystem.CoreLibrary) ?? throw new AssertException(
"Cannot find a type System.Runtime.CompilerServices.FixedBufferAttribute.");
var fixedBufferCtor = new MethodReference(".ctor", context.TypeSystem.Void, fixedBufferAttributeType);
fixedBufferCtor.Parameters.Add(new ParameterDefinition(typeType));
Expand Down Expand Up @@ -104,12 +104,12 @@ private static TypeDefinition CreateFixedBufferType(
// }

ModuleDefinition module = context.Module;
var compilerGeneratedAttributeType = context.AssemblyContext.MscorlibAssembly.GetType("System.Runtime.CompilerServices.CompilerGeneratedAttribute") ?? throw new AssertException(
var compilerGeneratedAttributeType = new TypeReference("System.Runtime.CompilerServices", "CompilerGeneratedAttribute", context.AssemblyContext.MscorlibAssembly.MainModule, context.AssemblyContext.MscorlibAssembly.MainModule.TypeSystem.CoreLibrary) ?? throw new AssertException(
"Cannot find a type System.Runtime.CompilerServices.CompilerGeneratedAttribute.");
var compilerGeneratedCtor = new MethodReference(".ctor", context.TypeSystem.Void, compilerGeneratedAttributeType);
var compilerGeneratedAttribute = new CustomAttribute(module.ImportReference(compilerGeneratedCtor));

var unsafeValueTypeAttributeType = context.AssemblyContext.MscorlibAssembly.GetType("System.Runtime.CompilerServices.UnsafeValueTypeAttribute") ?? throw new AssertException(
var unsafeValueTypeAttributeType = new TypeReference("System.Runtime.CompilerServices", "UnsafeValueTypeAttribute", context.AssemblyContext.MscorlibAssembly.MainModule, context.AssemblyContext.MscorlibAssembly.MainModule.TypeSystem.CoreLibrary) ?? throw new AssertException(
"Cannot find a type System.Runtime.CompilerServices.UnsafeValueTypeAttribute.");
var unsafeValueTypeCtor = new MethodReference(".ctor", context.TypeSystem.Void, unsafeValueTypeAttributeType);
var unsafeValueTypeAttribute = new CustomAttribute(module.ImportReference(unsafeValueTypeCtor));
Expand All @@ -118,7 +118,7 @@ private static TypeDefinition CreateFixedBufferType(
"",
$"<SyntheticBuffer>{fieldName}",
TypeAttributes.Public | TypeAttributes.Sealed | TypeAttributes.SequentialLayout | TypeAttributes.NestedPublic,
module.ImportReference(context.AssemblyContext.MscorlibAssembly.GetType("System.ValueType")))
module.ImportReference(new TypeReference("System", "ValueType", context.AssemblyContext.MscorlibAssembly.MainModule, context.AssemblyContext.MscorlibAssembly.MainModule.TypeSystem.CoreLibrary)))
{
PackingSize = 0,
ClassSize = sizeInBytes,
Expand Down
4 changes: 2 additions & 2 deletions Cesium.CodeGen/Ir/Types/StructType.cs
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ public TypeDefinition StartEmit(string name, TranslationUnitContext context)
"",
Identifier is null ? "<typedef>" + name : Identifier,
TypeAttributes.Sealed,
context.Module.ImportReference(context.AssemblyContext.MscorlibAssembly.GetType("System.ValueType")));
context.Module.ImportReference(new TypeReference("System", "ValueType", context.AssemblyContext.MscorlibAssembly.MainModule, context.AssemblyContext.MscorlibAssembly.MainModule.TypeSystem.CoreLibrary)));
switch (context.AssemblyContext.ArchitectureSet)
{
case TargetArchitectureSet.Dynamic:
Expand Down Expand Up @@ -106,7 +106,7 @@ private void EmitAsAnonStructure(TranslationUnitContext context)
string.Empty,
CreateAnonIdentifier(Members, IsUnion),
TypeAttributes.Public | TypeAttributes.Sealed,
context.Module.ImportReference(context.AssemblyContext.MscorlibAssembly.GetType("System.ValueType")));
context.Module.ImportReference(new TypeReference("System", "ValueType", context.AssemblyContext.MscorlibAssembly.MainModule, context.AssemblyContext.MscorlibAssembly.MainModule.TypeSystem.CoreLibrary)));

FinishEmit(type, type.Name, context); // emit fields

Expand Down
4 changes: 3 additions & 1 deletion Cesium.Compiler/Cesium.Compiler.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<PublishAot Condition="$(NETCoreSdkRuntimeIdentifier) != 'osx-x64'">true</PublishAot>
<!-- <PublishAot Condition="$(NETCoreSdkRuntimeIdentifier) != 'osx-x64' AND $(NETCoreSdkRuntimeIdentifier) != 'osx-arm64'">true</PublishAot>-->
seclerp marked this conversation as resolved.
Show resolved Hide resolved
<PublishSingleFile>true</PublishSingleFile>
<PublishTrimmed>true</PublishTrimmed>
<RollForward>Major</RollForward>
<RuntimeIdentifiers>win-x64;win-x86;win-arm64;linux-x64;linux-arm64;osx-x64;osx-arm64</RuntimeIdentifiers>
</PropertyGroup>

<ItemGroup>
Expand Down
71 changes: 40 additions & 31 deletions Cesium.Compiler/Main.cs
Original file line number Diff line number Diff line change
@@ -1,43 +1,52 @@
using System.Diagnostics.CodeAnalysis;
using Cesium.CodeGen;
using Cesium.Compiler;
using Cesium.Core;
using Mono.Cecil;

await CommandLineParser.ParseCommandLineArgs(args, new CompilerReporter(), async args =>
namespace Cesium.Compiler;

public static class Program
{
[DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(Arguments))]
public static async Task<int> Main(string[] args)
{
var targetArchitectureSet = args.TargetArchitectureSet;
var targetRuntime = args.Framework switch
return await CommandLineParser.ParseCommandLineArgs(args, new CompilerReporter(), async args =>
{
TargetFrameworkKind.NetFramework => TargetRuntimeDescriptor.Net48,
TargetFrameworkKind.NetStandard => TargetRuntimeDescriptor.NetStandard20,
_ => TargetRuntimeDescriptor.Net60
};
var targetArchitectureSet = args.TargetArchitectureSet;
var targetRuntime = args.Framework switch
{
TargetFrameworkKind.NetFramework => TargetRuntimeDescriptor.Net48,
TargetFrameworkKind.NetStandard => TargetRuntimeDescriptor.NetStandard20,
_ => TargetRuntimeDescriptor.Net60
};

var cesiumRuntime = args.CesiumCRuntime ?? Path.Combine(AppContext.BaseDirectory, "Cesium.Runtime.dll");
var defaultImportsAssembly = args.DefaultImportAssemblies ?? Array.Empty<string>();
var cesiumRuntime = args.CesiumCRuntime ?? Path.Combine(AppContext.BaseDirectory, "Cesium.Runtime.dll");
var defaultImportsAssembly = args.DefaultImportAssemblies ?? Array.Empty<string>();
#pragma warning disable IL3000 // Automatic discovery of corelib is fallback option, if tooling do not pass that parameter
var corelibAssembly = args.CoreLib ?? typeof(Math).Assembly.Location; // System.Runtime.dll
var corelibAssembly = args.CoreLib ?? typeof(Math).Assembly.Location; // System.Runtime.dll
#pragma warning restore IL3000
var moduleKind = args.ProducePreprocessedFile ? ModuleKind.Console : args.ModuleKind ?? Path.GetExtension(args.OutputFilePath).ToLowerInvariant() switch
{
".exe" => ModuleKind.Console,
".dll" => ModuleKind.Dll,
var o => throw new CompilationException($"Unknown file extension: {o}. \"modulekind\" is not specified.")
};
var compilationOptions = new CompilationOptions(
targetRuntime,
targetArchitectureSet,
moduleKind,
corelibAssembly,
cesiumRuntime,
defaultImportsAssembly,
args.Namespace,
args.GlobalClass,
args.DefineConstant.ToList(),
args.IncludeDirectories.ToList(),
args.ProducePreprocessedFile);
return await Compilation.Compile(args.InputFilePaths, args.OutputFilePath, compilationOptions);
});
var moduleKind = args.ProducePreprocessedFile ? ModuleKind.Console : args.ModuleKind ?? Path.GetExtension(args.OutputFilePath).ToLowerInvariant() switch
{
".exe" => ModuleKind.Console,
".dll" => ModuleKind.Dll,
var o => throw new CompilationException($"Unknown file extension: {o}. \"modulekind\" is not specified.")
};
var compilationOptions = new CompilationOptions(
targetRuntime,
targetArchitectureSet,
moduleKind,
corelibAssembly,
cesiumRuntime,
defaultImportsAssembly,
args.Namespace,
args.GlobalClass,
args.DefineConstant.ToList(),
args.IncludeDirectories.ToList(),
args.ProducePreprocessedFile);
return await Compilation.Compile(args.InputFilePaths, args.OutputFilePath, compilationOptions);
});
}
}

class CompilerReporter : ICompilerReporter
{
Expand Down
Loading
Loading