feat: Open source the AST model (#205)

Adds a new package `celest_ast` which contains structured and serializable representations of Celest projects. This is the same package used by by the CLI and Celest Cloud as it deploys your projects.
celest-dev · Oct 10, 2024 · 0d39d8c · 0d39d8c
1 parent 7d5592b
commit 0d39d8c
Show file tree

Hide file tree

Showing 73 changed files with 36,420 additions and 7 deletions.
diff --git a/.github/workflows/celest_ast.yaml b/.github/workflows/celest_ast.yaml
@@ -0,0 +1,36 @@
+name: celest_ast
+on:
+  pull_request:
+    paths:
+      - ".github/workflows/celest_ast.yaml"
+      - "packages/celest_ast/**"
+
+# Prevent duplicate runs due to Graphite
+# https://graphite.dev/docs/troubleshooting#why-are-my-actions-running-twice
+concurrency:
+  group: ${{ github.repository }}-${{ github.workflow }}-${{ github.ref }}-${{ github.ref == 'refs/heads/main' && github.sha || ''}}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Git Checkout
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # 4.1.7
+        with:
+          submodules: recursive
+      - name: Setup Flutter
+        uses: subosito/flutter-action@44ac965b96f18d999802d4b807e3256d5a3f9fa1 # 2.16.0
+        with:
+          cache: true
+      - name: Setup Melos
+        run: dart pub global activate melos
+      - name: Get Packages
+        run: melos bootstrap
+      - name: Analyze
+        working-directory: packages/celest_ast
+        run: dart analyze --fatal-infos --fatal-warnings .
+      - name: Format
+        working-directory: packages/celest_ast
+        run: dart format --set-exit-if-changed .
diff --git a/README.md b/README.md
@@ -59,7 +59,7 @@ flutter_app/
     └── test/                             # Tests for your backend
 ```
 
-To start building your serverless cloud function, navigate to the `my_celest_app/celest/functions/` folder and create a file named `<api_name>.dart`. You can create as many APIs as you want in this directory. 
+To start building your serverless cloud function, navigate to the `my_celest_app/celest/functions/` folder and create a file named `<api_name>.dart`. You can create as many APIs as you want in this directory.
 Each file groups and organizes multiple Celest Functions of similar functionality into a namespace.
 
 Celest Functions are defined as top-level functions as shown below.
@@ -131,12 +131,13 @@ You have now set up your Celest project and integrated it into your Flutter app.
 
 ## Packages
 
-| Package                                    | Description                                                              | Pub                                                                                                                                | Checks                                                                                                                                                                                |
-| ------------------------------------------ | ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| [celest](packages/celest/)                 | The main package for defining Celest backends.                           | <a href="https://pub.dev/packages/celest" target="_blank"> <img src="https://img.shields.io/pub/v/celest.svg"></a>                 | [![Celest](https://github.com/celest-dev/celest/actions/workflows/celest.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest.yaml)                         |
-| [celest_auth](packages/celest_auth/)       | The authentication and authorization runtimes for Celest.                | <a href="https://pub.dev/packages/celest_auth" target="_blank"> <img src="https://img.shields.io/pub/v/celest_auth.svg"></a>       | [![Celest Auth](https://github.com/celest-dev/celest/actions/workflows/celest_auth.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest_auth.yaml)          |
-| [celest_cloud](packages/celest_cloud/)       | API contracts and Dart clients for the Celest Cloud platform.                 | <a href="https://pub.dev/packages/celest_cloud" target="_blank"> <img src="https://img.shields.io/pub/v/celest_cloud.svg"></a>       | [![Celest Cloud](https://github.com/celest-dev/celest/actions/workflows/celest_cloud.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest_cloud.yaml)          |
-| [celest_core](packages/celest_core/)       | Core types and utilities shared between Celest packages.                 | <a href="https://pub.dev/packages/celest_core" target="_blank"> <img src="https://img.shields.io/pub/v/celest_core.svg"></a>       | [![Celest Core](https://github.com/celest-dev/celest/actions/workflows/celest_core.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest_core.yaml)          |
+| Package                                | Description                                                   | Pub                                                                                                                            | Checks                                                                                                                                                                          |
+| -------------------------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [celest](packages/celest/)             | The main package for defining Celest backends.                | <a href="https://pub.dev/packages/celest" target="_blank"> <img src="https://img.shields.io/pub/v/celest.svg"></a>             | [![Celest](https://github.com/celest-dev/celest/actions/workflows/celest.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest.yaml)                   |
+| [celest_ast](packages/celest_ast/)     | A structured representation of Celest projects.               | <a href="https://pub.dev/packages/celest_ast" target="_blank"> <img src="https://img.shields.io/pub/v/celest_ast.svg"></a>     | [![Celest AST](https://github.com/celest-dev/celest/actions/workflows/celest_ast.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest_ast.yaml)      |
+| [celest_auth](packages/celest_auth/)   | The authentication and authorization runtimes for Celest.     | <a href="https://pub.dev/packages/celest_auth" target="_blank"> <img src="https://img.shields.io/pub/v/celest_auth.svg"></a>   | [![Celest Auth](https://github.com/celest-dev/celest/actions/workflows/celest_auth.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest_auth.yaml)    |
+| [celest_cloud](packages/celest_cloud/) | API contracts and Dart clients for the Celest Cloud platform. | <a href="https://pub.dev/packages/celest_cloud" target="_blank"> <img src="https://img.shields.io/pub/v/celest_cloud.svg"></a> | [![Celest Cloud](https://github.com/celest-dev/celest/actions/workflows/celest_cloud.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest_cloud.yaml) |
+| [celest_core](packages/celest_core/)   | Core types and utilities shared between Celest packages.      | <a href="https://pub.dev/packages/celest_core" target="_blank"> <img src="https://img.shields.io/pub/v/celest_core.svg"></a>   | [![Celest Core](https://github.com/celest-dev/celest/actions/workflows/celest_core.yaml/badge.svg)](https://github.com/celest-dev/celest/actions/workflows/celest_core.yaml)    |
 
 ## License
 

diff --git a/packages/celest_ast/.gitignore b/packages/celest_ast/.gitignore
@@ -0,0 +1,7 @@
+# https://dart.dev/guides/libraries/private-files
+# Created by `dart pub`
+.dart_tool/
+
+# Avoid committing pubspec.lock for library packages; see
+# https://dart.dev/guides/libraries/private-files#pubspeclock.
+pubspec.lock
diff --git a/packages/celest_ast/CHANGELOG.md b/packages/celest_ast/CHANGELOG.md
@@ -0,0 +1,3 @@
+## 0.1.2
+
+- Initial version.
diff --git a/packages/celest_ast/LICENSE b/packages/celest_ast/LICENSE
@@ -0,0 +1,46 @@
+Copyright (c) 2024 Teo, Inc. (Celest)
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+this list of conditions and the following disclaimer in the documentation and/or
+other materials provided with the distribution.
+
+Subject to the terms and conditions of this license, each copyright holder and
+contributor hereby grants to those receiving rights under this license a
+perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+(except for failure to satisfy the conditions of this license) patent license to
+make, have made, use, offer to sell, sell, import, and otherwise transfer this
+software, where such license applies only to those patent claims, already
+acquired or hereafter acquired, licensable by such copyright holder or
+contributor that are necessarily infringed by:
+
+(a) their Contribution(s) (the licensed copyrights of copyright holders and
+non-copyrightable additions of contributors, in source or binary form) alone; or
+
+(b) combination of their Contribution(s) with the work of authorship to which
+such Contribution(s) was added by such copyright holder or contributor, if, at
+the time the Contribution is added, such addition causes such combination to be
+necessarily infringed. The patent license shall not apply to any other
+combinations which include the Contribution.
+
+Except as expressly stated above, no rights or licenses from any copyright
+holder or contributor is granted under this license, whether expressly, by
+implication, estoppel or otherwise.
+
+DISCLAIMER
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
+TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/packages/celest_ast/Makefile b/packages/celest_ast/Makefile
@@ -0,0 +1,8 @@
+.PHONY: all
+all:
+	@echo "Generating protobuf files..."
+	buf generate
+	dart format .
+	@echo "Generating built_value files..."
+	dart run build_runner build --delete-conflicting-outputs
+	@echo "Done"
diff --git a/packages/celest_ast/README.md b/packages/celest_ast/README.md
@@ -0,0 +1,17 @@
+# Celest AST
+
+A structured representation of Celest projects and services.
+
+There are two representations of Celest projects in this package: `Project` and `ResolvedProject`. The Celest CLI manipulates a `Project` model
+as it parses and interprets your Celest project. At this stage, there still may be missing pieces--for example, the value of environment variables
+which have not been provided yet. Once the CLI has collected the necessary information to piece together all the project components, it transforms
+the `Project` model into a `ResolvedProject` which is consumed by Celest Cloud and the Celest runtime. Once a project has been resolved, it contains
+all the necessary information to run and deploy as a Celest service.
+
+## Protobufs
+
+The `ResolvedProject` model is available as both a Dart type and a Protobuf message. The Dart type serves to provide a more ergonomic interface for
+walking and manipulating the project model, while the Protobuf message is used for serializing and deserializing the project model across languages
+and across the wire.
+
+To regenerate the protobufs, run `make` from the package root.
diff --git a/packages/celest_ast/analysis_options.yaml b/packages/celest_ast/analysis_options.yaml
@@ -0,0 +1,5 @@
+include: package:lints/recommended.yaml
+
+analyzer:
+  errors:
+    implementation_imports: ignore
diff --git a/packages/celest_ast/buf.gen.yaml b/packages/celest_ast/buf.gen.yaml
@@ -0,0 +1,10 @@
+version: v2
+inputs:
+  - directory: proto
+  - module: buf.build/celest-dev/cedar
+plugins:
+  - remote: buf.build/protocolbuffers/dart:v21.1.2
+    include_imports: true
+    include_wkt: true
+    out: lib/src/proto
+    opt: grpc
diff --git a/packages/celest_ast/buf.lock b/packages/celest_ast/buf.lock
@@ -0,0 +1,9 @@
+# Generated by buf. DO NOT EDIT.
+version: v2
+deps:
+  - name: buf.build/celest-dev/cedar
+    commit: 37266fb8032240bc887fea377784e864
+    digest: b5:d69e51c2a8a2e037c1f5fceb96f0420c4f6e8ae8d44d1de427380070d5f6119175dcaf2eb7c7d5385ebde266d60d500744953a8bf81881bbb02d9c0237766462
+  - name: buf.build/googleapis/googleapis
+    commit: e7f8d366f5264595bcc4cd4139af9973
+    digest: b5:0cd69a689ee320ed815663d57d1bc3a1d6823224a7a717d46fee3a68197c25a6f5f932c0b0e49f8370c70c247a6635969a6a54af5345cafd51e0667298768aca
diff --git a/packages/celest_ast/buf.yaml b/packages/celest_ast/buf.yaml
@@ -0,0 +1,14 @@
+version: v2
+deps:
+  - buf.build/celest-dev/cedar
+  - buf.build/googleapis/googleapis
+modules:
+  - path: proto
+    name: buf.build/celest-dev/ast
+breaking:
+  use:
+    - PACKAGE
+lint:
+  use:
+    - MINIMAL
+  disallow_comment_ignores: true
diff --git a/packages/celest_ast/lib/celest_ast.dart b/packages/celest_ast/lib/celest_ast.dart
@@ -0,0 +1,6 @@
+export 'src/ast.dart';
+export 'src/dart_value.dart';
+export 'src/resolved_ast.dart';
+export 'src/sdk_configuration.dart';
+export 'src/serializers.dart';
+export 'src/visitor.dart';