-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Gerd Zellweger <[email protected]>
- Loading branch information
Showing
3 changed files
with
58 additions
and
43 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,70 +1,84 @@ | ||
# 1bln Cell Spreadsheet | ||
# 1bln Cell Spreadsheet Techdemo | ||
|
||
This is a tech demo showing incremental computation for a simple, spreadsheet-like application. | ||
The application uses feldera as the DBMS/incremental compute engine, and axum as the backend | ||
The application uses feldera as the DBMS/incremental compute engine, axum as the backend | ||
and egui as the frontend. | ||
|
||
## Local development | ||
The project is split into three components from the root directory: | ||
|
||
You'll need to have rust, `cargo`, `fda` and `trunk` (last two both through `cargo install`) | ||
installed to run this application locally. | ||
- The `feldera` directory contains the feldera pipeline (written in Feldera/SQL and some Rust UDF code). | ||
- The `server` directory contains the backend application (written in Rust using the axum webserver). | ||
- The `client` directory contains the frontend application (written in Rust using the egui UI library). | ||
|
||
Run the backend fom the `server` directory: | ||
## Local Installation | ||
|
||
```bash | ||
cd server | ||
RUST_LOG=debug FELDERA_KEY='' cargo run | ||
``` | ||
You'll need a working rust installation to run the project locally. | ||
|
||
To access the feldera instance if already defined in your environment `FELDERA_HOST` and `FELDERA_KEY`, take | ||
precedence. You can unset these variable to use the local instance: | ||
### Feldera | ||
|
||
```bash | ||
unset FELDERA_KEY | ||
unset FELDERA_HOST | ||
``` | ||
Install a [feldera instance](https://docs.feldera.com/get-started) and or alternatively use the | ||
feldera instance running on `https://try.feldera.com`. | ||
Also install the CLI tool [fda](https://docs.feldera.com/reference/cli). | ||
|
||
Run the frontend from the `client` directory: | ||
Then make sure to set the `FELDERA_API_KEY` and `FELDERA_HOST` environment variables to access the feldera instance. | ||
Execute the `deploy.sh` script in the `feldera` directory to deploy the pipeline to the feldera instance. | ||
|
||
```bash | ||
cd client | ||
trunk serve --port 7777 | ||
export FELDERA_API_KEY=apikey:... | ||
export FELDERA_HOST=https://try.feldera.com | ||
cd feldera && bash deploy.sh | ||
``` | ||
|
||
Run the database from the `feldera` directory: | ||
### Server | ||
|
||
Make sure you have a feldera instance running on `http://localhost:8080`. | ||
Run the `server` application with cargo: | ||
|
||
```bash | ||
git clone https://github.com/feldera/feldera.git feldera-service | ||
cd feldera-service | ||
cd sql-to-dbsp-compiler && ./build.sh | ||
cd .. && cargo run --package=pipeline-manager --features pg-embed --bin pipeline-manager | ||
cd server | ||
cargo run | ||
``` | ||
|
||
Make sure the `fda` tool is installed to deploy the SQL to the feldera instance: | ||
Now the backend should be running on `http://localhost:3000`. | ||
|
||
### Client | ||
|
||
Run the `client` application with trunk: | ||
|
||
```bash | ||
cd feldera | ||
export FELDERA_HOST=http://localhost:3000 | ||
export FELDERA_API_KEY= | ||
bash deploy.sh | ||
cd client | ||
API_HOST=http://localhost:3000 trunk serve --port 7777 | ||
``` | ||
|
||
Once all three components are running: | ||
Now the frontend should be running on `http://localhost:7777`. | ||
|
||
## Automated Deployment with Github Actions | ||
|
||
- Open [http://localhost:7777](http://localhost:7777) with your browser to see the spreadsheet. | ||
- Open [http://localhost:8080](http://localhost:8080) with your browser to see the feldera instance. | ||
- Open [http://localhost:3000/api/spreadsheet](http://localhost:3000/api/spreadsheet), or | ||
[http://localhost:3000/api/stats](http://localhost:3000/api/cellstream) to issue API calls. | ||
The project is set up to deploy the server application to fly.io and the client application to github pages. | ||
There are three github action files in the `.github/workflows` directory for deploying the feldera pipeline, | ||
the server application and the client application. | ||
|
||
## Deployment | ||
## Feldera | ||
|
||
- The `server` application is deployed to fly.io it includes and serves the web-assembly artefact built under `client`. | ||
- The `feldera` instance is deployed to run from `try.feldera.com`. An API key is added to the server application | ||
to access this instance. | ||
Make sure to set the `FELDERA_API_KEY` and `FELDERA_HOST` secrets in the github repository settings. | ||
|
||
## Server | ||
|
||
Get a fly.io account and install the [fly CLI tool](https://fly.io/docs/flyctl/install/). | ||
|
||
Next make sure to set the `FELDERA_API_KEY` and `FELDERA_HOST` secrets also in your fly.io application. | ||
|
||
```bash | ||
fly secrets set FELDERA_API_KEY=apikey:... | ||
cd server | ||
fly login | ||
fly secrets set FELDERA_HOST=https://try.feldera.com | ||
``` | ||
fly secrets set FELDERA_API_KEY=apikey:... | ||
``` | ||
|
||
Finally, you'll need to get an API token from fly.io and set it as a secret named `FLY_API_TOKEN` in the github | ||
repository settings. | ||
|
||
## Client | ||
|
||
Make sure to set the `API_HOST` secret in the github repository settings to point to your fly.io application URL. | ||
Enable github pages, set the source to `Github Actions`. Then adjust the `public_url` env variable in the `client.yml` | ||
github action file to point to your github pages URL. |