Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http2 gRPC go tutorial #83

Merged
merged 6 commits into from
Apr 24, 2024
Merged

http2 gRPC go tutorial #83

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5a93fa2
(wip) http2 gRPC go tutorial
thomas-tacquet Apr 18, 2024
fbafd6f
complete readme + tutorial files
thomas-tacquet Apr 19, 2024
bd442d4
add testing step
thomas-tacquet Apr 19, 2024
c93306e
replaced my container with placeholder
thomas-tacquet Apr 19, 2024
d86e243
Apply suggestions from code review
thomas-tacquet Apr 23, 2024
19f0108
smoothen packages
thomas-tacquet Apr 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 16 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Table of Contents:
- [🚀 Functions](#-functions)
- [📦 Containers](#-containers)
- [⚙️ Jobs](#️-jobs)
- [💬 Messaging and Queueing](#-mnq)
- [💬 Messaging and Queueing](#-messaging-and-queueing)
- [💜 Projects](#-projects)
- [Contributing](#contributing)

Expand All @@ -28,8 +28,9 @@ Table of Contents:
### 🚀 Functions

<!-- markdownlint-disable MD033 -->

| Example | Runtime | Deployment |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|------------------------|
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | ---------------------- |
| **[Badge PHP](functions/badge-php/README.md)** <br/> A PHP function to generate repository badges. | php82 | [Serverless Framework] |
| **[CORS Go](functions/cors-go/README.md)** <br/> A Go function which allows CORS requests. | go119 | [Serverless Framework] |
| **[CORS Node](functions/cors-node/README.md)** <br/> A Node function which allows CORS requests. | node18 | [Serverless Framework] |
Expand Down Expand Up @@ -57,12 +58,12 @@ Table of Contents:
| **[Typescript with Node runtime](functions/typescript-with-node/README.md)** <br/> A Typescript function using Node runtime. | node18 | [Serverless Framework] |
| **[Serverless Gateway Python Example](functions/serverless-gateway-python/README.md)** <br/> A Python serverless API using Serverless Gateway. | python310 | [Python API Framework] |
| **[Go and Transactional Email](functions/go-mail/README.md)** <br/> A Go function that send emails using Scaleway SDK. | go121 | [Serverless Framework] |
| **[Rotate RDB Credentials](functions/secret-manager-rotate-secret/README.md)** <br/> A Go function that rotates RDB credentials stored in Secret Manager. | go120 | [Serverless Framework] |
| **[Rotate RDB Credentials](functions/secret-manager-rotate-secret/README.md)** <br/> A Go function that rotates RDB credentials stored in Secret Manager. | go120 | [Serverless Framework] |

### 📦 Containers

| Example | Language | Deployment |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------------------|
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ---------------------- |
| **[Container Bash Script](containers/bash-scheduled-job/README.md)** <br/> A Bash script runnning on a schedule using serverless containers. | Bash | [Serverless Framework] |
| **[Function Handler Java](containers/function-handler-java/README.md)** <br/> A Java function handler deployed on CaaS. | Java | [Serverless Framework] |
| **[NGINX CORS Private](containers/nginx-cors-private-python/README.md)** <br/> An NGINX proxy to allow CORS requests to a private container. | Python Flask | [Terraform] |
Expand All @@ -71,26 +72,27 @@ Table of Contents:
| **[Python S3 upload](containers/python-s3-upload/README.md)** <br/> A Python + Flask HTTP server that receives file uploads and writes them to S3. | Python | [Terraform] |
| **[Terraform NGINX hello world](containers/terraform-nginx-hello-world/README.md)** <br/> A minimal example running the base NGINX image in a serverless container deployed with Terraform. | N/A | [Terraform] |
| **[Triggers with Terraform](containers/terraform-triggers/README.md)** <br/> Configuring two SQS triggers, used to trigger two containers, one public, one private. | N/A | [Terraform] |
| **[gRPC HTTP2 in Go](containers/grpc-http2-go/README.md)** <br/> A Go gRPC Container using http2 | Go/Protobuf | [CLI] |

### ⚙️ Jobs

| Example | Language | Deployment |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------------------|
| **[Serverless Jobs Hello World](jobs/terraform-hello-world/README.md)** <br/> An example of building a container image and running it as a Serverless Job using Terraform. | N/A |[Terraform]-[Console]|
| **[Serverless MLOps](jobs/ml-ops/README.md)** <br/> An example of running a Serverless Machine Leaning workflow. | Python |[Terraform]-[Console]-[CLI]|
| Example | Language | Deployment |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | --------------------------- |
| **[Serverless Jobs Hello World](jobs/terraform-hello-world/README.md)** <br/> An example of building a container image and running it as a Serverless Job using Terraform. | N/A | [Terraform]-[Console] |
| **[Serverless MLOps](jobs/ml-ops/README.md)** <br/> An example of running a Serverless Machine Leaning workflow. | Python | [Terraform]-[Console]-[CLI] |

### 💬 Messaging and Queueing

| Example | Services | Language | Deployment |
|---------------------------------------------------------------------------------------------------------------------------------------------------------|------------------|----------|-------------|
| **[Manage large message](mnq/large-messages/README.md)** <br/> An example of infrastructure to manage large messages. | PaaS & S3 | Python | [Terraform] |
| **[Serverless scraping](mnq/serverless-scraping/README.md)** <br/> An example of infrastructure to scrape the hackernews website. | PaaS & RDB | Python | [Terraform] |
| **[SNS Instances Notification System](mnq/sns-instances-notification-system/README.md)** <br/> An example of infrastructure to use SNS with Instances. | PaaS & Instances | Golang | [Terraform] |
| Example | Services | Language | Deployment |
| ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------- | -------- | ----------- |
| **[Manage large message](mnq/large-messages/README.md)** <br/> An example of infrastructure to manage large messages. | PaaS & S3 | Python | [Terraform] |
| **[Serverless scraping](mnq/serverless-scraping/README.md)** <br/> An example of infrastructure to scrape the hackernews website. | PaaS & RDB | Python | [Terraform] |
| **[SNS Instances Notification System](mnq/sns-instances-notification-system/README.md)** <br/> An example of infrastructure to use SNS with Instances. | PaaS & Instances | Golang | [Terraform] |

### 💜 Projects

| Example | Services | Language | Deployment |
|-------------------------------------------------------------------------------------------------------------------------------------------|-------------|----------|------------------------|
| ----------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -------- | ---------------------- |
| **[Kong API Gateway](projects/kong-api-gateway/README.md)** <br/> Deploying a Kong Gateway on containers to provide routing to functions. | CaaS & FaaS | Python | [Serverless Framework] |
| **[Serverless Gateway](https://github.com/scaleway/serverless-gateway)** <br/> Our serverless gateway for functions and containers. | API Gateway | Python | [Python API Framework] |
| **[Monitoring Glaciers](projects/blogpost-glacier/README.md)** <br/> A project to monitor glaciers and the impact of global warming. | S3 & RDB | Golang | [Serverless Framework] |
Expand Down
28 changes: 28 additions & 0 deletions containers/grpc-http2-go/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM golang:1.22-bookworm as build

# Set working directory
WORKDIR /app

# Copy required files
COPY go.mod .
COPY go.sum .
COPY server/*.go ./server/
COPY hello.proto ./

# We install the protobuf compilation stack on the image directly to simplify workflow
RUN apt-get update && apt-get install -y protobuf-compiler
RUN go install google.golang.org/protobuf/cmd/[email protected]
RUN go install google.golang.org/grpc/cmd/[email protected]

# Generate go protobuf files
RUN protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative hello.proto

# Build the executable
RUN GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build server/main.go

# Put the executable in a light scratch image
FROM scratch

COPY --from=build /app/main /server

ENTRYPOINT ["/server"]
91 changes: 91 additions & 0 deletions containers/grpc-http2-go/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# gRPC HTTP2 Server in Go using CLI

Example that demonstrates the deployment of a gRPC service on Scaleway Serverless Containers.

For this example we will use the CLI to deploy the container but you can use [other methods](https://www.scaleway.com/en/docs/serverless/containers/reference-content/deploy-container/).
You can also use the CLI directly from Scaleway console, it saves you operations of setting up your credentials.

## Container settings

To deploy a Serverless Container that uses the gRPC protocol it's important to enable http2 setting (named `h2c` in the API).

thomas-tacquet marked this conversation as resolved.
Show resolved Hide resolved
## Workflow

Here are the different steps we are going to proceed:

- Quick set-up of Container Registry to host our gRPC container
- Deploy the Serverless Container
- Run gRPC test command ot ensure everything is ok

## Deployment

### Requirements

- [Scaleway CLI](https://github.com/scaleway/scaleway-cli): be sure to be logged in
- **Docker** To build the image
- If you want to test locally, you will need to install the common gRPC stack (doc [here](https://grpc.io/blog/installation/))
but this is not required for deployment as everything is built in the Dockerfile
thomas-tacquet marked this conversation as resolved.
Show resolved Hide resolved

### Building the image

To store the image we need to create a namespace in Container Registery with the following command:

```bash
scw registry namespace create name=hello-grpc
```

As output it will give you some informations, keep the endpoint, on this case it's `rg.fr-par.scw.cloud/hello-grpc` (it can change in your case depending the region).

Now open a terminal to set-up Docker, it will allow you to send the built image in the Container Registry we created at the previous step.

Login command:

```bash
docker login rg.fr-par.scw.cloud/hello-grpc -u nologin --password-stdin <<< "$SCW_SECRET_KEY"
```

At this point you have correctly set up docker to be able to push your image online.

**Builb** the image by opening a terminal on your computer in this directory, and run:

```bash
docker build -t grpc:latest .
```

Now we will tag and push the image:

```bash
docker tag grpc:latest rg.fr-par.scw.cloud/hello-grpc/grpc:latest
docker push rg.fr-par.scw.cloud/hello-grpc/grpc:latest
```
thomas-tacquet marked this conversation as resolved.
Show resolved Hide resolved

### Deploying the image

Now we need to create a nemespace for the Serverless Container and the create the container:
thomas-tacquet marked this conversation as resolved.
Show resolved Hide resolved

```bash
scw container namespace create name=grpc-test
```

Save the ID of the container.

Now create and deploy the container with a single command:

```bash
scw container container create namespace-id=<PREVIOUS_NAMESPACE_ID> protocol=h2c name=grpc-test registry-image=rg.fr-par.scw.cloud/hello-grpc/grpc:latest
```

Save the DomainName (endpoint) for testing.

thomas-tacquet marked this conversation as resolved.
Show resolved Hide resolved
### Testing

Before testing, ensure your container is in status `ready`.

In `client/client.go` file, remplace the constant `containerEndpoint` with the DomainName. Do not forget to keep `:80` port at the end even
if you container port is set to 8080, these are two different settings.

And then execute `go run client/client.go' to check if your container reponds.
thomas-tacquet marked this conversation as resolved.
Show resolved Hide resolved

## Additional content

- [Basic Go gRPC tutorial](https://grpc.io/docs/languages/go/basics/)
36 changes: 36 additions & 0 deletions containers/grpc-http2-go/client/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package main

import (
"context"
"flag"
"log"
"time"

pb "github.com/scaleway/serverless-examples/containers/grpc-http2-go"
"google.golang.org/grpc"
)

const containerEndpoint = "YOUR_CONTAINER_ENDPOINT:80"

func main() {
flag.Parse()
// Set up a connection to the server.
conn, err := grpc.Dial(containerEndpoint, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}

defer conn.Close()
c := pb.NewGreeterClient(conn)

// Contact the server and print out its response.
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "Scaleway"})
if err != nil {
log.Fatalf("could not greet: %v", err)
}

log.Printf("Greeting: %s", r.GetMessage())
}
18 changes: 18 additions & 0 deletions containers/grpc-http2-go/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
module github.com/scaleway/serverless-examples/containers/grpc-http2-go

go 1.22.1

require (
github.com/scaleway/serverless-examples/containers/grpc v0.0.0-00010101000000-000000000000
google.golang.org/grpc v1.63.2
google.golang.org/protobuf v1.33.0
)

require (
golang.org/x/net v0.24.0 // indirect
golang.org/x/sys v0.19.0 // indirect
golang.org/x/text v0.14.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240415180920-8c6c420018be // indirect
)

replace github.com/scaleway/serverless-examples/containers/grpc => .
14 changes: 14 additions & 0 deletions containers/grpc-http2-go/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
golang.org/x/net v0.24.0 h1:1PcaxkF854Fu3+lvBIx5SYn9wRlBzzcnHZSiaFFAb0w=
golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8=
golang.org/x/sys v0.19.0 h1:q5f1RH2jigJ1MoAWp2KTp3gm5zAGFUTarQZ5U386+4o=
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240415180920-8c6c420018be h1:LG9vZxsWGOmUKieR8wPAUR3u3MpnYFQZROPIMaXh7/A=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240415180920-8c6c420018be/go.mod h1:WtryC6hu0hhx87FDGxWCDptyssuo68sk10vYjF+T9fY=
google.golang.org/grpc v1.63.2 h1:MUeiw1B2maTVZthpU5xvASfTh3LDbxHd6IJ6QQVU+xM=
google.golang.org/grpc v1.63.2/go.mod h1:WAX/8DgncnokcFUldAxq7GeB5DXHDbMF+lLvDomNkRA=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
Loading
Loading