Skip to content

Commit

Permalink
docs(examples): add NATS triggers in triggers-getting-started (#66)
Browse files Browse the repository at this point in the history
* docs(examples): add NATS triggers in triggers-getting-started

* fix: add missing comma

* docs: update README
  • Loading branch information
Bemilie authored Jan 11, 2024
1 parent 3b63d8f commit e8db0a0
Show file tree
Hide file tree
Showing 11 changed files with 131 additions and 31 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/go-function-offline-testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
matrix:
tests: [
{ directory: 'cors-go', go-version: '1.19' },
{ directory: 'go-hello-world', go-version: '1.20' }
{ directory: 'go-hello-world', go-version: '1.20' },
{ directory: 'go-mnq-sqs-publish', go-version: '1.18' },
{ directory: 'go-upload-file-s3-multipart', go-version: '1.20' }
]
Expand Down
21 changes: 21 additions & 0 deletions functions/triggers-getting-started/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 32 additions & 12 deletions functions/triggers-getting-started/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
# Triggers Getting Started

Simple starter examples that showcase using SQS triggers in all Scaleway Functions supported languages.
Simple starter examples that showcase using SQS triggers or NATS triggers in all Scaleway Functions supported languages.

In each example, a function is triggered by a SQS queue with a message containing a number. The function will then print the factorial of this number to the logs.
In each example, a function is triggered by a SQS queue or a NATS subject with a message containing a number.
The function will then print the factorial of this number to the logs.

## Requirements

This example requires [Terraform](https://www.scaleway.com/en/docs/tutorials/terraform-quickstart/).

Also, SQS **must** be [activated](https://www.scaleway.com/en/docs/serverless/messaging/how-to/get-started/#how-to-activate-sqs-or-sns) on your project.
Also,[SQS](https://www.scaleway.com/en/docs/serverless/messaging/how-to/get-started/#how-to-activate-sqs-or-sns) **must** be activated on your project,
and you **must** have a [NATS account](https://www.scaleway.com/en/docs/serverless/messaging/how-to/get-started/#how-to-create-a-nats-account).

## Setup

The Terraform configuration will deploy a function for each language, showing how to use triggers with each language.

It will also create a SQS queue per function to trigger it.
It will also create a SQS queue per function and a NATS account to trigger it.

After exporting `SCW_ACCESS_KEY`, `SCW_SECRET_KEY` and `SCW_DEFAULT_PROJECT_ID` variables (for authentication), you can:

Expand All @@ -25,22 +27,31 @@ terraform apply

You should be able to see your resources in the Scaleway console:

- Queues can be found in the [MnQ section](https://console.scaleway.com/messaging/protocols/fr-par/sqs/queues)
- Queues can be found in the Messaging section under [SQS](https://console.scaleway.com/messaging/protocols/fr-par/sqs/queues)
- NATS account can be found in the Messaging section under [NATS](https://console.scaleway.com/messaging/protocols/fr-par/nats/accounts)
- Functions can be found in the `triggers-getting-started` namespace in the [Serverless functions section](https://console.scaleway.com/functions/namespaces)

## Running

You can trigger your functions by sending messages to any of the associated SQS queues. Below is a description of how to do this with our dummy `tests/send_messages.py` script.
You can trigger your functions by sending messages to any of the associated SQS queues or NATS subject.
Below is a description of how to do this with our dummy `tests/send_[sqs|nats]_messages.py` script.

### Setup

First you need to expose your SQS access and secret keys:
First, you need to expose your SQS access and secret keys:

```console
export AWS_ACCESS_KEY_ID=$(terraform output -raw sqs_access_key)
export AWS_SECRET_ACCESS_KEY=$(terraform output -raw sqs_secret_key)
```

Or configure NATS credentials:

```console
export NATS_CREDS_FILE=$(terraform output -raw creds_file)
export SUBJECT_NAME=$(terraform output -raw subject_name)
```

Then you can set up a Python environment in the `tests` directory, e.g.

```console
Expand All @@ -52,17 +63,24 @@ pip3 install -r requirements.txt

### Sending messages

Now you can run the `send_messages.py` script to send a message to the SQS queue of each function:
Now you can run the `send_sqs_messages.py` script to send a message to the SQS queue of each function:

```console
python3 send_sqs_messages.py
```

Or you can run the `send_nats_messages.py` script to send a message to the NATS subject:

```console
python3 send_messages.py
python3 send_nats_messages.py
```

### Viewing function logs

In your [Cockpit](https://console.scaleway.com/cockpit), you can access the logs from your queues and functions.
In your [Cockpit](https://console.scaleway.com/cockpit), you can access the logs from your queues, NATS account and functions.

Navigate from your Cockpit to Grafana, and find the `Serverless Functions Logs` dashboard. There you should see something like the following, showing that your functions were triggered by messages on the queues:
Navigate from your Cockpit to Grafana, and find the `Serverless Functions Logs` dashboard.
There you should see something like the following, showing that your functions were triggered by messages on the queues/NATS subject:

```console
...
Expand All @@ -84,7 +102,9 @@ If you don't see these logs, make sure you have selected one of the "triggers ge

### Costs

Configuring and managing triggers is free, you only pay for the execution of your functions. For more information, please consult the [Scaleway Serverless pricing](https://www.scaleway.com/en/pricing/?tags=serverless). You will also be billed for the SQS queue usage when sending messages to it.
Configuring and managing triggers is free, you only pay for the execution of your functions.
For more information, please consult the [Scaleway Serverless pricing](https://www.scaleway.com/en/pricing/?tags=serverless).
You will also be billed for the SQS queue usage and NATS account when sending messages to it.

## Cleanup

Expand Down
21 changes: 12 additions & 9 deletions functions/triggers-getting-started/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ locals {
handler = "handler"
}
}
subject_name = "triggers-nats-factorial"
}

resource "scaleway_function_namespace" "main" {
Expand Down Expand Up @@ -59,21 +60,23 @@ resource "scaleway_function" "main" {
min_scale = 0
}

resource "scaleway_mnq_sqs_queue" "main" {
resource "scaleway_function_trigger" "main_sqs" {
for_each = local.functions

name = "factorial-requests-${each.key}"

access_key = scaleway_mnq_sqs_credentials.main.access_key
secret_key = scaleway_mnq_sqs_credentials.main.secret_key
function_id = scaleway_function.main[each.key].id
name = "sqs-on-factorial-request"
sqs {
queue = scaleway_mnq_sqs_queue.main[each.key].name
}
}

resource "scaleway_function_trigger" "main" {
resource "scaleway_function_trigger" "main_nats" {
for_each = local.functions

function_id = scaleway_function.main[each.key].id
name = "on-factorial-request"
sqs {
queue = scaleway_mnq_sqs_queue.main[each.key].name
name = "nats-on-factorial-request"
nats {
account_id = scaleway_mnq_nats_account.main.id
subject = local.subject_name
}
}
8 changes: 0 additions & 8 deletions functions/triggers-getting-started/messaging.tf

This file was deleted.

13 changes: 13 additions & 0 deletions functions/triggers-getting-started/nats.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
resource "scaleway_mnq_nats_account" "main" {
name = "nats-account"
}

resource "scaleway_mnq_nats_credentials" "main" {
account_id = scaleway_mnq_nats_account.main.id
name = "triggers-nats"
}

resource "local_sensitive_file" "nats" {
filename = "triggers-nats.creds"
content = scaleway_mnq_nats_credentials.main.file
}
10 changes: 9 additions & 1 deletion functions/triggers-getting-started/outputs.tf
Original file line number Diff line number Diff line change
@@ -1,9 +1,17 @@
output "sqs_access_key" {
value = scaleway_mnq_sqs_credentials.main.access_key
value = scaleway_mnq_sqs_credentials.main.access_key
sensitive = true
}

output "sqs_secret_key" {
value = scaleway_mnq_sqs_credentials.main.secret_key
sensitive = true
}

output "subject_name" {
value = local.subject_name
}

output "creds_file" {
value = local_sensitive_file.nats.filename
}
17 changes: 17 additions & 0 deletions functions/triggers-getting-started/sqs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
resource "scaleway_mnq_sqs_credentials" "main" {
name = "triggers-getting-started"
permissions {
can_publish = true
can_receive = true
can_manage = true
}
}

resource "scaleway_mnq_sqs_queue" "main" {
for_each = local.functions

name = "factorial-requests-${each.key}"

access_key = scaleway_mnq_sqs_credentials.main.access_key
secret_key = scaleway_mnq_sqs_credentials.main.secret_key
}
2 changes: 2 additions & 0 deletions functions/triggers-getting-started/tests/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@
boto3<1.28.81
botocore<1.31.81
requests~=2.31.0
nats-py==2.6.0
nkeys==0.1.0
24 changes: 24 additions & 0 deletions functions/triggers-getting-started/tests/send_nats_messages.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import os

import asyncio
import nats

MAX_FACTORIAL_REQUESTS = 20

# Credentials file generated by Terraform
NATS_CREDS_FILE = os.environ["NATS_CREDS_FILE"]
NATS_ENDPOINT_URL = "nats://nats.mnq.fr-par.scaleway.com:4222"
SUBJECT_NAME = os.environ["NATS_SUBJECT"]

async def send_nats():
nc = await nats.connect(NATS_ENDPOINT_URL, user_credentials=NATS_CREDS_FILE)

print(f"Sending 0..{MAX_FACTORIAL_REQUESTS} factorial requests to {SUBJECT_NAME}...")
for i in range(MAX_FACTORIAL_REQUESTS):
await nc.publish(SUBJECT_NAME, f"{i}".encode())

await nc.close()


if __name__ == "__main__":
asyncio.run(send_nats())

0 comments on commit e8db0a0

Please sign in to comment.