Skip to content

Commit

Permalink
feat(config): add loki config (#20)
Browse files Browse the repository at this point in the history
  • Loading branch information
PierreDemailly authored Aug 2, 2023
1 parent e5eefce commit 0bbcc73
Show file tree
Hide file tree
Showing 6 changed files with 168 additions and 52 deletions.
12 changes: 6 additions & 6 deletions src/agent/src/rules.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// Import Third-party Dependencies
import { SigynRule } from "@sigyn/config";
import { SigynRule, getConfig } from "@sigyn/config";
import { GrafanaLoki } from "@myunisoft/loki";
import dayjs from "dayjs";
import ms from "ms";
Expand All @@ -11,10 +11,6 @@ import * as utils from "./utils";
import { createAlert } from "./alert";
import { Logger } from ".";

// CONSTANTS
const kApi = new GrafanaLoki({
remoteApiURL: "https://loki.myunisoft.fr"
});

export interface RuleOptions {
logger: Logger;
Expand Down Expand Up @@ -48,7 +44,11 @@ export class Rule {

async handleLogs(): Promise<void> {
const db = getDB();
const logs = await kApi.queryRange(this.#config.logql, {
const lokiApi = new GrafanaLoki({
remoteApiURL: getConfig().loki.apiUrl
});

const logs = await lokiApi.queryRange(this.#config.logql, {
start: this.#getQueryRangeStartUnixTimestamp()
});
const rule = this.#getRuleFromDatabase();
Expand Down
92 changes: 52 additions & 40 deletions src/config/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,50 +28,59 @@

## ⚙️ Configuration

The **Sigyn** configuration object consists of two main properties: `rules` and `notifiers`. The `rules` property defines an array of rule objects, each representing a specific monitoring rule.
The **Sigyn** configuration object consists of three main properties: `loki`, `rules` and `notifiers`.
The `loki` property defines an object that allows configuring Loki API access.
The `rules` property defines an array of rule objects, each representing a specific monitoring rule.
The `notifiers` property is an object that allows configuring various notification methods.

### Schema Properties

1. `rules` (Required, Array of Objects):
- This property holds an array of monitoring rules.
- Each rule object must have the following properties:

| Property | Type | Required | Description |
|-------------|------------------------|----------|-------------|
| `name` | `string` | ✔️ | The name of the rule. Must be unique between each rule. |
| `logql` | `string` | ✔️ | The LogQL query associated with the rule. |
| `polling` | `string` or `string[]` | ✔️ | The polling interval for the rule. You can use a `duration` i.e. `2m` or a **Cron expression**. If given an array of polling, it should only be **Cron expressions**, this is usefull if you want a different polling the day and the night. |
| `alert` | `object` | ✔️ | An object defining the alerting configuration for the rule. |
| `disabled` | `boolean` || Weither the rule is enabled, default to `false`. |
| `notifiers` | `string[]` || An array of strings representing the notifiers for the rule. |

2. `alert` (Object, Required):
- This object specifies the alerting configuration for the rule.
- It must have the following properties:

| Property | Type | Required | Description |
|------------|----------|----------|-------------|
| `on` | `object` | ✔️ | An object specifying when the alert should trigger. |
| `template` | `object` | ✔️ | An object representing the notification template. |

3. `on` (Object, Required):
- An object specifying when the alert should trigger.
- It must have the following properties:

| Property | Type | Required | Description |
|------------|----------------------|----------|-------------|
| `count` | `number` or `string` | ✔️ | The count threshold of log that must triggers an alert. You can use a range string i.e. `<= 5`, `> 6`. |
| `interval` | `string` | ✔️ | The time interval for the alerting condition. |

4. `template` (Object, Required):
- An object representing the notification template.
- It can have either of the following properties:

| Property | Type | Required | Description |
|------------|------------|----------|-------------|
| `title` | `string` || The title of the notification template. |
| `content` | `string[]` || The content of the notification template. |
1. `loki` (Object, Required):
- This object specifies the Loki API configuration.

| Property | Type | Required | Description |
|------------|------------|----------|-------------|
| `apiUrl` | `string` | ✔️ | The Loki API url |

2. `rules` (Required, Array of Objects):
- This property holds an array of monitoring rules.
- Each rule object must have the following properties:

| Property | Type | Required | Description |
|-------------|------------------------|----------|-------------|
| `name` | `string` | ✔️ | The name of the rule. Must be unique between each rule. |
| `logql` | `string` | ✔️ | The LogQL query associated with the rule. |
| `polling` | `string` or `string[]` | ✔️ | The polling interval for the rule. You can use a `duration` i.e. `2m` or a **Cron expression**. If given an array of polling, it should only be **Cron expressions**, this is usefull if you want a different polling the day and the night. |
| `alert` | `object` | ✔️ | An object defining the alerting configuration for the rule. |
| `disabled` | `boolean` || Weither the rule is enabled, default to `false`. |
| `notifiers` | `string[]` || An array of strings representing the notifiers for the rule. It will enables all configured `notifiers` by default. |

3. `rules.alert` (Object, Required):
- This object specifies the alerting configuration for the rule.
- It must have the following properties:

| Property | Type | Required | Description |
|------------|----------|----------|-------------|
| `on` | `object` | ✔️ | An object specifying when the alert should trigger. |
| `template` | `object` | ✔️ | An object representing the notification template. |

4. `rules.alert.on` (Object, Required):
- An object specifying when the alert should trigger.
- It must have the following properties:

| Property | Type | Required | Description |
|------------|----------------------|----------|-------------|
| `count` | `number` or `string` | ✔️ | The count threshold of log that must triggers an alert. You can use a range string i.e. `<= 5`, `> 6`. |
| `interval` | `string` | ✔️ | The time interval for the alerting condition. |

5. `rules.alert.template` (Object, Required):
- An object representing the notification template.
- It can have either of the following properties:

| Property | Type | Required | Description |
|------------|------------|----------|-------------|
| `title` | `string` || The title of the notification template. |
| `content` | `string[]` || The content of the notification template. |

> **Note** At least one of `title` or `content` must be provided.
Expand All @@ -98,6 +107,9 @@ You can use any of theses variables, surrounding with `{}` (see example below):

```json
{
"loki": {
"apiUrl": "http://localhost:3100"
},
"notifiers": {
"slack": {
"webhookUrl": "https://hooks.slack.com/services/aaa/bbb"
Expand Down
4 changes: 2 additions & 2 deletions src/config/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
import fs from "node:fs";

// Import Internal Dependencies
import { SigynConfig, SigynRule, SigynAlert, SigynAlertTemplate } from "./types";
import { SigynConfig, LokiConfig, SigynRule, SigynAlert, SigynAlertTemplate } from "./types";
import { validate } from "./validate";

export { SigynConfig, SigynRule, SigynAlert, SigynAlertTemplate };
export { SigynConfig, LokiConfig, SigynRule, SigynAlert, SigynAlertTemplate };

let config: SigynConfig;

Expand Down
11 changes: 9 additions & 2 deletions src/config/src/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,13 @@ const ruleAlertSchema: JSONSchemaType<SigynAlert> = {
export const CONFIG_SCHEMA: JSONSchemaType<SigynConfig> = {
type: "object",
properties: {
loki: {
type: "object",
properties: {
apiUrl: { type: "string", minLength: 1 }
},
required: ["apiUrl"]
},
rules: {
type: "array",
uniqueItemProperties: ["name"],
Expand All @@ -59,7 +66,7 @@ export const CONFIG_SCHEMA: JSONSchemaType<SigynConfig> = {
polling: {
oneOf: [
{ type: "string", minLength: 1 },
{ type: "array", items: { type: "string", minLength: 1 }, minLength: 1 }
{ type: "array", items: { type: "string", minLength: 1 }, minItems: 1 }
]
},
alert: ruleAlertSchema,
Expand All @@ -78,6 +85,6 @@ export const CONFIG_SCHEMA: JSONSchemaType<SigynConfig> = {
additionalProperties: true
}
},
required: ["rules", "notifiers"],
required: ["loki", "rules", "notifiers"],
additionalProperties: false
};
5 changes: 5 additions & 0 deletions src/config/src/types.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
export interface SigynConfig {
loki: LokiConfig;
notifiers: Record<string, unknown>;
rules: SigynRule[]
}

export interface LokiConfig {
apiUrl: string;
}

export interface SigynRule {
name: string;
logql: string;
Expand Down
96 changes: 94 additions & 2 deletions src/config/test/validation.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ import { validate } from "../src/validate";
import { SigynConfig } from "../src/types";

const kValidConfig: SigynConfig = {
loki: {
apiUrl: "http://localhost:3100"
},
notifiers: {
foo: {
bar: "baz"
Expand Down Expand Up @@ -42,6 +45,44 @@ describe("Config validation", () => {
});
});

it("given a config without loki", () => {
assert.throws(() => {
validate({
...kValidConfig,
loki: undefined as any
});
}, {
name: "Error",
message: "Invalid config: : must have required property 'loki'"
});
});

it("given a config without loki apiUrl, it should throws", () => {
assert.throws(() => {
validate({
...kValidConfig,
loki: {} as any
});
}, {
name: "Error",
message: "Invalid config: /loki: must have required property 'apiUrl'"
});
});

it("loki apiUrl must be string", () => {
assert.throws(() => {
validate({
...kValidConfig,
loki: {
apiUrl: 42 as any
}
});
}, {
name: "Error",
message: "Invalid config: /loki/apiUrl: must be string"
});
});

it("given a rule template with only title, it should validate", () => {
assert.doesNotThrow(() => {
validate({
Expand Down Expand Up @@ -296,7 +337,7 @@ describe("Config validation", () => {
});
});

it("rule polling should not be empty", () => {
it("rule polling should not be empty string", () => {
assert.throws(() => {
validate({
...kValidConfig,
Expand All @@ -309,7 +350,58 @@ describe("Config validation", () => {
});
}, {
name: "Error",
message: "Invalid config: /rules/0/polling: must NOT have fewer than 1 characters"
// eslint-disable-next-line max-len
message: "Invalid config: /rules/0/polling: must NOT have fewer than 1 characters, /rules/0/polling: must be array, /rules/0/polling: must match exactly one schema in oneOf"
});
});

it("rule polling should not be empty array", () => {
assert.throws(() => {
validate({
...kValidConfig,
rules: [
{
...kValidConfig.rules[0],
polling: []
}
]
});
}, {
name: "Error",
// eslint-disable-next-line max-len
message: "Invalid config: /rules/0/polling: must be string, /rules/0/polling: must NOT have fewer than 1 items, /rules/0/polling: must match exactly one schema in oneOf"
});
});

it("rule polling should not be empty string in array", () => {
assert.throws(() => {
validate({
...kValidConfig,
rules: [
{
...kValidConfig.rules[0],
polling: ["* * * * *", ""]
}
]
});
}, {
name: "Error",
// eslint-disable-next-line max-len
message: "Invalid config: /rules/0/polling: must be string, /rules/0/polling/1: must NOT have fewer than 1 characters, /rules/0/polling: must match exactly one schema in oneOf"
});
});

it("rule polling can be string array", () => {
assert.doesNotThrow(() => {
validate({
...kValidConfig,
rules: [
{
...kValidConfig.rules[0],
polling: ["* * * * *"]
}
]
});
});
});

Expand Down

0 comments on commit 0bbcc73

Please sign in to comment.