Skip to content

Commit

Permalink
feat(fxworker): Provided module (#54)
Browse files Browse the repository at this point in the history
  • Loading branch information
ekkinox authored Jan 19, 2024
1 parent 7af5255 commit 71d2e5c
Show file tree
Hide file tree
Showing 24 changed files with 1,780 additions and 0 deletions.
1 change: 1 addition & 0 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ jobs:
- "fxmetrics"
- "fxorm"
- "fxtrace"
- "fxworker"
steps:
- name: Checkout
uses: actions/checkout@v3
Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/fxworker-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: "fxworker-ci"

on:
push:
branches:
- "feat**"
- "fix**"
- "hotfix**"
- "chore**"
paths:
- "fxworker/**.go"
- "fxworker/go.mod"
- "fxworker/go.sum"
pull_request:
types:
- opened
- synchronize
- reopened
branches:
- main
paths:
- "fxworker/**.go"
- "fxworker/go.mod"
- "fxworker/go.sum"

jobs:
ci:
uses: ./.github/workflows/common-ci.yml
secrets: inherit
with:
module: "fxworker"
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Yokai `Fx modules` are the plugins for your Yokai application.
| [fxmetrics](fxmetrics) | Fx module for [prometheus](https://github.com/prometheus/client_golang) |
| [fxorm](fxorm) | Fx module for [orm](orm) |
| [fxtrace](fxtrace) | Fx module for [trace](trace) |
| [fxworker](fxworker) | Fx module for [worker](worker) |

They can also be used in any [Fx](https://github.com/uber-go/fx) based Go application.

Expand Down
66 changes: 66 additions & 0 deletions fxworker/.golangci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
run:
timeout: 5m
concurrency: 8

linters:
enable:
- asasalint
- asciicheck
- bidichk
- bodyclose
- containedctx
- contextcheck
- cyclop
- decorder
- dogsled
- dupl
- durationcheck
- errcheck
- errchkjson
- errname
- errorlint
- exhaustive
- forbidigo
- forcetypeassert
- gocognit
- goconst
- gocritic
- gocyclo
- godot
- godox
- gofmt
- goheader
- gomoddirectives
- gomodguard
- goprintffuncname
- gosec
- gosimple
- govet
- grouper
- importas
- ineffassign
- interfacebloat
- logrlint
- maintidx
- makezero
- misspell
- nestif
- nilerr
- nilnil
- nlreturn
- nolintlint
- nosprintfhostport
- prealloc
- predeclared
- promlinter
- reassign
- staticcheck
- tenv
- thelper
- tparallel
- typecheck
- unconvert
- unparam
- unused
- usestdlibvars
- whitespace
234 changes: 234 additions & 0 deletions fxworker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
# Fx Worker Module

[![ci](https://github.com/ankorstore/yokai/actions/workflows/fxworker-ci.yml/badge.svg)](https://github.com/ankorstore/yokai/actions/workflows/fxworker-ci.yml)
[![go report](https://goreportcard.com/badge/github.com/ankorstore/yokai/fxworker)](https://goreportcard.com/report/github.com/ankorstore/yokai/fxworker)
[![codecov](https://codecov.io/gh/ankorstore/yokai/graph/badge.svg?token=ghUBlFsjhR&flag=fxworker)](https://app.codecov.io/gh/ankorstore/yokai/tree/main/fxworker)
[![Deps](https://img.shields.io/badge/osi-deps-blue)](https://deps.dev/go/github.com%2Fankorstore%2Fyokai%2Ffxworker)
[![PkgGoDev](https://pkg.go.dev/badge/github.com/ankorstore/yokai/fxworker)](https://pkg.go.dev/github.com/ankorstore/yokai/fxworker)

> [Fx](https://uber-go.github.io/fx/) module for [worker](https://github.com/ankorstore/yokai/tree/main/worker).
<!-- TOC -->
* [Installation](#installation)
* [Features](#features)
* [Documentation](#documentation)
* [Dependencies](#dependencies)
* [Loading](#loading)
* [Configuration](#configuration)
* [Registration](#registration)
* [Override](#override)
<!-- TOC -->

## Installation

```shell
go get github.com/ankorstore/yokai/fxworker
```

## Features

This module provides a workers pool to your Fx application with:

- automatic panic recovery
- automatic logging
- automatic metrics
- possibility to defer workers
- possibility to limit workers max execution attempts

## Documentation

### Dependencies

This module is intended to be used alongside:

- the [fxconfig](https://github.com/ankorstore/yokai/tree/main/fxconfig) module
- the [fxlog](https://github.com/ankorstore/yokai/tree/main/fxlog) module
- the [fxtrace](https://github.com/ankorstore/yokai/tree/main/fxtrace) module
- the [fxmetrics](https://github.com/ankorstore/yokai/tree/main/fxmetrics) module
- the [fxgenerate](https://github.com/ankorstore/yokai/tree/main/fxgenerate) module

### Loading

To load the module in your Fx application:

```go
package main

import (
"context"

"github.com/ankorstore/yokai/fxconfig"
"github.com/ankorstore/yokai/fxgenerate"
"github.com/ankorstore/yokai/fxlog"
"github.com/ankorstore/yokai/fxmetrics"
"github.com/ankorstore/yokai/fxtrace"
"github.com/ankorstore/yokai/fxworker"
"github.com/ankorstore/yokai/worker"
"go.uber.org/fx"
)

func main() {
fx.New(
fxconfig.FxConfigModule, // load the module dependencies
fxlog.FxLogModule,
fxtrace.FxTraceModule,
fxtrace.FxTraceModule,
fxmetrics.FxMetricsModule,
fxgenerate.FxGenerateModule,
fxworker.FxWorkerModule, // load the module
fx.Invoke(func(pool *worker.WorkerPool) {
pool.Start(context.Background()) // start the workers pool
}),
).Run()
}
```

### Configuration

Configuration reference:

```yaml
# ./configs/config.yaml
app:
name: app
env: dev
version: 0.1.0
debug: true
modules:
worker:
defer: 0.1 # threshold in seconds to wait before starting all workers, immediate start by default
attempts: 3 # max execution attempts in case of failures for all workers, no restart by default
metrics:
collect:
enabled: true # to collect metrics about workers executions
namespace: app # workers metrics namespace (default app.name value)
subsystem: worker # workers metrics subsystem (default worker)
```
Notes:
- the workers logging will be based on the [fxlog](https://github.com/ankorstore/yokai/tree/main/fxlog)
module configuration
- the workers tracing will be based on the [fxtrace](https://github.com/ankorstore/yokai/tree/main/fxtrace)
module configuration
### Registration
This module provides the possibility to register
several [Worker](https://github.com/ankorstore/yokai/blob/main/worker/worker.go) implementations, with
optional [WorkerExecutionOption](https://github.com/ankorstore/yokai/blob/main/worker/option.go).
They will be then collected and given by Fx to
the [WorkerPool](https://github.com/ankorstore/yokai/blob/main/worker/pool.go), made available in the Fx container.
This is done via the `AsWorker()` function:

```go
package main
import (
"context"
"github.com/ankorstore/yokai/fxconfig"
"github.com/ankorstore/yokai/fxgenerate"
"github.com/ankorstore/yokai/fxlog"
"github.com/ankorstore/yokai/fxmetrics"
"github.com/ankorstore/yokai/fxtrace"
"github.com/ankorstore/yokai/fxworker"
"github.com/ankorstore/yokai/worker"
"go.uber.org/fx"
)
type ExampleWorker struct{}
func NewExampleWorker() *ExampleWorker {
return &ExampleWorker{}
}
func (w *ExampleWorker) Name() string {
return "example-worker"
}
func (w *ExampleWorker) Run(ctx context.Context) error {
worker.CtxLogger(ctx).Info().Msg("run")
return nil
}
func main() {
fx.New(
fxconfig.FxConfigModule, // load the module dependencies
fxlog.FxLogModule,
fxtrace.FxTraceModule,
fxtrace.FxTraceModule,
fxmetrics.FxMetricsModule,
fxgenerate.FxGenerateModule,
fxworker.FxWorkerModule, // load the module
fx.Provide(
fxworker.AsWorker(
NewExampleWorker, // register the ExampleWorker
worker.WithDeferredStartThreshold(1), // with a deferred start threshold of 1 second
worker.WithMaxExecutionsAttempts(2), // and 2 max execution attempts
),
),
fx.Invoke(func(pool *worker.WorkerPool) {
pool.Start(context.Background()) // start the workers pool
}),
).Run()
}
```

To get more details about the features made available for your workers (contextual logging, tracing, etc.), check
the [worker module documentation](https://github.com/ankorstore/yokai/tree/main/worker).

### Override

By default, the `worker.WorkerPool` is created by
the [DefaultWorkerPoolFactory](https://github.com/ankorstore/yokai/blob/main/worker/factory.go).

If needed, you can provide your own factory and override the module:

```go
package main
import (
"context"
"github.com/ankorstore/yokai/fxconfig"
"github.com/ankorstore/yokai/fxgenerate"
"github.com/ankorstore/yokai/fxhealthcheck"
"github.com/ankorstore/yokai/fxlog"
"github.com/ankorstore/yokai/fxmetrics"
"github.com/ankorstore/yokai/fxtrace"
"github.com/ankorstore/yokai/fxworker"
"github.com/ankorstore/yokai/healthcheck"
"github.com/ankorstore/yokai/worker"
"go.uber.org/fx"
)
type CustomWorkerPoolFactory struct{}
func NewCustomWorkerPoolFactory() worker.WorkerPoolFactory {
return &CustomWorkerPoolFactory{}
}
func (f *CustomWorkerPoolFactory) Create(options ...worker.WorkerPoolOption) (*worker.WorkerPool, error) {
return &worker.WorkerPool{...}, nil
}
func main() {
fx.New(
fxconfig.FxConfigModule, // load the module dependencies
fxlog.FxLogModule,
fxtrace.FxTraceModule,
fxtrace.FxTraceModule,
fxmetrics.FxMetricsModule,
fxgenerate.FxGenerateModule,
fxworker.FxWorkerModule, // load the module
fx.Decorate(NewCustomWorkerPoolFactory), // override the module with a custom factory
fx.Invoke(func(pool *worker.WorkerPool) {
pool.Start(context.Background()) // start the custom worker pool
}),
).Run()
}
```
32 changes: 32 additions & 0 deletions fxworker/define.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package fxworker

import "github.com/ankorstore/yokai/worker"

// WorkerDefinition is the interface for workers definitions.
type WorkerDefinition interface {
ReturnType() string
Options() []worker.WorkerExecutionOption
}

type workerDefinition struct {
returnType string
options []worker.WorkerExecutionOption
}

// NewWorkerDefinition returns a new [WorkerDefinition].
func NewWorkerDefinition(returnType string, options ...worker.WorkerExecutionOption) WorkerDefinition {
return &workerDefinition{
returnType: returnType,
options: options,
}
}

// ReturnType returns the worker definition return type.
func (w *workerDefinition) ReturnType() string {
return w.returnType
}

// Options returns the worker definition execution options.
func (w *workerDefinition) Options() []worker.WorkerExecutionOption {
return w.options
}
Loading

0 comments on commit 71d2e5c

Please sign in to comment.