"ToDo API" Microservice Example

Introduction

Welcome! 👋

This is an educational repository that includes a microservice written in Go. It is used as the principal example of my video series: Building Microservices in Go.

This repository is not a template nor a framework, it's a collection of patterns and guidelines I've successfully used to deliver enterprise microservices when using Go, and just like with everything in Software Development some trade-offs were made.

My end goal with this project is to help you learn another way to structure your Go project with 3 final goals:

1. It is enterprise, meant to last for years,
2. It allows a team to collaborate efficiently with little friction, and
3. It is as idiomatic as possible.

Join the fun at https://youtube.com/MarioCarrion.

Domain Driven Design

This project uses a lot of the ideas introduced by Eric Evans in his book Domain Driven Design, I do encourage reading that book but before I think reading Domain-Driven Design Distilled makes more sense, also there's a free to download DDD Reference available as well.

On YouTube I created a playlist that includes some of my favorite talks and webinars, feel free to explore that as well.

Project Structure

Talking specifically about microservices only, the structure I like to recommend is the following, everything using < and > depends on the domain being implemented and the bounded context being defined.

• build/: defines the code used for creating infrastructure as well as docker containers.
  • <cloud-providers>/: define concrete cloud provider.
  • <executableN>/: contains a Dockerfile used for building the binary.
• cmd/
  • <primary-server>/: uses primary database.
  • <replica-server>/: uses readonly databases.
  • <binaryN>/
• db/
  • migrations/: contains database migrations.
  • seeds/: contains file meant to populate basic database values.
• internal/: defines the core domain.
  • <datastoreN>/: a concrete repository used by the domain, for example postgresql
  • http/: defines HTTP Handlers.
  • service/: orchestrates use cases and manages transactions.
• pkg/ public API meant to be imported by other Go package.

There are cases where requiring a new bounded context is needed, in those cases the recommendation would be to define a package like internal/<bounded-context> that then should follow the same structure, for example:

• internal/<bounded-context>/
  • internal/<bounded-context>/<datastoreN>
  • internal/<bounded-context>/http
  • internal/<bounded-context>/service

Tools

Please refer to the documentation in internal/tools/.

Features

Icons meaning:

• means a link to Youtube video.
• means a link to Blog post.

In no particular order:

• Project Layout
• Dependency Injection
• Secure Configuration
  • Using Hashicorp Vault
  • Using AWS SSM
• Infrastructure as code
• Metrics, Traces and Logging using OpenTelemetry
• Caching
  • Memcached
  • Redis
• Persistent Storage
  • Repository Pattern
  • Database migrations
  • MySQL
  • PostgreSQL
    • jmoiron/sqlx, jackc/pgx and database/sql
    • go-gorm/gorm and volatiletech/sqlboiler
    • Masterminds/squirrel and kyleconroy/sqlc
• REST APIs
  • HTTP Handlers
  • Custom JSON Types
  • Versioning
  • Error Handling
  • OpenAPI 3 and Swagger-UI
  • Authorization
• Events and Messaging
  • Apache Kafka
  • RabbitMQ
  • Redis
• Testing
  • Type-safe mocks with maxbrunsfeld/counterfeiter
  • Equality with google/go-cmp
  • Integration tests for Datastores with ory/dockertest
  • REST APIs
• Containerization using Docker
• Graceful Shutdown
• Search Engine using ElasticSearch
• Documentation
  • C4 Model
• Cloud Design Patterns
  • Reliability
    • Circuit Breaker
• Tools as Dependencies
• Whatever else I forgot to include

More ideas

• 2016: Peter Bourgon's: Repository structure
• 2016: Ben Johnson's: Standard Package Layout
• 2017: William Kennedy's: Design Philosophy On Packaging
• 2017: Jaana Dogan's: Style guideline for Go packages
• 2018: Kat Zien - How Do You Structure Your Go Apps

Running project locally using Docker Compose

Originally added as part of Building Microservices In Go: Containerization with Docker, docker compose has evolved and with it the way to run everything locally. Make sure you are running a recent version of Docker Compose. The configuration in this repository and the instructions below are known to work for at least the following versions:

• Engine: 27.4.0, and
• Compose: v2.31.0-desktop.2

This project takes advantage of Go's build constrains and Docker's arguments to build the ElasticSearch indexers and to run the rest-server using any of the following types of message broker:

• Redis (default one)
• RabbitMQ
• Kafka

The docker compose instructions are executed in the form of:

docker compose -f compose.yml -f compose.<type>.yml <command>

Where:

• <type>: Indicates what message broker to use, and effectively match the compose filename itself. The three supported values are:
  1. rabbitmq,
  2. kafka, and
  3. redis (default value when building the rest-server binary).
• <command>: Indicates the docker compose command to use.

For example to build images using RabbitMQ as the message broker you execute:

docker compose -f compose.yml -f compose.rabbitmq.yml build

Then to start the containers you execute:

docker compose -f compose.yml -f compose.rabbitmq.yml up

Once you all the containers are up you can access the Swagger UI at http://127.0.0.1:9234/static/swagger-ui/ .

Diagrams

To start a local HTTP server that serves a graphical editor:

mdl serve github.com/MarioCarrion/todo-api/internal/doc -dir docs/diagrams/

To generate JSON artifact for uploading to structurizr:

stz gen github.com/MarioCarrion/todo-api/internal/doc