Skip to content

Commit

Permalink
Merge branch 'master' into verkle-support
Browse files Browse the repository at this point in the history
  • Loading branch information
pk910 committed Apr 16, 2024
2 parents 040a2db + 7c7c1a8 commit 95140b5
Show file tree
Hide file tree
Showing 39 changed files with 669 additions and 347 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/_shared-check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
uses: golangci/golangci-lint-action@v3
with:
# Optional: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version
version: latest
version: v1.56.1

# Optional: working directory, useful for monorepos
# working-directory: somedir
Expand Down
33 changes: 33 additions & 0 deletions .github/workflows/build-dev-latest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@

name: Build latest image

on:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: write

jobs:

check_source:
name: "Run code checks"
uses: ./.github/workflows/_shared-check.yaml

build_binaries:
name: "Build Assertoor"
needs: [check_source]
uses: ./.github/workflows/_shared-build.yaml
with:
ref: ${{ github.sha }}
release: "snapshot"
docker: true
docker_repository: "ethpandaops/assertoor"
docker_tag_prefix: "master"
additional_tags: "['master','master-latest','latest']"
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
3 changes: 1 addition & 2 deletions .github/workflows/build-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,8 @@ jobs:
- name: "Load PR info"
id: loadinfo
run: |
run_builds="false"
has_docker_image_label="${{ contains(github.event.pull_request.labels.*.name, 'build-docker-image') }}"
run_builds="$has_docker_image_label"
echo "docker image label: $has_docker_image_label"
branch_name="${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}"
Expand Down
21 changes: 15 additions & 6 deletions docs/02-global-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,23 @@ It contains general settings, endpoints, validator names, global variables, and
The configuration file is structured as follows:

```yaml
endpoints:
- name: "node-1"
executionUrl: "http://127.0.0.1:8545"
consensusUrl: "http://127.0.0.1:5052"
coordinator:
maxConcurrentTests: 1 # max number of tests to run concurrently
testRetentionTime: 336h # delete test run (logs + status) after that duration

web:
server:
host: "0.0.0.0"
port: 8080
api:
enabled: true # enable rest api
frontend:
enabled: true
enabled: true # enable web ui

endpoints:
- name: "node-1"
executionUrl: "http://127.0.0.1:8545"
consensusUrl: "http://127.0.0.1:5052"

validatorNames:
inventoryYaml: "./validator-names.yaml"
Expand Down Expand Up @@ -48,12 +54,14 @@ externalTests:

```

- **`coordinator`**:\
Manages the execution of tests, specifying the maximum number of tests that can run concurrently (`maxConcurrentTests`) and how long to retain test runs, including logs and status, after completion (`testRetentionTime`).

- **`endpoints`**:\
A list of Ethereum consensus and execution clients. Each endpoint includes URLs for both RPC endpoints and a name for reference in subsequent tests.

- **`web`**:\
Configurations for the web frontend, detailing server host and port settings.
Configurations for the web api & frontend, detailing server host and port settings.

- **`validatorNames`**:\
Defines a mapping of validator index ranges to their respective names. \
Expand All @@ -75,3 +83,4 @@ externalTests:
- **`externalTests`**:\
This feature enables the integration of tests that are defined in separate files, fostering a modular and scalable test configuration approach. \
It allows for better organization and management of complex testing scenarios.

18 changes: 18 additions & 0 deletions docs/03-rest-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# REST API

When enabled, Assertoor offers a REST API that facilitates interaction with the testing framework, enabling users to programmatically fetch statuses and results of tests and tasks. Additionally, the API supports basic operations such as scheduling or canceling test runs, making it a powerful tool for automating and integrating Assertoor into broader testing and CI/CD pipelines.

## Key Features of the REST API:

- **Status and Results Retrieval**: Users can query the API to obtain detailed status updates and results for both tests and individual tasks, allowing for real-time monitoring and analysis of test executions.

- **Test Management**: The API supports scheduling new test runs and canceling existing ones, providing flexibility in managing test execution according to dynamic testing requirements or conditions.

- **Integration Friendly**: The REST API's standard interface ensures it can be easily integrated with external tools and systems, enhancing Assertoor's utility in automated testing environments.

### Accessing the API Documentation:

The detailed API documentation, including all supported endpoints, request formats, and response structures, is accessible via the Assertoor web UI. This comprehensive documentation is designed to be user-friendly, offering examples and explanations to facilitate easy adoption and integration of the API into your workflows.

To access the API documentation, ensure the Assertoor web interface is enabled and navigate to the designated URL. The documentation provides interactive examples and the ability to test endpoints directly from your browser, offering a hands-on approach to learning and utilizing the REST API.

71 changes: 71 additions & 0 deletions docs/04-test-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Test Configuration

Assertoor allows you to set up tests to check various aspects of the Ethereum network. You can organize these tests in two ways: directly within your main configuration file or through external files for more complex scenarios. Here’s how it works:

## Local Tests

Local tests are defined in your main Assertoor configuration file. You can list tasks that you want to run as part of your test, along with any cleanup tasks to run afterward, regardless of whether your test passes or fails.

```yaml
- id: "test1"
name: "Test with local tasks"
timeout: "48h"
config: {}
tasks: []
cleanupTasks: []
schedule:
startup: true
cron:
- "* * * * *"
```
- **`id`**:\
A unique identifier for your test.
- **`name`**:\
The test's name.
- **`timeout`**:\
How long to run the test before stopping it. This is optional.
- **`config`**:\
A place to set static variables for your test. Optional.
- **`tasks`**:\
The tasks to run for your test.
- **`cleanupTasks`**:\
Tasks that clean up after your main tasks. These run no matter if the main tasks pass or fail. This is optional.
- **`schedule`**:\
Determines when your test runs. You can set it to start when Assertoor starts or on a schedule using cron format. This is optional.

## External Tests

External test playbooks, loaded via the `file` attribute in the Assertoor configuration, follow a structured format. These playbooks allow for defining comprehensive tests with specific tasks, variable configurations, and scheduling options.

**Example External Test Playbook:**

```yaml
id: test1
name: "Test 1"
timeout: 1h
config:
# walletPrivkey: ""
# validatorPairNames: []
configVars: {}
tasks: []
cleanupTasks: []
schedule:
startup: true
cron:
- "* * * * *"
```

**Key Properties Explained:**

- **`id`**: A unique identifier for the test, allowing for easy reference.
- **`name`**: The descriptive name of the test, providing clarity on its purpose.
- **`timeout`**: Specifies the duration after which the test should be considered failed if not completed.
- **`config`**: Static variable configuration, where you can define variables directly used by the test.
- **`configVars`**: Dynamic variable configuration that copies variables from the global scope, supporting complex expressions through jq syntax.
- **`tasks`**: The list of tasks to be executed as part of the test. Refer to the task configuration section for detailed task structures.
- **`cleanupTasks`**: Specifies tasks to be executed after the main tasks, regardless of their success or failure.
- **`schedule`**: Determines when the test should be run. If omitted, the test is scheduled to start upon Assertoor startup. It also supports cron expressions for more precise scheduling.

This format provides a flexible and powerful way to define tests outside the main configuration file, allowing for modular test management and reusability across different scenarios or environments.

File renamed without changes.
100 changes: 49 additions & 51 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
module github.com/ethpandaops/assertoor

go 1.21
go 1.21.1

require (
github.com/attestantio/go-eth2-client v0.19.10
github.com/attestantio/go-eth2-client v0.21.1
github.com/donovanhide/eventsource v0.0.0-20210830082556-c59027999da0
github.com/ethereum/go-ethereum v1.13.8
github.com/ethereum/go-ethereum v1.13.14
github.com/ethpandaops/ethwallclock v0.3.0
github.com/gorhill/cronexpr v0.0.0-20180427100037-88b0669f7d75
github.com/gorilla/mux v1.8.1
github.com/herumi/bls-eth-go-binary v1.31.0
github.com/herumi/bls-eth-go-binary v1.33.0
github.com/holiman/uint256 v1.2.4
github.com/itchyny/gojq v0.12.14
github.com/itchyny/gojq v0.12.15
github.com/juliangruber/go-intersect v1.1.0
github.com/mashingan/smapping v0.1.19
github.com/prometheus/client_golang v1.17.0
github.com/protolambda/zrnt v0.30.0
github.com/prometheus/client_golang v1.19.0
github.com/protolambda/zrnt v0.32.3
github.com/protolambda/ztyp v0.2.2
github.com/rs/zerolog v1.29.1
github.com/rs/zerolog v1.32.0
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.5.0
github.com/spf13/cobra v1.8.0
github.com/swaggo/http-swagger v1.3.4
github.com/swaggo/swag v1.16.3
github.com/tdewolff/minify v2.3.6+incompatible
Expand All @@ -33,72 +33,70 @@ require (
require (
github.com/KyleBanks/depth v1.2.1 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/StackExchange/wmi v1.2.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bits-and-blooms/bitset v1.10.0 // indirect
github.com/btcsuite/btcd/btcec/v2 v2.2.0 // indirect
github.com/bits-and-blooms/bitset v1.13.0 // indirect
github.com/btcsuite/btcd/btcec/v2 v2.3.2 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/consensys/bavard v0.1.13 // indirect
github.com/consensys/gnark-crypto v0.12.1 // indirect
github.com/crate-crypto/go-kzg-4844 v0.7.0 // indirect
github.com/deckarep/golang-set/v2 v2.1.0 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 // indirect
github.com/ethereum/c-kzg-4844 v0.4.0 // indirect
github.com/deckarep/golang-set/v2 v2.6.0 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0 // indirect
github.com/ethereum/c-kzg-4844 v1.0.1 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/ferranbt/fastssz v0.1.3 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/go-logr/logr v1.2.4 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.5 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.20.0 // indirect
github.com/go-openapi/spec v0.20.6 // indirect
github.com/go-openapi/swag v0.19.15 // indirect
github.com/goccy/go-yaml v1.9.5 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/huandu/go-clone v1.6.0 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/spec v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/goccy/go-yaml v1.11.3 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.1 // indirect
github.com/huandu/go-clone v1.7.2 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/itchyny/timefmt-go v0.1.5 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/kilic/bls12-381 v0.1.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.6 // indirect
github.com/mailru/easyjson v0.7.6 // indirect
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mmcloughlin/addchain v0.4.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/prometheus/client_model v0.4.1-0.20230718164431-9a2bf3000d16 // indirect
github.com/prometheus/common v0.44.0 // indirect
github.com/prometheus/procfs v0.11.1 // indirect
github.com/protolambda/bls12-381-util v0.0.0-20220416220906-d8552aa452c7 // indirect
github.com/prysmaticlabs/go-bitfield v0.0.0-20210809151128-385d8c5e3fb7 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.51.1 // indirect
github.com/prometheus/procfs v0.13.0 // indirect
github.com/protolambda/bls12-381-util v0.1.0 // indirect
github.com/prysmaticlabs/go-bitfield v0.0.0-20240328144219-a1caa50c3a1e // indirect
github.com/r3labs/sse/v2 v2.10.0 // indirect
github.com/shirou/gopsutil v3.21.4-0.20210419000835-c7a38de76ee5+incompatible // indirect
github.com/shirou/gopsutil v3.21.11+incompatible // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/supranational/blst v0.3.11 // indirect
github.com/swaggo/files v0.0.0-20220610200504-28940afbdbfe // indirect
github.com/swaggo/files v1.0.1 // indirect
github.com/tdewolff/parse v2.3.4+incompatible // indirect
github.com/tdewolff/test v1.0.10 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/tklauser/go-sysconf v0.3.13 // indirect
github.com/tklauser/numcpus v0.7.0 // indirect
github.com/wealdtech/go-bytesutil v1.2.1 // indirect
go.opentelemetry.io/otel v1.16.0 // indirect
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/otel/trace v1.16.0 // indirect
golang.org/x/crypto v0.18.0 // indirect
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/net v0.18.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/sys v0.16.0 // indirect
golang.org/x/tools v0.15.0 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.opentelemetry.io/otel v1.24.0 // indirect
go.opentelemetry.io/otel/metric v1.24.0 // indirect
go.opentelemetry.io/otel/trace v1.24.0 // indirect
golang.org/x/crypto v0.21.0 // indirect
golang.org/x/exp v0.0.0-20240325151524-a685a6edb6d8 // indirect
golang.org/x/mod v0.16.0 // indirect
golang.org/x/net v0.22.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/tools v0.19.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
google.golang.org/protobuf v1.31.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/cenkalti/backoff.v1 v1.1.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
rsc.io/tmplfunc v0.0.3 // indirect
Expand Down
Loading

0 comments on commit 95140b5

Please sign in to comment.