Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Miner] feat: add events query client #64

Merged
merged 64 commits into from
Oct 27, 2023
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
3f2971b
feat: add the map channel observable operator
bryanchriswhite Oct 20, 2023
2baddb3
Merge remote-tracking branch 'pokt/main' into feat/observable-map
bryanchriswhite Oct 23, 2023
4af6643
feat: add replay observable
bryanchriswhite Oct 23, 2023
765b0c9
Merge branch 'feat/observable-map' into merge/map_x_replay
bryanchriswhite Oct 23, 2023
5f9ce1b
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 23, 2023
9c42698
chore: add query client interface
bryanchriswhite Oct 14, 2023
6273f52
chore: add query client errors
bryanchriswhite Oct 19, 2023
30a0a28
test: fix false positive, prevent regression, & add comments
bryanchriswhite Oct 23, 2023
6a67cb2
chore: add godoc comment
bryanchriswhite Oct 23, 2023
ad0121d
feat: add query client implementation
bryanchriswhite Oct 14, 2023
ee42737
chore: add connection & dialer wrapper implementations
bryanchriswhite Oct 17, 2023
66fdc79
test: query client & add testquery helper pkg
bryanchriswhite Oct 14, 2023
891faf9
chore: add go_test_integration make target
bryanchriswhite Oct 14, 2023
c12afe6
chore: add internal mocks pkg
bryanchriswhite Oct 23, 2023
de4defe
test: query client integration test
bryanchriswhite Oct 23, 2023
0453b62
docs: add event query client docs
bryanchriswhite Oct 20, 2023
31099ec
chore: update go.mod
bryanchriswhite Oct 14, 2023
a28ad44
chore: re-order `eventsQueryClient` methods to improve readability
bryanchriswhite Oct 24, 2023
bab1465
chore: add godoc comments to testclient helpers
bryanchriswhite Oct 24, 2023
09d16b4
fix: comment formatting
bryanchriswhite Oct 24, 2023
1c2e38e
chore: improve comment & naming in evt query client test
bryanchriswhite Oct 24, 2023
b670aec
test: tune events query client parameters
bryanchriswhite Oct 24, 2023
01278b0
chore: improve godoc comments
bryanchriswhite Oct 24, 2023
f962995
chore: review improvements
bryanchriswhite Oct 24, 2023
72f2916
Merge remote-tracking branch 'pokt/main' into feat/observable-map
bryanchriswhite Oct 24, 2023
163bb45
refactor: `replayObservable` as its own interface type
bryanchriswhite Oct 24, 2023
82e361e
refactor: `replayObservable#Next() V` to `ReplayObservable#Last(ctx,…
bryanchriswhite Oct 24, 2023
299ffb1
chore: add constructor func for `ReplayObservable`
bryanchriswhite Oct 24, 2023
a52603f
test: reorder to improve readibility
bryanchriswhite Oct 24, 2023
65c9e6e
refactor: rename and add godoc comments
bryanchriswhite Oct 24, 2023
de9c0ee
Merge remote-tracking branch 'pokt/main' into feat/replay-observable
bryanchriswhite Oct 24, 2023
cb79b0a
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 24, 2023
52c26e0
Merge branch 'feat/observable-map' into merge/map_x_replay
bryanchriswhite Oct 24, 2023
dfecf80
Merge branch 'merge/map_x_replay' into feat/query-client
bryanchriswhite Oct 24, 2023
507c79a
chore: improve naming & comments
bryanchriswhite Oct 24, 2023
bebe700
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 24, 2023
8782c11
Merge branch 'merge/map_x_replay' into feat/query-client
bryanchriswhite Oct 24, 2023
31c0ceb
chore: add warning log and improve comments
bryanchriswhite Oct 25, 2023
f7a8df3
test: improve and add tests
bryanchriswhite Oct 25, 2023
2225e97
fix: interface assertion
bryanchriswhite Oct 25, 2023
84e21b7
fix: comment typo
bryanchriswhite Oct 25, 2023
7df6220
chore: review improvements
bryanchriswhite Oct 25, 2023
6f1cfcb
Merge remote-tracking branch 'pokt/main' into feat/replay-observable
bryanchriswhite Oct 25, 2023
00e0918
fix: race
bryanchriswhite Oct 25, 2023
9027c1e
fix: race on eventsBytesAndConns map
bryanchriswhite Oct 25, 2023
a19fcc0
fix: interface assertions
bryanchriswhite Oct 25, 2023
71344b3
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 25, 2023
8978fa8
fix: race
bryanchriswhite Oct 25, 2023
9b90930
Merge branch 'merge/map_x_replay' into feat/query-client
bryanchriswhite Oct 25, 2023
a6d77b9
Merge remote-tracking branch 'pokt/feat/query-client' into feat/query…
bryanchriswhite Oct 25, 2023
4419602
Small updates to the README
Olshansk Oct 25, 2023
c093488
chore: review improvements
bryanchriswhite Oct 26, 2023
874d424
refactor: eliminate `EventsQueryClient#requestId` field
bryanchriswhite Oct 26, 2023
73dedf1
refactor: eliminate `EventsQueryClient#requestId` field
bryanchriswhite Oct 26, 2023
fab303f
refactor: move websocket dialer and connection to own pkg
bryanchriswhite Oct 26, 2023
fe681f9
chore: add comment
bryanchriswhite Oct 26, 2023
4830c06
chore: move `EventsBytesObservable type above interfaces
bryanchriswhite Oct 26, 2023
4762fa3
chore: review improvements
bryanchriswhite Oct 26, 2023
27ed494
fix: bug & improve naming & comments
bryanchriswhite Oct 26, 2023
676ff91
chore: review improvements
bryanchriswhite Oct 27, 2023
d7490ac
chore: review improvements
bryanchriswhite Oct 27, 2023
a094a9f
chore: add comment
bryanchriswhite Oct 27, 2023
9069667
Merge branch 'main' into feat/query-client
bryanchriswhite Oct 27, 2023
74ec7e8
revert: replay observable, merged into previous base branch
bryanchriswhite Oct 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -114,13 +114,18 @@ test_e2e: ## Run all E2E tests

.PHONY: go_test
go_test: go_version_check ## Run all go tests
go test -v ./...
go test -v -race -tags test ./...

.PHONY: go_test_integration
go_test_integration: go_version_check ## Run all go tests, including integration
go test -v -race -tags test,integration ./...

.PHONY: go_mockgen
go_mockgen: ## Use `mockgen` to generate mocks used for testing purposes of all the modules.
go generate ./x/application/types/
go generate ./x/gateway/types/
go generate ./x/supplier/types/
go generate ./pkg/...

.PHONY: go_develop
go_develop: proto_regen go_mockgen ## Generate protos and mocks
Expand Down
104 changes: 104 additions & 0 deletions docs/pkg/client/events_query.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Package `pocket/pkg/client/events_query` <!-- omit in toc -->

> An event query package for interfacing with [CometBFT](https://cometbft.com/) and the [Cosmos SDK](https://v1.cosmos.network/sdk), facilitating subscriptions to chain event messages.

- [Overview](#overview)
- [Architecture Diagrams](#architecture-diagrams)
- [Installation](#installation)
- [Features](#features)
- [Usage](#usage)
- [Basic Example](#basic-example)
- [Advanced Usage](#advanced-usage)
- [Configuration](#configuration)
- [API Reference](#api-reference)
- [Best Practices](#best-practices)
- [FAQ](#faq)
- [Why use `events_query` over directly using Gorilla WebSockets?](#why-use-events_query-over-directly-using-gorilla-websockets)
- [How can I use a different connection mechanism other than WebSockets?](#how-can-i-use-a-different-connection-mechanism-other-than-websockets)
- [Contributing](#contributing)
- [Changelog](#changelog)
- [License](#license)

## Overview

The `events_query` package provides a client interface to subscribe to chain event messages. It abstracts the underlying connection mechanisms and offers a clear and easy-to-use way to get events from the chain. Highlights:

- Offers subscription to chain event messages matching a given query.
- Uses the Gorilla WebSockets package for underlying connection operations.
- Provides a modular structure with interfaces allowing for mock implementations and testing.
- Offers considerations for potential improvements and replacements, such as integration with the cometbft RPC client.

## Architecture Diagrams

_TODO(@bryanchriswhite): Add architecture diagrams for the package._
bryanchriswhite marked this conversation as resolved.
Show resolved Hide resolved

## Installation

```bash
go get github.com/pokt-network/poktroll/pkg/client/events_query
```

## Features

- **Websocket Connection**: Uses the [Gorilla WebSockets](https://github.com/gorilla/websocket) for implementing the connection interface.
- **Events Subscription**: Subscribe to chain event messages using a simple query mechanism.
- **Dialer Interface**: Offers a `Dialer` interface for constructing connections, which can be easily mocked for tests.
- **Observable Pattern**: Integrates the observable pattern, making it easier to react to chain events.

## Usage

### Basic Example

```go
// Creating a new EventsQueryClient with the default websocket dialer:
cometWebsocketURL := "ws://example.com"
evtClient := eventsquery.NewEventsQueryClient(cometWebsocketURL)

// Subscribing to a specific event:
observable, errCh := evtClient.EventsBytes(context.Background(), "your-query-string")
```

### Advanced Usage

_TODO(@bryanchriswhite): Add examples of advanced usage_

### Configuration

- **WithDialer**: Configure the client to use a custom dialer for connections.

## API Reference
bryanchriswhite marked this conversation as resolved.
Show resolved Hide resolved

- `EventsQueryClient`: Main interface to query events. Methods include:
- `EventsBytes(ctx, query)`: Returns an observable for chain events.
- `Close()`: Close any existing connections and unsubscribe all observers.
- `Connection`: Interface representing a bidirectional message-passing connection.
- `Dialer`: Interface encapsulating the creation of connections.

For the complete API details, see the [godoc](https://pkg.go.dev/github.com/pokt-network/poktroll/pkg/client/events_query).

## Best Practices

- **Connection Handling**: Ensure to close the `EventsQueryClient` when done to free up resources and avoid potential leaks.
- **Error Handling**: Always check the error channel returned by `EventsBytes` for asynchronous errors during operation.

## FAQ

#### Why use `events_query` over directly using Gorilla WebSockets?

`events_query` abstracts many of the underlying details and provides a streamlined interface for subscribing to chain events. It also integrates the observable pattern and provides mockable interfaces for better testing.

#### How can I use a different connection mechanism other than WebSockets?

You can implement the `Dialer` and `Connection` interfaces and use the `WithDialer` configuration to provide your custom dialer.

## Contributing

If you're interested in improving the `events_query` package or adding new features, please start by discussing your ideas in the project's issues section. Check our main contributing guide for more details.

## Changelog
bryanchriswhite marked this conversation as resolved.
Show resolved Hide resolved

For detailed release notes, see the [CHANGELOG](../CHANGELOG.md).

## License

This package is released under the XYZ License. For more information, see the [LICENSE](../LICENSE) file at the root level.
8 changes: 4 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,22 @@ require (
cosmossdk.io/math v1.0.1
github.com/cometbft/cometbft v0.37.2
github.com/cometbft/cometbft-db v0.8.0
github.com/cosmos/cosmos-proto v1.0.0-beta.2
github.com/cosmos/cosmos-sdk v0.47.3
github.com/cosmos/gogoproto v1.4.10
github.com/cosmos/ibc-go/v7 v7.1.0
github.com/golang/mock v1.6.0
github.com/golang/protobuf v1.5.3
github.com/gorilla/mux v1.8.0
github.com/gorilla/websocket v1.5.0
github.com/grpc-ecosystem/grpc-gateway v1.16.0
github.com/grpc-ecosystem/grpc-gateway/v2 v2.15.2
github.com/regen-network/gocuke v0.6.2
github.com/spf13/cast v1.5.1
github.com/spf13/cobra v1.7.0
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.8.4
go.uber.org/multierr v1.11.0
golang.org/x/sync v0.3.0
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1
google.golang.org/grpc v1.56.1
gopkg.in/yaml.v2 v2.4.0
)
Expand Down Expand Up @@ -68,6 +68,7 @@ require (
github.com/containerd/cgroups v1.1.0 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/cosmos/btcutil v1.0.5 // indirect
github.com/cosmos/cosmos-proto v1.0.0-beta.2 // indirect
github.com/cosmos/go-bip39 v1.0.0 // indirect
github.com/cosmos/gogogateway v1.2.0 // indirect
github.com/cosmos/iavl v0.20.0 // indirect
Expand Down Expand Up @@ -122,7 +123,6 @@ require (
github.com/googleapis/gax-go/v2 v2.8.0 // indirect
github.com/gorilla/handlers v1.5.1 // indirect
github.com/gorilla/rpc v1.2.0 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c // indirect
github.com/gtank/merlin v0.1.1 // indirect
Expand Down Expand Up @@ -251,7 +251,6 @@ require (
go.uber.org/atomic v1.10.0 // indirect
go.uber.org/dig v1.16.1 // indirect
go.uber.org/fx v1.19.2 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.24.0 // indirect
golang.org/x/crypto v0.12.0 // indirect
golang.org/x/exp v0.0.0-20230626212559-97b1e661b5df // indirect
Expand All @@ -266,6 +265,7 @@ require (
gonum.org/v1/gonum v0.11.0 // indirect
google.golang.org/api v0.122.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Empty file added internal/mocks/.gitkeep
Empty file.
3 changes: 3 additions & 0 deletions internal/testclient/common.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
package testclient
Olshansk marked this conversation as resolved.
Show resolved Hide resolved

const CometLocalWebsocketURL = "ws://localhost:36657/websocket"
17 changes: 17 additions & 0 deletions internal/testclient/testeventsquery/client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package testeventsquery

import (
"testing"

"pocket/internal/testclient"
"pocket/pkg/client"
eventsquery "pocket/pkg/client/events_query"
)

// NewLocalnetClient returns a new events query client which is configured to
// connect to the localnet sequencer.
func NewLocalnetClient(t *testing.T, opts ...client.EventsQueryClientOption) client.EventsQueryClient {
t.Helper()

return eventsquery.NewEventsQueryClient(testclient.CometLocalWebsocketURL, opts...)
}
49 changes: 49 additions & 0 deletions internal/testclient/testeventsquery/connection.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
package testeventsquery

import (
"pocket/pkg/either"
"testing"

"github.com/golang/mock/gomock"

"pocket/internal/mocks/mockclient"
)

// NewOneTimeMockConnAndDialer returns a new mock connection and mock dialer that
// will return the mock connection when DialContext is called. The mock dialer
// will expect DialContext to be called exactly once. The connection mock will
// expect Close to be called exactly once.
// Callers must mock the Receive method with an EXPECT call before the connection
bryanchriswhite marked this conversation as resolved.
Show resolved Hide resolved
// mock can be used.
func NewOneTimeMockConnAndDialer(t *testing.T) (
*mockclient.MockConnection,
*mockclient.MockDialer,
) {
ctrl := gomock.NewController(t)
connMock := mockclient.NewMockConnection(ctrl)
connMock.EXPECT().Close().
Return(nil).
Times(1)

dialerMock := NewOneTimeMockDialer(t, either.Success(connMock))

return connMock, dialerMock
}

// NewOneTimeMockDialer returns a mock dialer that will return either the given
// connection mock or error when DialContext is called. The mock dialer will
// expect DialContext to be called exactly once.
func NewOneTimeMockDialer(
t *testing.T,
eitherConnMock either.Either[*mockclient.MockConnection],
) *mockclient.MockDialer {
ctrl := gomock.NewController(t)
dialerMock := mockclient.NewMockDialer(ctrl)

connMock, err := eitherConnMock.ValueOrError()
dialerMock.EXPECT().DialContext(gomock.Any(), gomock.Any()).
Return(connMock, err).
Times(1)

return dialerMock
}
Loading