Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Miner] feat: add block client #65

Merged
merged 97 commits into from
Nov 1, 2023
Merged
Changes from 1 commit
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
3f2971b
feat: add the map channel observable operator
bryanchriswhite Oct 20, 2023
2baddb3
Merge remote-tracking branch 'pokt/main' into feat/observable-map
bryanchriswhite Oct 23, 2023
4af6643
feat: add replay observable
bryanchriswhite Oct 23, 2023
765b0c9
Merge branch 'feat/observable-map' into merge/map_x_replay
bryanchriswhite Oct 23, 2023
5f9ce1b
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 23, 2023
9c42698
chore: add query client interface
bryanchriswhite Oct 14, 2023
6273f52
chore: add query client errors
bryanchriswhite Oct 19, 2023
30a0a28
test: fix false positive, prevent regression, & add comments
bryanchriswhite Oct 23, 2023
6a67cb2
chore: add godoc comment
bryanchriswhite Oct 23, 2023
ad0121d
feat: add query client implementation
bryanchriswhite Oct 14, 2023
ee42737
chore: add connection & dialer wrapper implementations
bryanchriswhite Oct 17, 2023
66fdc79
test: query client & add testquery helper pkg
bryanchriswhite Oct 14, 2023
891faf9
chore: add go_test_integration make target
bryanchriswhite Oct 14, 2023
c12afe6
chore: add internal mocks pkg
bryanchriswhite Oct 23, 2023
de4defe
test: query client integration test
bryanchriswhite Oct 23, 2023
0453b62
docs: add event query client docs
bryanchriswhite Oct 20, 2023
31099ec
chore: update go.mod
bryanchriswhite Oct 14, 2023
a28ad44
chore: re-order `eventsQueryClient` methods to improve readability
bryanchriswhite Oct 24, 2023
bab1465
chore: add godoc comments to testclient helpers
bryanchriswhite Oct 24, 2023
09d16b4
fix: comment formatting
bryanchriswhite Oct 24, 2023
1c2e38e
chore: improve comment & naming in evt query client test
bryanchriswhite Oct 24, 2023
b670aec
test: tune events query client parameters
bryanchriswhite Oct 24, 2023
01278b0
chore: improve godoc comments
bryanchriswhite Oct 24, 2023
f962995
chore: review improvements
bryanchriswhite Oct 24, 2023
72f2916
Merge remote-tracking branch 'pokt/main' into feat/observable-map
bryanchriswhite Oct 24, 2023
163bb45
refactor: `replayObservable` as its own interface type
bryanchriswhite Oct 24, 2023
82e361e
refactor: `replayObservable#Next() V` to `ReplayObservable#Last(ctx,…
bryanchriswhite Oct 24, 2023
299ffb1
chore: add constructor func for `ReplayObservable`
bryanchriswhite Oct 24, 2023
a52603f
test: reorder to improve readibility
bryanchriswhite Oct 24, 2023
65c9e6e
refactor: rename and add godoc comments
bryanchriswhite Oct 24, 2023
de9c0ee
Merge remote-tracking branch 'pokt/main' into feat/replay-observable
bryanchriswhite Oct 24, 2023
cb79b0a
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 24, 2023
52c26e0
Merge branch 'feat/observable-map' into merge/map_x_replay
bryanchriswhite Oct 24, 2023
dfecf80
Merge branch 'merge/map_x_replay' into feat/query-client
bryanchriswhite Oct 24, 2023
507c79a
chore: improve naming & comments
bryanchriswhite Oct 24, 2023
bebe700
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 24, 2023
8782c11
Merge branch 'merge/map_x_replay' into feat/query-client
bryanchriswhite Oct 24, 2023
31c0ceb
chore: add warning log and improve comments
bryanchriswhite Oct 25, 2023
f7a8df3
test: improve and add tests
bryanchriswhite Oct 25, 2023
2225e97
fix: interface assertion
bryanchriswhite Oct 25, 2023
84e21b7
fix: comment typo
bryanchriswhite Oct 25, 2023
7df6220
chore: review improvements
bryanchriswhite Oct 25, 2023
6f1cfcb
Merge remote-tracking branch 'pokt/main' into feat/replay-observable
bryanchriswhite Oct 25, 2023
00e0918
fix: race
bryanchriswhite Oct 25, 2023
a963e24
chore: add block client interface
bryanchriswhite Oct 14, 2023
74ebb62
chore: add `MapReplay` operator
bryanchriswhite Oct 24, 2023
44b84ae
feat: add block client
bryanchriswhite Oct 14, 2023
b817851
test: block client integration
bryanchriswhite Oct 14, 2023
9955dbb
test: block client
bryanchriswhite Oct 23, 2023
acc69ef
docs: fix install instructions
bryanchriswhite Oct 25, 2023
9027c1e
fix: race on eventsBytesAndConns map
bryanchriswhite Oct 25, 2023
a19fcc0
fix: interface assertions
bryanchriswhite Oct 25, 2023
92a9db6
Merge branch 'feat/replay-observable' into feat/block-client
bryanchriswhite Oct 25, 2023
71344b3
Merge branch 'feat/replay-observable' into merge/map_x_replay
bryanchriswhite Oct 25, 2023
8978fa8
fix: race
bryanchriswhite Oct 25, 2023
9b90930
Merge branch 'merge/map_x_replay' into feat/query-client
bryanchriswhite Oct 25, 2023
a6d77b9
Merge remote-tracking branch 'pokt/feat/query-client' into feat/query…
bryanchriswhite Oct 25, 2023
a123a13
Merge branch 'feat/query-client' into feat/block-client
bryanchriswhite Oct 25, 2023
4419602
Small updates to the README
Olshansk Oct 25, 2023
6d06bf7
refactor: add observableInternals interface
bryanchriswhite Oct 26, 2023
deed860
chore: update last; only block for 1 value min
bryanchriswhite Oct 26, 2023
7f81ce0
chore: review improvements
bryanchriswhite Oct 26, 2023
fc6f161
Merge branch 'main' into feat/replay-observable
bryanchriswhite Oct 26, 2023
729bcd1
Merge branch 'feat/replay-observable' into feat/block-client
bryanchriswhite Oct 26, 2023
4b2d922
refactor: move add `channelObservableInternals` & migrate its relevan…
bryanchriswhite Oct 26, 2023
614bc01
refactor: simplify, cleanup, & improve comments
bryanchriswhite Oct 26, 2023
c093488
chore: review improvements
bryanchriswhite Oct 26, 2023
874d424
refactor: eliminate `EventsQueryClient#requestId` field
bryanchriswhite Oct 26, 2023
0737a4a
Merge branch 'feat/query-client' into feat/block-client
bryanchriswhite Oct 26, 2023
777e340
refactor: review improvements
bryanchriswhite Oct 26, 2023
73dedf1
refactor: eliminate `EventsQueryClient#requestId` field
bryanchriswhite Oct 26, 2023
fab303f
refactor: move websocket dialer and connection to own pkg
bryanchriswhite Oct 26, 2023
fe681f9
chore: add comment
bryanchriswhite Oct 26, 2023
b0181dd
fix: notify `retryOnError()` of async error propagating through `#Eve…
bryanchriswhite Oct 26, 2023
ec3d475
chore: review improvements
bryanchriswhite Oct 26, 2023
4830c06
chore: move `EventsBytesObservable type above interfaces
bryanchriswhite Oct 26, 2023
4762fa3
chore: review improvements
bryanchriswhite Oct 26, 2023
27ed494
fix: bug & improve naming & comments
bryanchriswhite Oct 26, 2023
1a4169d
Merge branch 'feat/query-client' into feat/block-client
bryanchriswhite Oct 26, 2023
f08fb04
chore: review improvements
bryanchriswhite Oct 27, 2023
f835b13
Merge remote-tracking branch 'pokt/main' into feat/replay-observable
bryanchriswhite Oct 27, 2023
f8ba998
Merge branch 'feat/replay-observable' into feat/block-client
bryanchriswhite Oct 27, 2023
00a025b
fix: bug in `accumulateReplayValues()`
bryanchriswhite Oct 27, 2023
3d0dd0a
Merge branch 'feat/replay-observable' into feat/block-client
bryanchriswhite Oct 27, 2023
b407e16
Merge remote-tracking branch 'pokt/main' into feat/block-client
bryanchriswhite Oct 27, 2023
e04177f
Merge remote-tracking branch 'pokt/main' into feat/replay-observable
bryanchriswhite Oct 27, 2023
27b805e
Merge branch 'feat/replay-observable' into feat/block-client
bryanchriswhite Oct 27, 2023
6c3c2b9
refactor: promote `retryOnError` to its own pkg: `retry.OnError`
bryanchriswhite Oct 30, 2023
6b005de
Merge remote-tracking branch 'pokt/main' into feat/block-client
bryanchriswhite Oct 30, 2023
b226a10
chore: improve comments
bryanchriswhite Oct 30, 2023
7ade4a0
test: inline wip test helpers
bryanchriswhite Oct 30, 2023
7cfe788
test: skip retry.OnError tests & comment
bryanchriswhite Oct 30, 2023
486dba0
chore: review feedback improvements
bryanchriswhite Oct 31, 2023
1677e3f
chore: review feedback improvements
bryanchriswhite Oct 31, 2023
cc466b2
Merge branch 'main' into feat/block-client
bryanchriswhite Oct 31, 2023
2f49089
Merge branch 'main' into feat/block-client
bryanchriswhite Nov 1, 2023
7ae2650
fix: format placeholder error
bryanchriswhite Nov 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge remote-tracking branch 'pokt/main' into feat/block-client
* pokt/main:
  [Miner] feat: add events query client (#64)
  • Loading branch information
bryanchriswhite committed Oct 27, 2023
commit b407e169eee618961b5ea8ef18a72e7b2587c741
164 changes: 132 additions & 32 deletions docs/pkg/client/events_query.md
Original file line number Diff line number Diff line change
@@ -10,14 +10,10 @@
- [Basic Example](#basic-example)
- [Advanced Usage](#advanced-usage)
- [Configuration](#configuration)
- [API Reference](#api-reference)
- [Best Practices](#best-practices)
- [FAQ](#faq)
- [Why use `events_query` over directly using Gorilla WebSockets?](#why-use-events_query-over-directly-using-gorilla-websockets)
- [How can I use a different connection mechanism other than WebSockets?](#how-can-i-use-a-different-connection-mechanism-other-than-websockets)
- [Contributing](#contributing)
- [Changelog](#changelog)
- [License](#license)

## Overview

@@ -30,7 +26,98 @@ The `events_query` package provides a client interface to subscribe to chain eve

## Architecture Diagrams

_TODO(@bryanchriswhite): Add architecture diagrams for the package._
### Components
```mermaid
---
title: Component Diagram Legend
---

flowchart

a[Component A]
b[Component B]
c[Component C]
d[Component D]

a --"A uses B via B#MethodName()"--> b
a =="A returns C from A#MethodName()"==> c
b -."A uses D via network IO".-> d
```
```mermaid
---
title: EventsQueryClient Components
---

flowchart

subgraph comet[Cometbft Node]
subgraph rpc[JSON-RPC]
sub[subscribe endpoint]
end
end

subgraph eqc[EventsQueryClient]
q1_eb[EventsBytesObservable]
q1_conn[Connection]
q1_dial[Dialer]
end

q1_obsvr1[Observer 1]
q1_obsvr2[Observer 2]


q1_obsvr1 --"#Subscribe()"--> q1_eb
q1_obsvr2 --"#Subscribe()"--> q1_eb


q1_dial =="#DialContext()"==> q1_conn
q1_eb --"#Receive()"--> q1_conn

q1_conn -.-> sub

```

### Subscriptions
```mermaid
---
title: Event Subscription Data Flow
---

flowchart

subgraph comet[Cometbft Node]
subgraph rpc[JSON-RPC]
sub[subscribe endpoint]
end
end

subgraph eqc[EventsQueryClient]
subgraph q1[Query 1]
q1_eb[EventsBytesObservable]
q1_conn[Connection]
end
subgraph q2[Query 2]
q2_conn[Connection]
q2_eb[EventsBytesObservable]
end
end

q1_obsvr1[Query 1 Observer 1]
q1_obsvr2[Query 1 Observer 2]
q2_obsvr[Query 2 Observer]

q1_eb -.-> q1_obsvr1
q1_eb -.-> q1_obsvr2
q2_eb -.-> q2_obsvr


q1_conn -.-> q1_eb
q2_conn -.-> q2_eb

sub -.-> q1_conn
sub -.-> q2_conn

```

## Installation

@@ -50,55 +137,68 @@ go get github.com/pokt-network/poktroll/pkg/client/events_query
### Basic Example

```go
// Creating a new EventsQueryClient with the default websocket dialer:
ctx := context.Background()

// Creating a new EventsQueryClient with the default, websocket dialer:
cometWebsocketURL := "ws://example.com"
evtClient := eventsquery.NewEventsQueryClient(cometWebsocketURL)

// Subscribing to a specific event:
observable, errCh := evtClient.EventsBytes(context.Background(), "your-query-string")
// Subscribing to a specific event, e.g. newly committed blocks:
// (see: https://docs.cosmos.network/v0.47/core/events#subscribing-to-events)
observable := evtClient.EventsBytes(ctx, "tm.event='NewBlock'")

// Subscribe and receive from the observer channel, typically in some other scope.
observer := observable.Subscribe(ctx)

// Observer channel closes when the context is cancelled, observer is
// unsubscribed, or after the subscription returns an error.
for eitherEvent := range observer.Ch() {
// (see either.Either: https://github.com/pokt-network/poktroll/blob/main/pkg/either/either.go#L3)
eventBz, err := eitherEvent.ValueOrError()

// ...
}
```

### Advanced Usage

_TODO(@bryanchriswhite): Add examples of advanced usage_
```go
// Given some custom dialer & connection implementation, e.g.:
var (
tcpDialer eventsquery.Dialer = exampletcp.NewTcpDialerImpl()
grcpDialer eventsquery.Dialer = examplegrpc.NewGrpcDialerImpl()
)

### Configuration
// Both TCP and gRPC use the TCP scheme as gRPC uses TCP for its transport layer.
cometUrl = "tcp://example.com"

- **WithDialer**: Configure the client to use a custom dialer for connections.
// Creating new EventsQueryClients with a custom tcpDialer:
tcpDialerOpt := eventsquery.WithDialer(tcpDialer)
tcpEvtClient := eventsquery.NewEventsQueryClient(cometUrl, tcpDialerOpt)

## API Reference
// Alternatively, with a custom gRPC dialer:
gcpDialerOpt := eventsquery.WithDialer(grcpDialer)
grpcEvtClient := eventsquery.NewEventsQueryClient(cometUrl, grpcDialerOpt)

- `EventsQueryClient`: Main interface to query events. Methods include:
- `EventsBytes(ctx, query)`: Returns an observable for chain events.
- `Close()`: Close any existing connections and unsubscribe all observers.
- `Connection`: Interface representing a bidirectional message-passing connection.
- `Dialer`: Interface encapsulating the creation of connections.
// ... rest follows the same as the basic example.
```

For the complete API details, see the [godoc](https://pkg.go.dev/github.com/pokt-network/poktroll/pkg/client/events_query).
### Configuration

- **WithDialer**: Configure the client to use a custom dialer for connections.

## Best Practices

- **Connection Handling**: Ensure to close the `EventsQueryClient` when done to free up resources and avoid potential leaks.
- **Error Handling**: Always check the error channel returned by `EventsBytes` for asynchronous errors during operation.
- **Error Handling**: Always check both the synchronous error returned by `EventsBytes` as well as asynchronous errors send over the observable.

## FAQ

#### Why use `events_query` over directly using Gorilla WebSockets?

`events_query` abstracts many of the underlying details and provides a streamlined interface for subscribing to chain events. It also integrates the observable pattern and provides mockable interfaces for better testing.
`events_query` abstracts many of the underlying details and provides a streamlined interface for subscribing to chain events.
It also integrates the observable pattern and provides mockable interfaces for better testing.

#### How can I use a different connection mechanism other than WebSockets?

You can implement the `Dialer` and `Connection` interfaces and use the `WithDialer` configuration to provide your custom dialer.

## Contributing

If you're interested in improving the `events_query` package or adding new features, please start by discussing your ideas in the project's issues section. Check our main contributing guide for more details.

## Changelog

For detailed release notes, see the [CHANGELOG](../CHANGELOG.md).

## License

This package is released under the XYZ License. For more information, see the [LICENSE](../LICENSE) file at the root level.
10 changes: 6 additions & 4 deletions pkg/client/events_query/client.go
Original file line number Diff line number Diff line change
@@ -153,7 +153,10 @@ func (eqc *eventsQueryClient) newEventsBytesAndConn(
// Construct an eventsBytes for the given query.
eventsBzObservable, eventsBzPublishCh := channel.NewObservable[either.Either[[]byte]]()

// TODO_INVESTIGATE: does this require retry on error?
// Publish either events bytes or an error received from the connection to
// the eventsBz observable.
// NB: intentionally not retrying on error, leaving that to the caller.
// (see: https://github.com/pokt-network/poktroll/pull/64#discussion_r1373826542)
go eqc.goPublishEventsBz(ctx, conn, eventsBzPublishCh)

return &eventsBytesAndConn{
@@ -271,9 +274,8 @@ func (eqc *eventsQueryClient) eventSubscriptionRequest(query string) ([]byte, er
// randRequestId returns a random 8 byte, base64 request ID which is intended
// for in JSON-RPC requests to uniquely identify distinct RPC requests.
// These request IDs only need to be unique to the extent that they are useful
// to this client for identifying distinct RPC requests.
// These IDs
// are expected to be unique (per request). Its size and keyspace are arbitrary.
// to this client for identifying distinct RPC requests. Their size and keyspace
// are arbitrary.
func randRequestId() string {
requestIdBz := make([]byte, 8) // 8 bytes = 64 bits = uint64
if _, err := rand.Read(requestIdBz); err != nil {
1 change: 1 addition & 0 deletions pkg/client/events_query/client_integration_test.go
Original file line number Diff line number Diff line change
@@ -12,6 +12,7 @@ import (
"pocket/internal/testclient/testeventsquery"
)

// The query use to subscribe for new block events on the websocket endpoint exposed by CometBFT nodes
const committedBlockEventsQuery = "tm.event='NewBlock'"

func TestQueryClient_EventsObservable_Integration(t *testing.T) {
You are viewing a condensed version of this merge commit. You can view the full changes here.