Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

app: fix broken event indexing #5009

Merged
merged 2 commits into from
Jan 29, 2025
Merged

app: fix broken event indexing #5009

merged 2 commits into from
Jan 29, 2025

Conversation

erwanor
Copy link
Member

@erwanor erwanor commented Jan 28, 2025

Describe your changes

We missed this on reviewing #4963 which causes a series of unfortunate events in the data pipeline: #4999

The important diff:

-set_index(false)
+set_index(true)

Issue ticket number and link

Checklist before requesting a review

  • I have added guiding text to explain how a reviewer should test these changes.

I have not tested this change. To test it we can run pd and check if events are making it to the comet pg indexer.

  • If this code contains consensus-breaking changes, I have added the "consensus-breaking" label. Otherwise, I declare my belief that there are not consensus-breaking changes, for the following reason:

    event indexing layer

@conorsch
Copy link
Contributor

Great find, and sufficient to resolve the problem reported in #4999. I've tacked on some integration tests for pindexer, off by default, that can be run like so: cargo nextest run --release -p pindexer --features network-integration

When running against a full node created from the main branch, the tests fail:

        PASS [   7.019s] pindexer::network_integration chain_is_progressing
------------
     Summary [   7.020s] 9 tests run: 2 passed, 7 failed, 0 skipped
        FAIL [   0.010s] pindexer::network_integration cometbft_events_are_not_null::case_1
        FAIL [   0.009s] pindexer::network_integration cometbft_events_are_not_null::case_2
        FAIL [   0.007s] pindexer::network_integration cometbft_events_are_not_null::case_3
        FAIL [   0.008s] pindexer::network_integration cometbft_events_are_not_null::case_4
        FAIL [   0.008s] pindexer::network_integration cometbft_events_are_not_null::case_5
        FAIL [   0.007s] pindexer::network_integration cometbft_events_are_not_null::case_6
        FAIL [   0.017s] pindexer::network_integration pindexer_is_working
error: test run failed

However when run on this feature branch, they pass:

------------
 Nextest run ID b1082db8-d479-4d8b-85c8-4b7736312728 with nextest profile: default
    Starting 9 tests across 3 binaries
        PASS [   0.008s] pindexer::network_integration cometbft_events_are_not_null::case_3
        PASS [   0.009s] pindexer::network_integration cometbft_events_are_not_null::case_1
        PASS [   0.008s] pindexer::network_integration cometbft_events_are_not_null::case_4
        PASS [   0.009s] pindexer::network_integration cometbft_events_are_not_null::case_2
        PASS [   0.008s] pindexer::network_integration cometbft_events_are_not_null::case_5
        PASS [   0.008s] pindexer::network_integration cometbft_events_are_not_null::case_6
        PASS [   0.015s] pindexer::network_integration cometbft_indexing_is_working
        PASS [   0.015s] pindexer::network_integration pindexer_is_working
        PASS [   7.019s] pindexer::network_integration chain_is_progressing
------------
     Summary [   7.019s] 9 tests run: 9 passed, 0 skipped

which is exactly what we want to see. Ideally these tests would run in PRs, so we can catch breakage early, but I'll make that change in subsequent PRs, cleaning up the smoke-test logic. This fix is solid and we want it in now!

Adds some feature-gated integration tests, not yet hooked up to CI,
to sanity check that the ABCI events pipeline is working:

  pd -> cometbft -> postgres -> pindexer

The integration tests talk to local postgres databases and make
assertions about their contents. There's also a small change to the
process-compose orchestration, instructing pindexer to wait for the
fullnode services to come up, lest it error out early due to an empty
database. The error I observed on an empty db was:

  Error: error occurred while decoding column 0: unexpected null;
  try decoding as an `Option`
  Caused by:
    unexpected null; try decoding as an `Option`

Which is distinct from the error reported in #4999. By waiting
a bit longer we can ensure a clean start even on a fresh devnet.

Will follow up with subsequent PRs to make sure these tests run in CI.
@conorsch conorsch merged commit bc912f0 into main Jan 29, 2025
15 checks passed
@conorsch conorsch deleted the erwan/fix_broken_events branch January 29, 2025 03:59
conorsch added a commit that referenced this pull request Feb 5, 2025
Follow-up to #5009, in which these tests were introduced, and #5048,
which refactored the smoke tests to make additions like this easier.
conorsch added a commit that referenced this pull request Feb 5, 2025
## Describe your changes

Follow-up to #5009, in which these tests were introduced, and #5048,
which refactored the smoke tests to make additions like this easier.

## Issue ticket number and link
See above.

## Testing and review

The tests were already merged, this is just enabling them in CI, so CI
passing, specifically on the smoke-test job, is enough.

## Checklist before requesting a review

- [x] I have added guiding text to explain how a reviewer should test
these changes.

- [x] If this code contains consensus-breaking changes, I have added the
"consensus-breaking" label. Otherwise, I declare my belief that there
are not consensus-breaking changes, for the following reason:

  > tests/ci only, no changes to application code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants