Skip to content

Commit

Permalink
feat: Use feature for amendment name (#291)
Browse files Browse the repository at this point in the history
## High Level Overview of Change

<!--
Please include a summary/list of the changes.
If too broad, please consider splitting into multiple PRs.
If a relevant Asana task, please link it here.
-->
Resolve: #278

### Context of Change

<!--
Please include the context of a change.
If a bug fix, when was the bug introduced? What was the behavior?
If a new feature, why was this architecture chosen? What were the
alternatives?
If a refactor, how is this better than the previous implementation?

If there is a design document for this feature, please link it here.
-->

Missing new amendments due to the change in rippled code structure to
parse name:

![Screenshot 2024-12-19 at 11 32
43 AM](https://github.com/user-attachments/assets/0da97dff-9d6d-4840-b8cf-6b44ba628053)

### Type of Change

<!--
Please check relevant options, delete irrelevant ones.
-->

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Refactor (non-breaking change that only restructures code)
- [ ] Tests (You added tests for code that already exists, or your new
feature included in this PR)
- [ ] Documentation Updates
- [ ] Release

## Before / After

<!--
If just refactoring / back-end changes, this can be just an in-English
description of the change at a technical level.
If a UI change, screenshots should be included.
-->
Data should show up:

![Screenshot 2024-12-19 at 11 58
52 AM](https://github.com/user-attachments/assets/5198638f-4c69-4158-9a31-fa004aab78c4)

<!--
## Future Tasks
For future tasks related to PR.
-->
VHS is lacking tests at the moment. There will be a separate ticket to
add comprehensive tests throughout the code
  • Loading branch information
pdp2121 authored Dec 23, 2024
1 parent 7c0180e commit 66c023f
Show file tree
Hide file tree
Showing 4 changed files with 118 additions and 80 deletions.
20 changes: 0 additions & 20 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@
"homepage": "https://github.com/ripple/validator-history-service#readme",
"devDependencies": {
"@types/axios": "^0.14.0",
"@types/create-hash": "^1.2.2",
"@types/express": "4.17.21",
"@types/jest": "^26.0.19",
"@types/nconf": "^0.10.0",
Expand Down Expand Up @@ -62,7 +61,6 @@
"@types/bunyan": "^1.8.7",
"axios": "^0.21.1",
"bunyan": "^1.8.15",
"create-hash": "^1.2.0",
"dotenv": "^16.3.1",
"express": "4.21.2",
"knex": "2.5.1",
Expand Down
10 changes: 4 additions & 6 deletions src/connection-manager/wsHandling.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,10 @@ import {
saveAmendmentStatus,
saveAmendmentsStatus,
} from '../shared/database'
import { deleteAmendmentStatus } from '../shared/database/amendments'
import {
NETWORKS_HOSTS,
deleteAmendmentStatus,
} from '../shared/database/amendments'
import {
AmendmentStatus,
DatabaseValidator,
Expand All @@ -32,11 +35,6 @@ const LEDGER_HASHES_SIZE = 10
const GOT_MAJORITY_FLAG = 65536
const LOST_MAJORITY_FLAG = 131072
const FOURTEEN_DAYS_IN_MILLISECONDS = 14 * 24 * 60 * 60 * 1000
const NETWORKS_HOSTS = new Map([
['main', 'ws://s2.ripple.com:51233'],
['test', 'wss://s.altnet.rippletest.net:51233'],
['dev', 'wss://s.devnet.rippletest.net:51233'],
])

const log = logger({ name: 'connections' })

Expand Down
166 changes: 114 additions & 52 deletions src/shared/database/amendments.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
import axios from 'axios'
import createHash from 'create-hash'
import { Client, ErrorResponse } from 'xrpl'
import {
FeatureAllResponse,
FeatureOneResponse,
} from 'xrpl/dist/npm/models/methods/feature'

import { AmendmentInfo } from '../types'
import logger from '../utils/logger'
Expand All @@ -9,78 +13,135 @@ import { query } from './utils'
const log = logger({ name: 'amendments' })

const amendmentIDs = new Map<string, { name: string; deprecated: boolean }>()
const votingAmendmentsToTrack = new Set<string>()
const rippledVersions = new Map<string, string>()

const ACTIVE_AMENDMENT_REGEX =
/^\s*REGISTER_F[A-Z]+\s*\((?<amendmentName>\S+),\s*.*$/u
const RETIRED_AMENDMENT_REGEX =
/^ .*retireFeature\("(?<amendmentName>\S+)"\)[,;].*$/u
// TODO: Use feature RPC instead when this issue is fixed and released:
// https://github.com/XRPLF/rippled/issues/4730
const RETIRED_AMENDMENTS = [
'MultiSign',
'TrustSetAuth',
'FeeEscalation',
'PayChan',
'CryptoConditions',
'TickSize',
'fix1368',
'Escrow',
'fix1373',
'EnforceInvariants',
'SortedDirectories',
'fix1201',
'fix1512',
'fix1523',
'fix1528',
]

const AMENDMENT_VERSION_REGEX =
/\| \[(?<amendmentName>[a-zA-Z0-9_]+)\][^\n]+\| (?<version>v[0-9]*\.[0-9]*\.[0-9]*|TBD) *\|/u

// TODO: Clean this up when this PR is merged:
// https://github.com/XRPLF/rippled/pull/4781
export const NETWORKS_HOSTS = new Map([
['main', 'ws://s2.ripple.com:51233'],
['test', 'wss://s.altnet.rippletest.net:51233'],
['dev', 'wss://s.devnet.rippletest.net:51233'],
])

/**
* Fetch a list of amendments names from rippled file.
* Fetch amendments information including id, name, and deprecated status.
*
* @returns The list of amendment names.
* @returns Void.
*/
async function fetchAmendmentNames(): Promise<Map<string, boolean> | null> {
async function fetchAmendmentsList(): Promise<void> {
for (const [network, url] of NETWORKS_HOSTS) {
await fetchNetworkAmendments(network, url)
}
}

/**
* Fetch amendments information including id, name, and deprecated status of a network.
*
* @param network - The network being retrieved.
* @param url - The Faucet URL of the network.
*
* @returns Void.
*/
async function fetchNetworkAmendments(
network: string,
url: string,
): Promise<void> {
try {
const response = await axios.get(
'https://raw.githubusercontent.com/XRPLF/rippled/develop/src/libxrpl/protocol/Feature.cpp',
)
const text = response.data
const amendmentNames: Map<string, boolean> = new Map()
text.split('\n').forEach((line: string) => {
const name = ACTIVE_AMENDMENT_REGEX.exec(line)
if (name) {
amendmentNames.set(name[1], name[0].includes('VoteBehavior::Obsolete'))
} else {
const name2 = RETIRED_AMENDMENT_REGEX.exec(line)
if (name2) {
amendmentNames.set(name2[1], true)
}
}
log.info(`Updating amendment info for ${network}...`)
const client = new Client(url)
await client.connect()
const featureAllResponse: FeatureAllResponse = await client.request({
command: 'feature',
})
return amendmentNames
} catch (err) {
log.error('Error getting amendment names', err)
return null

const featuresAll = featureAllResponse.result.features

for (const id of Object.keys(featuresAll)) {
addAmendmentToCache(id, featuresAll[id].name)
}

// Some amendments in voting are not available in feature all request.
// This loop tries to fetch them in feature one.
for (const amendment_id of votingAmendmentsToTrack) {
const featureOneResponse: FeatureOneResponse | ErrorResponse =
await client.request({
command: 'feature',
feature: amendment_id,
})

// eslint-disable-next-line max-depth -- The depth is only 2, try catch should not count.
if ('result' in featureOneResponse) {
const feature = featureOneResponse.result[amendment_id]
addAmendmentToCache(amendment_id, feature.name)
}
}

await client.disconnect()

log.info(`Finished updating amendment info for ${network}...`)
} catch (error) {
log.error(
`Failed to update amendment info for ${network} due to error: ${String(
error,
)}`,
)
}
}

/**
* Extracts Amendment ID from Amendment name inside a buffer.
* Add an amendment to amendmentIds cache and remove it from the votingAmendmentToTrack cache.
*
* @param buffer -- The buffer containing the amendment name.
*
* @returns The amendment ID string.
* @param id - The id of the amendment to add.
* @param name - The name of the amendment to add.
*/
function sha512Half(buffer: Buffer): string {
return createHash('sha512')
.update(buffer)
.digest('hex')
.toUpperCase()
.slice(0, 64)
function addAmendmentToCache(id: string, name: string): void {
amendmentIDs.set(id, {
name,
deprecated: RETIRED_AMENDMENTS.includes(name),
})
votingAmendmentsToTrack.delete(id)
}

/**
* Maps the id of Amendments to its corresponding names.
* Fetch amendments in voting.
*
* @returns Void.
*/
async function nameOfAmendmentID(): Promise<void> {
// The Amendment ID is the hash of the Amendment name
const amendmentNames = await fetchAmendmentNames()
if (amendmentNames !== null) {
amendmentNames.forEach((deprecated, name) => {
amendmentIDs.set(sha512Half(Buffer.from(name, 'ascii')), {
name,
deprecated,
})
})
async function fetchVotingAmendments(): Promise<void> {
const votingDb = await query('ballot')
.select('amendments')
.then(async (res) =>
res.map((vote: { amendments: string | null }) => vote.amendments),
)
for (const amendmentsDb of votingDb) {
if (!amendmentsDb) {
continue
}
const amendments = amendmentsDb.split(',')
for (const amendment of amendments) {
votingAmendmentsToTrack.add(amendment)
}
}
}

Expand Down Expand Up @@ -145,7 +206,8 @@ export async function deleteAmendmentStatus(

export async function fetchAmendmentInfo(): Promise<void> {
log.info('Fetch amendments info from data sources...')
await nameOfAmendmentID()
await fetchVotingAmendments()
await fetchAmendmentsList()
await fetchMinRippledVersions()
amendmentIDs.forEach(async (value, id) => {
const amendment: AmendmentInfo = {
Expand Down

0 comments on commit 66c023f

Please sign in to comment.