Add some new prometheus metrics (#19)

* gitignore localtemp * PoolTest Conf * ShareLog init * External Hello Init * shares_logger init * log_share Share hash * static LOGGER_SENDER Bounded channel * primary_rx backup_rx * ShareLogProto * pub struct ShareLog tested * ShareLogProto removed * imports remove * ClickHouse init tested * insert_batch clickhouse 0.13.1 * share_processor.rs Tested * with share_diff * TEMP remove clickhouse channel * 169-3-1 DebugLog Proto * 169-3-1 Debugger-Logger-v.1 * services::debug_log commented * MATERIALIZED VIEW mv_hash_rate_stats * shares_per_minute to 10 * Backtrace::force_capture * 169-3-1 ExtendedChannelKind * rm -rf hello :) * 169-3-1 new config.rs * 169-3-1 comments to English * 169-3-1 README * 169-3-1 README * 169-3-1 rm handshake ) * 169-5 ShareLog to models * ClickhouseShare to models * 169-5 share to share_log * TEMP 169-5 New Config Mod (ENV in progress) * 169-5 TEMP ENV * 169-5 ENV to settings * 169-5 README * 169-5 Syntax Fix * 169-5 Spaces Fix * 169-5 ClickhouseError to mod * 169-5 is_valid to ShareStatus * 169-5 ClickHouse to trait * 169-5 Deps Removed * 169-5 ShareLogger refactoring * test_highload_share_processing * 169-5 test_highload_share_processing improve * 169-5 README Refreshed * 169-6 ShareData * 169-6 test_share_types * 169-6 TEMP BlockFound * 169-6 shares_logger::log_block(block) * 169-6 store_share on BlockFound * 169-7 Queries To Files * 169-7 Small Fix * 169-7 README * 169-7 Cargo Fix * 169-7 share.is_block_found fn * 169-7 Small Fixes * min_individual_miner_hashrate * RRP-318 nomium_test_worker from Translator to Pool hardcoded * RRP-318 user_identity from worker * RRP-318 user_identity to prepare_share_log * RRP-318 user_identity to ClickHouse 1 * RRP-318 user_identity DB optimized * RRP-318 user_identity on BlockFound * 169-11 handle_authorize with reqwest proto * 169-11 worker_name_store MOD * 169-11 worker_name_store to shares-logger * 169-11 shares with worker_id * 169-11 worker_id on blocks * 169-11 Tables n MV updated * 169-11 mv_hash_rate_stats Readme-Ru * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * 169-11 wo OpenSSL * Update rrp-translator.Dockerfile * Update docker-compose.yml * Update docker-compose.yml * 169-11 new mv_hash_rate_stats * 169-11 Remove Obsolete Tests and fn to_storage_format * 169-11 README-RU.md mod * 169-11 share_data Trait rm * 169-11 ENV REDROCK_API_URL REDROCK_API_KEY * 169-11 New MV n HR Algo * 169-11 small * 169-11 Timeout to ENV etc * RRP-428 Logging For Proxy * RRP-428 Translator To Log File * Add environment to docker-compose (#13) * Update docker-compose.yml * Update docker-compose.yml * Update default_config.toml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update deploy.py * Update deploy.py * Update deploy.py --------- Co-authored-by: Akulov Oleg <[email protected]> * RRP-428 logs moved to roles * Update docker-compose.yml * RRP-428 Pool To Log-File * Update docker-compose.yml * RRP-428 Log Dir Fix * RRP-428 Ban Pool Shares * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * gitignore logs * RRP-428 TPROXY_LOG_LEVEL To ENV * Update pool-config-hosted-tp-example.toml * Update rrp-pool.Dockerfile * Update rrp-pool.Dockerfile * Update rrp-pool.Dockerfile * Update deploy.py * Update rrp-pool.Dockerfile * Update docker-compose.yml * Update docker-compose.yml * Update pool-config-hosted-tp-example.toml * Update rrp-pool.Dockerfile * RRP-433 Prometheus Basic * Update rrp-pool.Dockerfile * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update deploy.py * RRP-433 ACTIVE_CONNECTIONS * RRP-433 mining_shares_valid_jobid_total * RRP-433 mining_shares_received_total * RRP-433 SHARES_UPSTREAM_TARGET_MEET SHARES_DOWNSTREAM_TARGET_MEET * RRP-433 SOME CONNECTIONS * RRP-428 Pub Key * RRP-428 P2WSH output_script_value * RRP-433 Content-Type * Update docker-compose.yml * Update rrp-pool.Dockerfile * RRP-428 Pool ENV Base * RRP-428 ENV on TP_ADDRESS SCRIPT_TYPE SCRIPT_VALUE * RRP-428 TPROXY_METRICS_IP To ENV * RRP-428 m.get_sequence_number * RRP-426 is_rewards_calculated * RRP-426 account_name * RRP-426 user_identity REMOVED * RRP-426 block_hash.reverse * RRP-426 pub user_identity REMOVED * RRP-426 is_rewards_calculated Bool DEFAULT false * RRP-429 pre-alfa with error job_creator.rs:128:18: not yet implemented: 2 * RRP-429 template.len > 1 FIXED * RRP-429 POOL_TP_RECONNECT_TIMEOUT * Update docker-compose.yml * Update docker-compose.yml * SILENT_DEATH sigterm sigint * Update docker-compose.yml * Update docker-compose.yml * Merge nomium-main to reconnect branch (#18) * RRP-426: Add db fields for reward calculation (#16) * gitignore localtemp * PoolTest Conf * ShareLog init * External Hello Init * shares_logger init * log_share Share hash * static LOGGER_SENDER Bounded channel * primary_rx backup_rx * ShareLogProto * pub struct ShareLog tested * ShareLogProto removed * imports remove * ClickHouse init tested * insert_batch clickhouse 0.13.1 * share_processor.rs Tested * with share_diff * TEMP remove clickhouse channel * 169-3-1 DebugLog Proto * 169-3-1 Debugger-Logger-v.1 * services::debug_log commented * MATERIALIZED VIEW mv_hash_rate_stats * shares_per_minute to 10 * Backtrace::force_capture * 169-3-1 ExtendedChannelKind * rm -rf hello :) * 169-3-1 new config.rs * 169-3-1 comments to English * 169-3-1 README * 169-3-1 README * 169-3-1 rm handshake ) * 169-5 ShareLog to models * ClickhouseShare to models * 169-5 share to share_log * TEMP 169-5 New Config Mod (ENV in progress) * 169-5 TEMP ENV * 169-5 ENV to settings * 169-5 README * 169-5 Syntax Fix * 169-5 Spaces Fix * 169-5 ClickhouseError to mod * 169-5 is_valid to ShareStatus * 169-5 ClickHouse to trait * 169-5 Deps Removed * 169-5 ShareLogger refactoring * test_highload_share_processing * 169-5 test_highload_share_processing improve * 169-5 README Refreshed * 169-6 ShareData * 169-6 test_share_types * 169-6 TEMP BlockFound * 169-6 shares_logger::log_block(block) * 169-6 store_share on BlockFound * 169-7 Queries To Files * 169-7 Small Fix * 169-7 README * 169-7 Cargo Fix * 169-7 share.is_block_found fn * 169-7 Small Fixes * min_individual_miner_hashrate * RRP-318 nomium_test_worker from Translator to Pool hardcoded * RRP-318 user_identity from worker * RRP-318 user_identity to prepare_share_log * RRP-318 user_identity to ClickHouse 1 * RRP-318 user_identity DB optimized * RRP-318 user_identity on BlockFound * 169-11 handle_authorize with reqwest proto * 169-11 worker_name_store MOD * 169-11 worker_name_store to shares-logger * 169-11 shares with worker_id * 169-11 worker_id on blocks * 169-11 Tables n MV updated * 169-11 mv_hash_rate_stats Readme-Ru * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * Update rrp-translator.Dockerfile * 169-11 wo OpenSSL * Update rrp-translator.Dockerfile * Update docker-compose.yml * Update docker-compose.yml * 169-11 new mv_hash_rate_stats * 169-11 Remove Obsolete Tests and fn to_storage_format * 169-11 README-RU.md mod * 169-11 share_data Trait rm * 169-11 ENV REDROCK_API_URL REDROCK_API_KEY * 169-11 New MV n HR Algo * 169-11 small * 169-11 Timeout to ENV etc * RRP-428 Logging For Proxy * RRP-428 Translator To Log File * Add environment to docker-compose (#13) * Update docker-compose.yml * Update docker-compose.yml * Update default_config.toml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update deploy.py * Update deploy.py * Update deploy.py --------- Co-authored-by: Akulov Oleg <[email protected]> * RRP-428 logs moved to roles * Update docker-compose.yml * RRP-428 Pool To Log-File * Update docker-compose.yml * RRP-428 Log Dir Fix * RRP-428 Ban Pool Shares * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * gitignore logs * RRP-428 TPROXY_LOG_LEVEL To ENV * Update pool-config-hosted-tp-example.toml * Update rrp-pool.Dockerfile * Update rrp-pool.Dockerfile * Update rrp-pool.Dockerfile * Update deploy.py * Update rrp-pool.Dockerfile * Update docker-compose.yml * Update docker-compose.yml * Update pool-config-hosted-tp-example.toml * Update rrp-pool.Dockerfile * RRP-433 Prometheus Basic * Update rrp-pool.Dockerfile * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update docker-compose.yml * Update deploy.py * RRP-433 ACTIVE_CONNECTIONS * RRP-433 mining_shares_valid_jobid_total * RRP-433 mining_shares_received_total * RRP-433 SHARES_UPSTREAM_TARGET_MEET SHARES_DOWNSTREAM_TARGET_MEET * RRP-433 SOME CONNECTIONS * RRP-428 Pub Key * RRP-428 P2WSH output_script_value * RRP-433 Content-Type * Update docker-compose.yml * Update rrp-pool.Dockerfile * RRP-428 Pool ENV Base * RRP-428 ENV on TP_ADDRESS SCRIPT_TYPE SCRIPT_VALUE * RRP-428 TPROXY_METRICS_IP To ENV * RRP-428 m.get_sequence_number * RRP-426 is_rewards_calculated * RRP-426 account_name * RRP-426 user_identity REMOVED * RRP-426 block_hash.reverse * RRP-426 pub user_identity REMOVED * RRP-426 is_rewards_calculated Bool DEFAULT false --------- Co-authored-by: Roman Dukhanin <[email protected]> Co-authored-by: inha1e <[email protected]> Co-authored-by: Akulov Oleg <[email protected]> Co-authored-by: Oleg Akulov <[email protected]> * Update docker-compose.yml * Update docker-compose.yml --------- Co-authored-by: MLGODS <[email protected]> Co-authored-by: Roman Dukhanin <[email protected]> Co-authored-by: inha1e <[email protected]> Co-authored-by: Oleg Akulov <[email protected]> * RRP-429 SMALL FIX * RRP-433 TPROXY_SHARES_REJECTED_TOTAL * RRP-433 nomium-prometheus :-) * RRP-433 CHFACT_SHARES_LOGGED_TOTAL * RRP-433 SHALOG_ * RRP-433 SHALOG_PRIMARY_STORE_FAILED_TOTAL * RRP-433 README.md --------- Co-authored-by: Roman Dukhanin <[email protected]> Co-authored-by: inha1e <[email protected]> Co-authored-by: Akulov Oleg <[email protected]> Co-authored-by: Oleg Akulov <[email protected]>
nomium-org · Jan 10, 2025 · 3760832 · 3760832
1 parent 5b79237
commit 3760832
Show file tree

Hide file tree

Showing 11 changed files with 111 additions and 9 deletions.
diff --git a/nomium/prometheus/Cargo.toml b/nomium/prometheus/Cargo.toml
@@ -0,0 +1,12 @@
+[package]
+name = "nomium-prometheus"
+version = "0.1.0"
+edition = "2021"
+
+[dependencies]
+lazy_static = "1.5"
+prometheus = "0.13"
+prometheus-static-metric = "0.5"
+tiny_http = "0.12"
+tracing = { version = "0.1" }
+dotenvy = "0.15"
diff --git a/nomium/prometheus/README.md b/nomium/prometheus/README.md
@@ -0,0 +1,20 @@
+```
+mining_shares_received_total - Общее количество шар, полученных от майнеров до любой валидации
+mining_shares_valid_jobid_total - Количество шар с корректным идентификатором работы
+mining_shares_upstream_target_meet_total - Количество шар, соответствующих таргету апстрим пула
+mining_shares_downstream_target_meet_total - Количество шар, соответствующих таргету майнера
+tproxy_shares_rejected_total - Общее количество отклоненных шар
+mining_active_connections - Количество активных подключений майнеров
+mining_connection_attempts_total - Общее количество попыток подключения
+mining_connection_failures_total - Количество неудачных попыток подключения
+mining_auth_failures_total - Количество ошибок аутентификации
+mining_timeout_failures_total - Количество таймаутов подключения
+chfact_shares_logged_total - Общее количество шар, отправленных в логгер
+shalog_shares_received_total - Общее количество шар, полученных логгером шар
+shalog_primary_channel_shares_total - Общее количество шар, отправленных через основной канал
+shalog_backup_channel_shares_total - Общее количество шар, отправленных через резервный канал
+shalog_primary_try_stored_total - Общее количество попыток сохранения шар через основной канал
+shalog_backup_try_stored_total - Общее количество попыток сохранения шар через резервный канал
+shalog_primary_store_failed_total - Количество неудачных попыток сохранения шар через основной канал
+shalog_backup_store_failed_total - Количество неудачных попыток сохранения шар через резервный канал
+```
diff --git a/roles/translator/src/lib/metrics.rs → nomium/prometheus/src/lib.rs b/roles/translator/src/lib/metrics.rs → nomium/prometheus/src/lib.rs
@@ -23,6 +23,10 @@ lazy_static! {
         register_int_counter!("mining_shares_downstream_target_meet_total", 
             "Shares that meet downstream (miner) target").unwrap();
 
+    pub static ref TPROXY_SHARES_REJECTED_TOTAL: IntCounter =
+        register_int_counter!("tproxy_shares_rejected_total", 
+            "Total number of rejected shares").unwrap();
+
     pub static ref ACTIVE_CONNECTIONS: IntGauge =
         register_int_gauge!("mining_active_connections", 
             "Number of active miner connections").unwrap();
@@ -43,6 +47,38 @@ lazy_static! {
         register_int_counter!("mining_timeout_failures_total",
             "Number of connection timeouts").unwrap();
 
+    pub static ref CHFACT_SHARES_LOGGED_TOTAL: IntCounter =
+        register_int_counter!("chfact_shares_logged_total", 
+            "Total number of shares sent to logger").unwrap();
+
+    pub static ref SHALOG_SHARES_RECEIVED_TOTAL: IntCounter =
+        register_int_counter!("shalog_shares_received_total", 
+            "Total number of shares received by shares-logger").unwrap();
+
+    pub static ref SHALOG_PRIMARY_CHANNEL_SHARES_TOTAL: IntCounter =
+        register_int_counter!("shalog_primary_channel_shares_total", 
+            "Total number of shares sent through primary channel").unwrap();
+
+    pub static ref SHALOG_BACKUP_CHANNEL_SHARES_TOTAL: IntCounter =
+        register_int_counter!("shalog_backup_channel_shares_total", 
+            "Total number of shares sent through backup channel").unwrap();
+
+    pub static ref SHALOG_PRIMARY_TRY_STORED_TOTAL: IntCounter =
+        register_int_counter!("shalog_primary_try_stored_total", 
+            "Total number of shares stored from primary channel").unwrap();
+
+    pub static ref SHALOG_BACKUP_TRY_STORED_TOTAL: IntCounter =
+        register_int_counter!("shalog_backup_try_stored_total", 
+            "Total number of shares stored from backup channel").unwrap();
+
+    pub static ref SHALOG_PRIMARY_STORE_FAILED_TOTAL: IntCounter =
+        register_int_counter!("shalog_primary_store_failed_total", 
+            "Total number of shares failed to store from primary channel").unwrap();
+
+    pub static ref SHALOG_BACKUP_STORE_FAILED_TOTAL: IntCounter =
+        register_int_counter!("shalog_backup_store_failed_total", 
+            "Total number of shares failed to store from backup channel").unwrap();
+
 }
 
 pub fn start_metrics_server() {

diff --git a/nomium/shares-logger/Cargo.toml b/nomium/shares-logger/Cargo.toml
@@ -17,6 +17,7 @@ primitive-types = "0.12"
 lazy_static = "1.4"
 config = "0.13"
 parking_lot = "0.12"
+nomium-prometheus = { path = "../prometheus" }
 
 [dev-dependencies]
 tokio = { version = "1.0", features = ["rt", "macros", "test-util", "time"] }
diff --git a/nomium/shares-logger/src/lib.rs b/nomium/shares-logger/src/lib.rs
@@ -21,6 +21,16 @@ use std::time::Instant;
 use serde::Serialize;
 use serde::de::DeserializeOwned;
 
+use nomium_prometheus::{
+    SHALOG_SHARES_RECEIVED_TOTAL,
+    SHALOG_PRIMARY_CHANNEL_SHARES_TOTAL, 
+    SHALOG_BACKUP_CHANNEL_SHARES_TOTAL,
+    SHALOG_PRIMARY_TRY_STORED_TOTAL,
+    SHALOG_BACKUP_TRY_STORED_TOTAL,
+    SHALOG_PRIMARY_STORE_FAILED_TOTAL,
+    SHALOG_BACKUP_STORE_FAILED_TOTAL
+};
+
 lazy_static! {
     static ref GLOBAL_LOGGER: ShareLogger<ShareLog> = {
         let storage = ClickhouseStorage::new()
@@ -107,9 +117,14 @@ impl<T: Send + Sync + Clone + Serialize + DeserializeOwned + 'static> ShareLogge
 
 impl<T: Send + Sync + Clone + Serialize + DeserializeOwned + 'static> ShareLogger<T> {
     pub fn log_share(&self, share: T) {
+        SHALOG_SHARES_RECEIVED_TOTAL.inc();
+
         match self.primary_tx.try_send(share.clone()) {
-            Ok(_) => (),
+            Ok(_) => {
+                SHALOG_PRIMARY_CHANNEL_SHARES_TOTAL.inc();
+            },
             Err(TrySendError::Full(share)) | Err(TrySendError::Closed(share)) => {
+                SHALOG_BACKUP_CHANNEL_SHARES_TOTAL.inc();
                 if let Err(e) = self.backup_tx.send(share) {
                     info!("Failed to send share to backup logger: {}", e);
                 }
@@ -136,7 +151,9 @@ async fn process_shares<T: Send + Sync + Clone + Serialize + DeserializeOwned>(
         tokio::select! {
             Some(share) = primary_rx.recv() => {
                 info!("Processing share from primary channel");
+                SHALOG_PRIMARY_TRY_STORED_TOTAL.inc();
                 if let Err(e) = storage.lock().await.store_share(share).await {
+                    SHALOG_PRIMARY_STORE_FAILED_TOTAL.inc();
                     info!("Failed to store share: {}", e);
                 }
             }
@@ -146,7 +163,10 @@ async fn process_shares<T: Send + Sync + Clone + Serialize + DeserializeOwned>(
                     backup_shares.push(share);
                 }
                 if !backup_shares.is_empty() {
+                    let shares_count = backup_shares.len() as u64;
+                    SHALOG_BACKUP_TRY_STORED_TOTAL.inc_by(shares_count);
                     if let Err(e) = storage.lock().await.store_batch(backup_shares).await {
+                        SHALOG_BACKUP_STORE_FAILED_TOTAL.inc_by(shares_count);
                         info!("Failed to store backup shares: {}", e);
                     }
                 }

diff --git a/protocols/v2/roles-logic-sv2/Cargo.toml b/protocols/v2/roles-logic-sv2/Cargo.toml
@@ -28,6 +28,7 @@ chacha20poly1305 = { version = "0.10.1"}
 nohash-hasher = "0.2.0"
 siphasher = "1"
 shares-logger = { path = "../../../nomium/shares-logger" }
+nomium-prometheus = { path = "../../../nomium/prometheus" }
 
 [dev-dependencies]
 quickcheck = "1.0.3"

diff --git a/protocols/v2/roles-logic-sv2/src/channel_logic/channel_factory.rs b/protocols/v2/roles-logic-sv2/src/channel_logic/channel_factory.rs
@@ -870,6 +870,7 @@ impl ChannelFactory {
                 );
                 info!("Calling share logging for PROXY");
                 shares_logger::log_share(share_log);
+                nomium_prometheus::CHFACT_SHARES_LOGGED_TOTAL.inc();
             }
         }
         // ---- NOMIUM share_log injection ----

diff --git a/roles/Cargo.lock b/roles/Cargo.lock
diff --git a/roles/translator/Cargo.toml b/roles/translator/Cargo.toml
@@ -48,10 +48,8 @@ reqwest = { version = "0.12", features = ["json", "rustls-tls"], default-feature
 shares-logger = { path = "../../nomium/shares-logger" }
 dotenvy = "0.15"
 tracing-appender = "0.2"
-prometheus = "0.13"
-prometheus-static-metric = "0.5"
 lazy_static = "1.5"
-tiny_http = "0.12"
+nomium-prometheus = { path = "../../nomium/prometheus" }
 
 [dev-dependencies]
 sha2 = "0.10.6"

diff --git a/roles/translator/src/lib/mod.rs b/roles/translator/src/lib/mod.rs
@@ -1,4 +1,4 @@
-pub mod metrics;
+pub use nomium_prometheus as metrics;
 
 use async_channel::{bounded, unbounded};
 use futures::FutureExt;

diff --git a/roles/translator/src/lib/proxy/bridge.rs b/roles/translator/src/lib/proxy/bridge.rs
@@ -1,4 +1,4 @@
-use crate::metrics::{SHARES_UPSTREAM_TARGET_MEET, SHARES_DOWNSTREAM_TARGET_MEET};
+use crate::metrics::{SHARES_UPSTREAM_TARGET_MEET, SHARES_DOWNSTREAM_TARGET_MEET, TPROXY_SHARES_REJECTED_TOTAL};
 use async_channel::{Receiver, Sender};
 use roles_logic_sv2::{
     channel_logic::channel_factory::{ExtendedChannelKind, ProxyExtendedChannelFactory, Share},
@@ -250,6 +250,7 @@ impl Bridge {
                     "Submit share error {:?}",
                     std::str::from_utf8(&e.error_code.to_vec()[..])
                 );
+                TPROXY_SHARES_REJECTED_TOTAL.inc();
             }
             Ok(Ok(OnNewShare::SendSubmitShareUpstream((share, _)))) => {
                 info!("SHARE MEETS UPSTREAM TARGET");