SRE quarterly update 2024-Q3

quarterly/2024-Q3-sre.md

@@ -0,0 +1,85 @@
+---
+title: SRE Q3 2024 Update
+slug: 2024-Q3-sre
+authors: johnalotoski
+tags: [sre]
+hide_table_of_contents: false
+---
+
+## 2024-07 - 2024-09
+
+### Main achievements
+
+In addition to ongoing general maintenance and support of cardano environments,
+SRE achievements for this quarter include:
+
+* All IOE cardano-parts supported node environments, including preview,
+  preprod, sanchonet, mainnet and other clusters were upgraded through various
+  cardano-node releases of `9.0.0`, `9.1.0`, `9.1.1`, `9.2.0`, and finally into
+  `9.2.1` by the end of September.
+
+* All IOE cardano-parts supported node environments had dual stack ipv4/ipv6
+  capability added and configured, including supporting scripts and recipes,
+  module updates, terraform/openTofu resource changes and software updates to
+  make previously ipv6 incompatible software ipv6 compatible, example:
+  cardano-faucet. Cardano-parts clusters can now seamlessly participate in ipv6
+  cardano-node traffic and other ipv6 traffic.
+
+* Preview, preprod and mainnet networks were hard forked to Conway.
+
+* Legacy mainnet cluster shelley-era high-load relays were scaled down over the
+  quarter and stopped now that p2p has removed the need for them.
+
+* Legacy cardano [explorer](https://explorer.cardano.org/) was retired and
+  Cardano Foundation is now providing the replacement landing page which links
+  to several community explorers.
+
+* Cardano-smash production load was retired from equinix metal hosting from the
+  cardano-world repo and transferred to the new cardano-mainnet cluster.
+
+* New cardano-mainnet cluster scaling capability was added for the bootstrap
+  machines.  Block performance analysis was used to tune RTS parameters on the
+  bootstraps and other mainnet pool machines.
+
+* Sanchonet environment was re-spun for cardano-node `9.1.0` and greater compatibility.
+
+* Private chain was stopped and re-spun with 2 hr epochs for testing.
+
+* New nixosModules were added to cardano-parts and cardano-playground,
+  including: `profile-blockperf`, `profile-tcpdump` (for saving node traffic
+  pcaps to s3) and `ogmios`.
+
+* Documentation for playground and mainnet cluster operations was improved,
+  such as documents for: debugging of peer-to-peer connections; governance
+  voting with the playground stakepools; faucet setup; faucet pool
+  de-delegation and mainnet dbsync cardano-snapshot operations. See the
+  `docs/explain` directory of both the
+  [cardano-playground](https://github.com/input-output-hk/cardano-playground)
+  and [cardano-mainnet](https://github.com/input-output-hk/cardano-mainnet)
+  repos for details.
+
+* The [cardano-monitoring](https://github.com/input-output-hk/cardano-monitoring)
+  repository received a lot of documentation and improvements and now also
+  serves as the home for `devx-ci` metrics after migration away from Grafana
+  cloud hosting.
+
+* An improved [cardano-airgap](https://github.com/Intersectmbo/cardano-airgap)
+  image for secure signing operations was created and made available.
+
+* Hydra CI performance was improved with changes to our custom Nix evaluator
+  and optimized resource usage while waiting for IFDs.
+
+### Next steps
+
+* Add a production protocol-parameters cardano-api based server to facilitate
+  community transaction creation without requiring a live node.
+
+* Migrate from deprecated grafana agent to grafana alloy.
+
+* Finalize support for the new cardano-node tracing system once the service is
+  rewritten for general consumption.
+
+* Extend govtool frontend and backend to a process-compose stack once govtool
+  is publicly buildable again.
+
+* Continue cardano-parts and operations improvements.