Merge pull request #10 from ii/add-cross-posts-from-caleb+cncf

feat: add cross-posts from Caleb+CNCF

content/post/experimenting-with-migrating-from-fedora-workstation-to-fedora-silverblue.org

@@ -0,0 +1,162 @@
++++
+title = "Experimenting with Migrating from Fedora Workstation to Fedora Silverblue"
+author = ["Caleb Woodbine"]
+date = "2023-05-24"
+categories = ["linux", "os", "desktop", "fedora", "ostree", "oci"]
+draft = false
+summary = "Migrating Fedora from Workstation to Silverblue"
++++
+
+#+begin_quote
+Cross-posted from https://blog.calebwoodbine.com/experimenting-with-migrating-from-fedora-workstation-to-fedora-silverblue/
+#+end_quote
+
+* :newspaper: Introduction
+
+The desktop has always been an important decision point for my digital life.
+Where all my work gets done, my daily flows and the ecosystems I choose to adopt.
+
+Since somewhere around 2019, Fedora has been my desktop distro of choice.
+I ran Fedora Workstation back then and switched to Fedora Silverblue at the end of 2019.
+
+Whilst both are fine OSes; however with Silverblue in particular
+I found that the power of the reliability of an image based OS
+enabled me to focus on things like work instead of system maintenance.
+
+* :rocket: The objective
+
+#+begin_quote
+Migrate or replace an existing Fedora Workstation install to a Fedora Silverblue deployment.
+#+end_quote
+
+Important things are to
+- retain as much basic system config as possible
+- retain accounts and user data
+- make the process simple (considering all things)
+(more things?)
+
+Given it's a highly experimental thing, there will always be some level of expectation that some things will either be lost or not the same.
+
+* :two: A tale of two architectures
+
+The way these two OSes operate and are laid out is fairly different.
+
+[[https://fedoraproject.org/workstation/][Fedora Workstation]]:
+- system file system is read+write
+- dnf used for package updates
+- home folder in ~/home~
+- system package upgrades take place on reboot or manually on a live system
+
+[[https://fedoraproject.org/silverblue/][Fedora Silverblue]]:
+- system file system is read-only
+- the OS is state-managed through [[https://ostreedev.github.io/ostree/introduction/][OSTree]] under the hood
+- packages are managed through [[https://coreos.github.io/rpm-ostree][rpm-ostree]] and it's OSTree integration
+- home folder in ~/var/home~
+- system upgrades take place on reboot (experimental ~apply-live~ available. Read more here: https://coreos.github.io/rpm-ostree/apply-live)
+- file system layout (things of note)
+  - ~/home~ :: soft symlink to ~/var/home~
+  - ~/ostree~ :: for the OSTree repo and deploy
+  - ~/sysroot~ :: the path to the /real/ root filesystem
+  - ~/usr/local~ :: soft symlink to ~/var/usrlocal~ for per-machine programs and stuff
+  - ~/var~ :: all the system state, including ~home~ and ~roothome~
+
+* :computer: A thought-filled hack
+
+Please note that this process is potentially **harmful, irreversible and is a hack**.
+
+[[https://github.com/ublue-os/upgrade-tools][github.com/ublue-os/upgrade-tools]]
+
+Here are some lines taken out of the ~silverblueize.sh~ script.
+Please note that these steps are only to visualise the process
+and if you wish to be daring and migrate in this way to please
+refer to the script in the repo above.
+
+:one:. create an OSTree repo in the root filesystem
+#+begin_src shell
+ostree admin init-fs /
+#+end_src
+
+now you'll find a ~/ostree~ folder in the root filesystem
+
+:two:. add the Fedora remote (kind of like with git)
+#+begin_src shell
+ostree remote add \
+    fedora https://ostree.fedoraproject.org \
+    --set=gpgkeypath="/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-38-$(uname -m)" \
+    --set=contenturl=mirrorlist=https://ostree.fedoraproject.org/mirrorlist"
+#+end_src
+
+Fedora ships with the public keys for the various versions Fedora's repos.
+Now it's configured to know the Fedora OSTree remote.
+
+:three:. pull the OSTree branch into the repo
+#+begin_src shell
+ostree --repo=/ostree/repo pull "fedora:fedora/38/$(uname -m)/silverblue"
+#+end_src
+
+This one take a while, but will vary depending on internet connection.
+
+:four:. Ensure that the directories exist which are required for an OS deploy
+#+begin_src shell
+ostree admin os-init fedora
+#+end_src
+
+Prepares the folders in the root filesystem for OSTree.
+
+:five:. Make the branch be live on reboot
+#+begin_src shell
+ostree admin deploy --os=fedora --karg-proc-cmdline "fedora:fedora/38/$(uname -m)/silverblue"
+#+end_src
+
+Similar to ~git checkout~, now the version which was pull earlier is now ready to boot.
+
+:six:. Copy across a few important system configs
+#+begin_src shell
+for i in /etc/fstab /etc/default/grub /etc/locale.conf /etc/ostree/remotes.d/fedora.conf; do
+      cp "${i}" "${OSTREE_DEPLOY_ROOT}/${i}"
+done
+#+end_src
+
+may add some more things here later but this seems to make it work
+
+:seven:. Remap the mount home folder to the new expected location
+#+begin_src shell
+sed -i -e 's, /home , /var/home ,g' "${OSTREE_DEPLOY_ROOT}/etc/fstab"
+#+end_src
+
+no place like home!
+
+#+begin_quote
+and... that's pretty much it!
+#+end_quote
+
+After the script and a reboot, you'll be put into the emergency shell.
+Just press enter to continue. Not sure why it appears.
+
+Now you'll be prompted to create a user account. Go ahead with the
+same username as matching in the home folder.
+
+:sparkles: You made it to Silverblue! :sparkles:
+
+* :thinking: Things to consider
+
+- the previous install of Fedora Workstation is still kinda around and may be used in the boot process
+  - issue here: https://github.com/ublue-os/upgrade-tools/issues/7 (UPDATE: yes they are)
+-  reaching out to Fedora maintainers at Red Hat for interest, direction and roadmap considerations
+  - is it on the roadmap to migrate users from Fedora Workstation to Fedora Silverblue or future versions?
+
+There are some uncertainties
+- are there remaining Fedora Workstation files install on disk? (e.g /usr/bin)
+  - UPDATE: yes, those files may also be depended on too
+- why does it prompt for user account creation again?
+- non-standard/non-default partition configs
+
+* :door: That's all folks!
+
+Please reach out if this has helped you or if you have any ideas where to take this process!
+
+Inspired by the work of https://asamalik.fedorapeople.org/fedora-docs-translations/en-US/fedora-silverblue/installation-dual-boot
+
+Shout-out to Jorge Castro ([[https://github.com/castrojo]]) for interest in furthering the idea and providing a space in the ublue community!
+
+check out how the installer deploys Silverblue here https://github.com/rhinstaller/anaconda/blob/master/pyanaconda/modules/payloads/payload/rpm_ostree/installation.py

content/post/kubernetes-conformance-updates-for-october-2022.md

@@ -0,0 +1,146 @@
++++
+title = "Kubernetes conformance updates for October 2022"
+author = ["Caleb Woodbine", "Riaan Kleinhans", "Stephen Heywood"]
+date = "2022-10-19"
+categories = ["kubernetes", "k8s-conformance", "github"]
+draft = false
+summary = "Kubernetes conformance updates for October 2022"
++++
+
+> Cross-posted from https://www.cncf.io/blog/2022/10/19/kubernetes-conformance-updates-for-october-2022/
+
+# Introduction
+
+The Kubernetes Conformance project has been running for a number of years now, recently there are a few minor updates to it and a few things to celebrate.
+
+# Coverage
+
+Looking back at the history of Kubernetes Conformance we need to go back to February 2018 when [Kenichi Omichi](https://github.com/oomichi) dug into the Kubernetes logs and found that Kubernetes had 481 API endpoints, only 53 endpoints, or 11% were covered by tests. Dan Kohn knew that for the [Certified Kubernetes brand](https://github.com/cncf/k8s-conformance/pulls#certified-kubernetes) to have meaning, they needed to invest to make sure test coverage was much higher.
+
+To address the gap in conformance testing Kubernetes Conformance subproject was established and met for the first time on [26 March 2019](https://www.youtube.com/watch?v=uqA1JtRtLXs&list=PL69nYSiGNLP2m6198LaLN6YahX7EEac5g&index=192). The aim for the subproject was to accelerate work on improving conformance test coverage, perform conformance test reviews, develop processes to support the work, prioritize work and discuss test criteria.
+
+The diligent work of test writers and reviewers has paid off, with less than 6% of the endpoints not covered by Conformance tests when 1.25 was released. Also, in that release we finally have 100% coverage for all app endpoints.
+
+Not only was it important to ensure that existing technical debt is being paid off through the delivery of quality conformance tests, but also ensuring that no new technical debt is created through API endpoints graduating to GA without Conformance tests. This is achieved through weekly updates of [APISnoop](https://apisnoop.cncf.io) and careful monitoring of the results.
+
+# Community Operations
+
+Automation is a fundamental value of the Kubernetes community.
+For each release, Kubernetes product and distribution owners submit a PR to the k8s-conformance repo in order to certify their product or distribution.
+
+Since around the time of the release of Kubernetes v1.18, a bot was built by Berno Kleinhans and Rob Kielty to automate the checking of the PRs in this repo.
+In order to verify conformance, the Kubernetes e2e conformance suite must be run against a cluster.
+
+This is where [sonobuoy](https://github.com/vmware-tanzu/sonobuoy), a community tool from [vmware-tanzu](https://tanzu.vmware.com/) for generating conformance test run data, comes in. When sonobuoy is run, it runs the test suite against a target cluster. Once the suite is complete the e2e.log and junit_01.xml can be collected. These files describe what was run and how it ran.
+
+Each PR has the four following files
+
+- a PRODUCT.yaml; to provide metadata on the product
+- an e2e.log: to store the test runner’s logs
+- a junit_01.xml: to store a structured list of the tests that were run; and finally
+- a README.md: to provide instructions on how the submitter created the cluster and generated results
+
+# Test definitions
+
+I was tasked with the rewrite of the bot. In this rewrite, the behaviour of the bot has been greatly improved as well as the tests made easier to understand.
+[Gherkin](https://cucumber.io/docs/gherkin/) is a way to plainly describe features for specifications and implement the code separately. Much earlier before Gherkin was implemented in the Kubernetes conformance bot, Zach Mandeville (also from [ii.nz](https://ii.nz)) had implemented Gherkin in the [XDS test suite](https://github.com/ii/xds-test-harness) through [godog](https://github.com/cucumber/godog). With this inspiration and a keener focus on understandability and maintainability, Gherkin was naturally chosen.
+
+Here is a snippet from the bot’s feature file, to provide very high-level language instructions for the tests for the submissions through the bot:
+
+```feature
+  Scenario: all tests pass in e2e.log
+    it appears that some tests failed in the product submission
+
+    Given an "e2e.log" file
+    Then the tests pass and are successful
+    And all required tests in e2e.log are present
+```
+
+The scenario above regards that the expected state of all tests pass in the submitted e2e.log file. The second line is the failure message.
+The test begins with ensuring that the file e2e.log exists in the PR, it states in an intended way for the outcome that all tests are passing and are successful, finally that all required tests are present.
+In Gherkin, there are directives like “Given”, “Then” and “And”. These directives begin declaring lines of the test. After each directive, there’s a statement like “the tests pass and are successful”; Each statement maps to some function in code through an implementation of the feature file, in this case Go functions with Godog.
+For instance, the line “Given an “e2e.log” file” matches to a handling function through this regex here
+
+```go
+ctx.Step(`^a[n]? "([^"]*)" file$`, s.aFile)
+```
+
+then the function that handles the test, checking for a file’s existence
+
+```go
+func (s *PRSuite) aFile(fileName string) error {
+	file := s.GetFileByFileName(fileName)
+	if file == nil {
+		return common.SafeError(fmt.Errorf("missing required file '%v'", fileName))
+	}
+	return nil
+}
+```
+
+You can find out more about Gherkin, [here](https://cucumber.io/docs/gherkin/) and Godog (the Go implementation) [here](https://github.com/cucumber/godog).
+
+There are currently 15 scenarios, covering requirements like
+
+- required tests present and passing
+- title format
+- folder structure
+- file presence
+- values in PRODUCT.yaml
+- URL resolution for URLs in PRODUCT.yaml
+- amount of commits
+
+and many more.
+# Behaviour
+
+The bot will now respond with a single message after each difference on a commit, as well as updating the labels on the PR. Under-the-hood, the bot is now declarative.
+Additionally, the bot also updates the status checks at the end of bottom of the PR page.
+A successful product submission will result in the following message,
+
+<figure>
+<img alt='Screenshot shwoing prow-cncf-io comment' src='/images/2021/k8s-conformance-oct21-checks-passing.png' />
+</figure>
+
+the following checks passing,
+
+<figure>
+<img alt='Screenshot showing checks passing' src='/images/2021/k8s-conformance-oct21-gh-status-checks.png' />
+</figure>
+
+and the similar labels
+
+<figure>
+<img alt='Screenshot showing labels' src='/images/2021/k8s-conformance-oct21-gh-labels.png' />
+</figure>
+
+A failing product submission result in a comment something like this,
+
+<figure>
+<img alt='Screenshot showing prow-cncf-io commented 13 of 15 requirements have passed, 2 fails items need to be reviewed' src='/images/2021/k8s-conformance-oct21-checks-failing.png' />
+</figure>
+
+the following status checks failing
+
+<figure>
+<img alt='Screenshot showing some checks were not successful (1 filing and 1 pending checks)' src='/images/2021/k8s-conformance-oct21-gh-status-checks-failing.png' />
+</figure>
+
+and labels similar to this
+
+<figure>
+<img alt='Screenshot showing labels' src='/images/2021/k8s-conformance-oct21-gh-labels-failing.png' />
+</figure>
+
+
+
+These changes have been shown to be incredibly helpful for product submitters, easing the process of submitting and determining potential issues.
+
+# Closing
+
+Overall, the Kubernetes conformance has come along way in the past four years.
+The project has almost reached completion and has received some smoother automations and feedback.
+
+You can find out more about the project here
+
+- apisnoop: https://apisnoop.cncf.io
+- k8s-conformance repo: https://github.com/cncf/k8s-conformance
+- verify-conformance bot repo: https://github.com/cncf-infra/verify-conformance