Add forc-migrate tool

[ironcev]

Description

This PR introduces forc-migrate, a Forc tool for migrating Sway projects to the next breaking change version of Sway.

The tool addresses two points crucial for code updates caused by breaking changes:

• it informs developers about the breaking changes and assists in planing and executing the migration.
• it automatically changes source code where possible, reducing the manual effort needed for code changes.

Besides adding the forc-migrate tool, the PR:

• extends Diagnostic to support migration diagnostics aside with errors and warnings.
• changes swayfmt to support generating source code from arbitrary lexed trees. The change is a minimal one done only in the parts of swayfmt that are executed by migration steps written in this PR. Adapting swayfmt to fully support arbitrary lexed trees will be done in Refactor swayfmt to generate code from arbitrary lexed trees, not necessarily backed by source code #6779.

The migration for the references feature, migrating ref mut to &mut, is developed only partially, to demonstrate the development and usage of automatic migrations that alter the original source code.

The intended usage of the tool is documented in detail in the "forc migrate" chapter of The Sway Book: Forc reference > Plugins > forc_migrate. (The generated documentation has issues that are caused by the documentation generation bug explained in #6792. These issues will be fixed in a separate PR that will fix it for all the plugins.)

We expect the forc-migrate to evolve based on the developer's feedback. Some of the possible extensions of the tool are:

• adding additional CLI options, e.g., for executing only specific migration steps, or ignoring them.
• passing parameters to migration steps from the CLI.
• not allowing updates by default, if the repository contains modified or untracked files.
• migrating workspaces.
• migrating other artifacts, e.g., Forc.toml files or contract IDs.
• migrating between arbitrary versions of Sway.
• migrating SDK code.
• etc.

forc-migrate also showed a clear need for better infrastructure for writing static analyzers and transforming Sway code. The approach used in the implementation of this PR should be seen as a pragmatic beginning, based on the reuse of what we currently have. Some future options are discussed in #6836.

Demo

forc migrate show

Shows the breaking change features and related migration steps. This command can be run anywhere and does not require a Sway project.

Breaking change features:
  - storage_domains    (https://github.com/FuelLabs/sway/issues/6701)
  - references         (https://github.com/FuelLabs/sway/issues/5063)

Migration steps (1 manual and 1 semiautomatic):
storage_domains
  [M] Review explicitly defined slot keys in storage declarations (`in` keywords)

references
  [S] Replace `ref mut` function parameters with `&mut`

Experimental feature flags:
- for Forc.toml:  experimental = { storage_domains = true, references = true }
- for CLI:        --experimental storage_domains,references

forc migrate check

Performs a dry-run of the migration on a concrete Sway project. It outputs all the occurrences in code that need to be reviewed or changed, as well as the migration time effort:

info: [storage_domains] Review explicitly defined slot keys in storage declarations (`in` keywords)
  --> /home/kebradalaonda/Desktop/M Forc migrate tool/src/main.sw:19:10
   |
...
19 |     y in b256::zero(): u64 = 0,
   |          ------------
20 |     z: u64 = 0,
21 |     a in calculate_slot_address(): u64 = 0,
   |          ------------------------
22 |     b in 0x0102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20: u64 = 0,
   |          ------------------------------------------------------------------
   |
   = help: If the slot keys used in `in` keywords represent keys generated for `storage` fields
   = help: by the Sway compiler, those keys might need to be recalculated.
   = help:  
   = help: The previous formula for calculating storage field keys was: `sha256("storage.<field name>")`.
   = help: The new formula is:                                          `sha256((0u8, "storage.<field name>"))`.
   = help:  
   = help: For a detailed migration guide see: https://github.com/FuelLabs/sway/issues/6701
____

Migration effort:

storage_domains
  [M] Review explicitly defined slot keys in storage declarations (`in` keywords)
      Occurrences:     3    Migration effort (hh::mm): ~00:06

references
  [S] Replace `ref mut` function parameters with `&mut`
      Occurrences:     0    Migration effort (hh::mm): ~00:00

Total migration effort (hh::mm): ~00:06

forc migrate run

Runs the migration steps and guides developers through the migration process.

Checklist

• I have linked to any relevant issues.
• I have commented my code, particularly in hard-to-understand areas.
• I have updated the documentation where relevant (API docs, the reference, and the Sway book).
  • If my change requires substantial documentation changes, I have requested support from the DevRel team
• I have added tests that prove my fix is effective or that my feature works.
• I have added (or requested a maintainer to add) the necessary Breaking* or New Feature labels where relevant.
• I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
• I have requested a review from the relevant team or maintainers.

[ironcev]

To try out the forc migrate tool locally:

1. Pull and checkout the ironcev/forc-migrate-tool branch.
2. Install the tool from the Sway repository: cargo install --path ./forc-plugins/forc-migrate

For the detailed documentation on migration and the usage of the tool see the scripts/mdbook-forc-documenter/examples/forc_migrate.md in this PR.

[IGI-111]

The tool seems fine, but the migration for #6701 is inverted: users won't need to migrate slots that specify their position with the in keyword, however any slot that doesn't specify their position will have to be set to a fixed, old, position to guarantee backwards compatibility.

The lifecycle of migrations is (primarily) for existing codebases that are deployed behind proxies and need to maintain storage compatibility.

The guide in that linked issue should be fixed to that effect too.

[tritao]

Overall looks great, nice work, very nicely commented and engineered.

Left just some minor pedantic nits (feel free to ignore!).

I do have a question regarding the overall design.

Given the code to track the manual effort duration for code changes and manual transformation suggestions, I take it (at least for now) the tool will work in a semi-automatic, as opposed to fully automatic?

[ironcev]

The lifecycle of migrations is (primarily) for existing codebases that are deployed behind proxies and need to maintain storage compatibility.

@IGI-111 Completely agree. I gave it a deeper thought and have a design that will cover both the rare case which is supported now and also contracts behind proxies. The two will play well together and also with the SRC-14 (no suggestions to check in keywords for SRC-14 fields and fileds that have in keywords for compatibility, etc.).

I am moving the PR to draft until that is implemented.

[ironcev]

Given the code to track the manual effort duration for code changes and manual transformation suggestions, I take it (at least for now) the tool will work in a semi-automatic, as opposed to fully automatic?

@tritao It depends on the concrete breaking changes and also on the effort we want to put into writing migrations, compared to eventual manual effort. E.g., in the sample case of changing ref mut to &mut the migration can find and change usages etc. and make the step fully automatic.

But in a completely general case, I don't see a fully automatic migration as always possible, even theoretically. Also, the sensitivity of any migration always requires manual supervision. That's why I've opted for the proposed design in which the tool deliberately stops after a single feature is fully migrated and asks for a code review, even if all the migration steps were fully automated.

Having manual supervision and semi-automatic migrations also corresponds to my experience so far with code migrations. I see the goal of the tool in taking over the mundane and repetitive effort and pointing where to look, but programmers should still be aware of the changes and understand their impact.

[codspeed-hq]

CodSpeed Performance Report

Merging #6790 will not alter performance

_{Comparing ironcev/forc-migrate-tool (d7e94a2) with master (3f7b5f1)}

Summary

✅ 22 untouched benchmarks

[ironcev]

@IGI-111 @tritao @JoshuaBatty Thanks for reviewing the forc migrate tool. Can I ask you for a quick re-review of the latest changes? I need this PR merged to finalize the next one which brings additional migrations.

The latest changes address your remarks and also add some ~1000 LOC to the anyhow big PR. Still, it should be fairly easy to review.

The majority of the changes goes to infrastructure for matching and modifying trees. I hoped to get the first version of the forc migrate without necessarily developing such infrastructure. But it turned out that the LOC ratio between the code that does manual matching and tree modifications and the actual migration logic went to almost 3:1! This made the migrations error-prone and difficult to maintain. I expect in the future more improvements in this area as already discussed between @tritao and me (see: #6836). But for the time being, based on lot of concrete examples, I see the simple infrastructure provided in this PR as more than sufficient for writing readable and easy-to-write migrations.