feat: reproducible choice interactive (#262)

this will look now like the following ```bash ❯ : cargo near build non-reproducible-wasm --help Runs on current filesystem state without many restrictions Usage: cargo-near near build non-reproducible-wasm [OPTIONS] Options: --locked --no-release --no-abi --no-embed-abi --no-doc --no-wasmopt --out-dir <OUT_DIR> --manifest-path <MANIFEST_PATH> --features <FEATURES> --no-default-features --color <COLOR> [possible values: auto, always, never] --env <ENV> -h, --help Print help -V, --version Print version ``` ```bash ❯ : cargo near build reproducible-wasm --help Requires `docker` config added and (git)committed to Cargo.toml, runs on clean (git)working tree state Usage: cargo-near near build reproducible-wasm [OPTIONS] Options: --no-locked --out-dir <OUT_DIR> --manifest-path <MANIFEST_PATH> --color <COLOR> [possible values: auto, always, never] -h, --help Print help -V, --version Print version ``` --- this is pending - [x] deploy cmd dispatch in current pr - [x] logic to enforce correct `container_build_command` for newer images (ones, that will be built after release of this pr) - [x] prettier indents in interactive help msgs in current pr - [x] separate pr onto current pr as base branch with tests fixed as per api change (#263) - [x] testing of this pr on a few common cases (sample-crate, factory-rust, neardevhub-contract) (in comment #262 (comment)) - [x] separate pr onto current pr as base branch with `.github/workflows` change /testing (#264) - [x] separate pr onto current pr as base branch with docs/README update (#265) - [ ] merge --------- Co-authored-by: dj8yf0μl <[email protected]>
near · Dec 17, 2024 · a63fb95 · a63fb95
1 parent 6a39fbf
commit a63fb95
Show file tree

Hide file tree

Showing 30 changed files with 966 additions and 688 deletions.
diff --git a/.github/workflows/crontab_new_template_renewal.yml b/.github/workflows/crontab_new_template_renewal.yml
@@ -30,7 +30,7 @@ jobs:
           let record = http get "https://hub.docker.com/v2/namespaces/sourcescan/repositories/cargo-near/tags" | get results | first;
 
           let mod_content = (
-            open cargo-near/src/commands/new/new-project-template/Cargo.toml.template --raw | lines
+            open cargo-near/src/commands/new/new-project-template/Cargo.template.toml --raw | lines
             | each { 
               |line| if ($line | str starts-with "image = ") { 
                 $'image = "sourcescan/cargo-near:($record.name)"' 
@@ -44,7 +44,7 @@ jobs:
             | to text
           );
 
-          $mod_content | save -f cargo-near/src/commands/new/new-project-template/Cargo.toml.template
+          $mod_content | save -f cargo-near/src/commands/new/new-project-template/Cargo.template.toml
 
           git diff
 

diff --git a/Cargo.lock b/Cargo.lock
diff --git a/README.md b/README.md
@@ -99,26 +99,57 @@ cargo near build
 
 Builds a NEAR smart contract along with its [ABI](https://github.com/near/abi) (while in the directory containing contract's Cargo.toml).
 
-By default, this runs a reproducible build in a [Docker](https://docs.docker.com/) container, which:
+Running the above command opens a menu with following variants: 
+
+### `non-reproducible-wasm`
+
+**Recommended variant for use during local development.**
+
+This is a regular build, which behaves much like and is a thin wrapper around a regular `cargo build --target wasm32-unknown-unknown --release`.
+
+Additional flags for build configuration can be looked up by
+
+```bash
+cargo near build non-reproducible-wasm --help
+```
+if needed.
+
+### `reproducible-wasm`
+
+**Recommended variant for the production releases.**
+
+This variant runs a reproducible build in a [Docker](https://docs.docker.com/) container, which:
 
 1. runs against source code version, committed to git, ignoring any uncommitted changes
 2. requires that `Cargo.lock` of project is created (e.g. via `cargo update`) and added to git. 
     - this enables `--locked` build by downstream `cargo` command. 
-3. will use configuration in [`[package.metadata.near.reproducible_build]`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.toml.template#L14-L25) 
-   section of contract's `Cargo.toml` and [`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.toml.template#L9) field
+3. will use configuration in [`[package.metadata.near.reproducible_build]`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L14-L25) 
+   section of contract's `Cargo.toml` and [`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L9) field
     - default values for this section can also be found in `Cargo.toml` of 
       template project, generated by `cargo near new`
 
-Important flags:
+**What's a reproducible build in context of NEAR?**
+Why is it needed? Explanation of these points and a step-by-step tutorial is present at [SourceScan/verification-guide](https://github.com/SourceScan/verification-guide).
 
-1. `--no-docker`
-    - flag can be used to perform a regular build with rust toolchain installed onto host, running the `cargo-near` cli. 
-    - *NO*-Docker builds run against actual state of code in filesystem and not against a version, committed to source control.   
-
-2. `--no-locked` 
-    - flag is allowed in *NO*-Docker builds, e.g. to generate a `Cargo.lock` *and* simultaneously build the contract.
-    - flag is allowed in Docker builds, but 
-      - such builds are not reproducible due to potential update of dependencies and compiled `wasm` mismatch as the result.
+<details>
+  <summary>Additional (optional) details on possible <code>[package.metadata.near.reproducible_build]</code> configuration</summary><p>
+
+1. available images can be found by this link https://hub.docker.com/r/sourcescan/cargo-near/tags 
+    - [`image`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L18) and [`image_digest`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L19) are straightforward to configure:
+  ![image_and_digest_pinpoint](./docs/image_and_digest_pinpoint.png)
+2. flags of build command, run inside of docker container, can be configured, if needed, by changing [`container_build_command`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L29) field
+      - base `container_build_command` for images starting with **sourcescan/cargo-near:0.13.0-rust-1.83.0** and after it  is `["cargo", "near", "build", "non-reproducible-wasm", "--locked"]`, where the `--locked` flag is required
+      - base `container_build_command` for images prior to **sourcescan/cargo-near:0.13.0-rust-1.83.0** is `["cargo", "near", "build"]` 
+      - additional flags, if needed, can be looked up on
+          - `cargo near build non-reproducible-wasm --help` for newer/latest images 
+          - `cargo near build --help` for older ones 
+          - running `docker run -it sourcescan/cargo-near:0.11.0-rust-1.82.0` (or another specific image) and checking the `--help` message of exact `cargo-near` in container may be helpful when in doubt    
+3. `cargo near` allows parameterizing build with values of environment variables, present at the time of the build and not present in a contract's source code,
+   by specifying their names in [`passed_env`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L24) array
+    - supported by **sourcescan/cargo-near:0.10.1-rust-1.82.0** image or later images
+    - SourceScan/Nearblocks does not support verifying such contracts with additional parameters present in their metadata yet 
+
+</p></details>
 
 ---
 
@@ -146,22 +177,22 @@ cargo near deploy
 
 Builds the smart contract (equivalent to `cargo near build`) and guides you to deploy it to the blockchain.
 
-By default, this runs a reproducible build in a Docker container. 
+Similar to `build`, running the above command opens a menu with following variants: 
+
+### `build-non-reproducible-wasm`
+
+This forwards to [non-reproducible-wasm](#non-reproducible-wasm) variant of `build` command.
+
+### `build-reproducible-wasm`
+
+This forwards to [reproducible-wasm](#reproducible-wasm) variant of `build` command.
 
 `deploy` command from Docker build requires that contract's source code:
 
 1. doesn't have any modified tracked files, any staged changes or any untracked content.  
 2. has been pushed to remote repository, identified by 
-   [`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.toml.template#L9).
-
-Important flags:
-
-1. `--no-docker`
-    - flag can be used to perform a regular *NO*-Docker build *and* deploy. 
-      - Similar to `build` command,  in this case none of the git-related concerns and restrictions apply.
+   [`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L9).
 
-2. `--no-locked` 
-    - flag is declined for deploy, due to its effects on `build` result
 
 ## Contribution
 

diff --git a/cargo-near-build/src/near/docker_build/docker_checks/mod.rs b/cargo-near-build/src/near/docker_build/docker_checks/mod.rs
@@ -19,7 +19,6 @@ pub fn handle_command_io_error<T>(
             println!();
             println!("{}", "`docker` executable isn't available".yellow());
             print_installation_links();
-            print_non_docker_suggestion();
             Err(report)
         }
         Err(io_err) => {
@@ -33,7 +32,6 @@ pub fn handle_command_io_error<T>(
                 .yellow()
             );
             println!("{}", format!("Error `{:?}`", io_err).yellow());
-            print_non_docker_suggestion();
             Err(report)
         }
     }
@@ -127,13 +125,4 @@ pub fn print_command_status(status: std::process::ExitStatus, command: std::proc
         )
         .yellow()
     );
-    print_non_docker_suggestion();
-}
-
-fn print_non_docker_suggestion() {
-    println!(
-        "{}",
-        "You can choose to opt out into non-docker build behaviour by using `--no-docker` flag."
-            .cyan()
-    );
 }
diff --git a/cargo-near-build/src/near/docker_build/mod.rs b/cargo-near-build/src/near/docker_build/mod.rs
@@ -1,5 +1,7 @@
 use std::process::{Command, ExitStatus};
 
+use colored::Colorize;
+
 use crate::docker::DockerBuildOpts;
 use crate::types::near::build::input::BuildContext;
 use crate::types::near::build::output::CompilationArtifact;
@@ -33,10 +35,13 @@ pub fn run(opts: DockerBuildOpts) -> eyre::Result<CompilationArtifact> {
         },
     )?;
 
-    let docker_build_meta =
-        pretty_print::handle_step("Parsing and validating `Cargo.toml` metadata...", || {
-            metadata::ReproducibleBuild::parse(cloned_repo.crate_metadata())
-        })?;
+    let docker_build_meta = pretty_print::handle_step(
+        &format!(
+            "Parsing and validating `{}` section of contract's `Cargo.toml` ...",
+            "[package.metadata.near.reproducible_build]".magenta()
+        ),
+        || metadata::ReproducibleBuild::parse(cloned_repo.crate_metadata()),
+    )?;
 
     if let BuildContext::Deploy {
         skip_git_remote_check,

diff --git a/cargo-near-build/src/types/cargo/metadata.rs b/cargo-near-build/src/types/cargo/metadata.rs
@@ -170,7 +170,9 @@ fn get_cargo_metadata(
             );
             println!(
                 "{}",
-                "You can choose to disable `--locked` flag for downstream `cargo` command with `--no-locked` flag.".cyan()
+                "You can choose to disable `--locked` flag for downstream `cargo` command \
+                by adding `--no-locked` flag OR by removing `--locked` flag"
+                    .cyan()
             );
             thread::sleep(Duration::new(5, 0));
             return Err(cargo_metadata::Error::CargoMetadata {

diff --git a/cargo-near-build/src/types/near/build/input/mod.rs b/cargo-near-build/src/types/near/build/input/mod.rs
@@ -88,7 +88,12 @@ impl Default for CliDescription {
     fn default() -> Self {
         Self {
             cli_name_abi: "cargo-near".into(),
-            cli_command_prefix: vec!["cargo".into(), "near".into(), "build".into()],
+            cli_command_prefix: vec![
+                "cargo".into(),
+                "near".into(),
+                "build".into(),
+                "non-reproducible-wasm".into(),
+            ],
         }
     }
 }
@@ -97,13 +102,16 @@ impl Opts {
     /// this is just 1-to-1 mapping of each struct's field to a cli flag
     /// in order of fields, as specified in struct's definition.
     /// `Default` implementation corresponds to plain `cargo near build` command without any args
-    pub(crate) fn get_cli_command_for_lib_context(&self) -> Vec<String> {
+    pub fn get_cli_command_for_lib_context(&self) -> Vec<String> {
         let cargo_args = self.cli_description.cli_command_prefix.clone();
         let mut cargo_args: Vec<&str> = cargo_args.iter().map(|ele| ele.as_str()).collect();
-        if self.no_locked {
-            cargo_args.push("--no-locked");
+        // this logical NOT is needed to avoid writing manually `Default` trait impl for `Opts`
+        // with `self.locked` field and to keep default (if nothing is specified) to *locked* behavior
+        // which is a desired default for [crate::extended::build] functionality
+        if !self.no_locked {
+            cargo_args.push("--locked");
         }
-        // `no_docker` field isn't present
+
         if self.no_release {
             cargo_args.push("--no-release");
         }
@@ -220,6 +228,8 @@ mod tests {
         assert_eq!(opts.get_cli_command_for_lib_context(), ["cargo".to_string(),
              "near".to_string(),
              "build".to_string(),
+             "non-reproducible-wasm".to_string(),
+             "--locked".to_string(),
              "--env".to_string(),
              "KEY=VALUE".to_string(),
              "--env".to_string(),