Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add function for generating bytecode identifier #6674

Merged
merged 69 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
5974362
Add 32 bytes of 0'd metadata in the preamble
vaivaswatha Sep 10, 2024
6f5c1b2
fix clippy
vaivaswatha Sep 10, 2024
1b99c89
update patch_test_bytecode for forc test
vaivaswatha Sep 11, 2024
d0c8236
allow setting metadata from forc-pkg
vaivaswatha Sep 11, 2024
7a80a14
add hash of the ABI as metadata
vaivaswatha Sep 12, 2024
fa836da
revert inadvertent change
vaivaswatha Sep 12, 2024
2fa3154
fix typos
vaivaswatha Sep 12, 2024
1a2f85e
fix bug in computing hash
vaivaswatha Sep 12, 2024
4e3a286
update contract IDs
vaivaswatha Sep 12, 2024
e1944a9
update configurable_consts abi json
vaivaswatha Sep 12, 2024
ded969b
update tests
vaivaswatha Sep 12, 2024
bf378c6
Merge branch 'master' into vaivaswatha/metadata
vaivaswatha Sep 12, 2024
7405df7
fix asset_ops_test
vaivaswatha Sep 12, 2024
798e03c
set first word of metadata to offset to configurables in data-section
vaivaswatha Sep 26, 2024
dd7a91d
update deployment tests
vaivaswatha Sep 26, 2024
49e42e6
typo bugfix
vaivaswatha Sep 26, 2024
4c28e64
change how we handle loading of non-copy types so that data_section i…
vaivaswatha Sep 30, 2024
075e699
cargo fmt
vaivaswatha Sep 30, 2024
f435e99
Merge remote-tracking branch 'origin/master' into vaivaswatha/metadata
vaivaswatha Sep 30, 2024
cc9b8a3
Add padding for bytecode word alignment before finalising data-section
vaivaswatha Oct 1, 2024
aeaf12c
update tests
vaivaswatha Oct 1, 2024
ddea001
update configurable_consts json
vaivaswatha Oct 1, 2024
f923af0
some fixes
vaivaswatha Oct 2, 2024
5cae76d
more fixes
vaivaswatha Oct 2, 2024
7cc793c
more test updates
vaivaswatha Oct 2, 2024
2dd7601
more test fixes
vaivaswatha Oct 3, 2024
52c06eb
update predicate root for test
vaivaswatha Oct 3, 2024
8cf03e5
Merge branch 'master' into vaivaswatha/metadata
vaivaswatha Oct 3, 2024
d99bc98
update forc test
vaivaswatha Oct 3, 2024
c029db3
review comments
vaivaswatha Oct 9, 2024
657e8fa
Merge branch 'master' into vaivaswatha/metadata
vaivaswatha Oct 9, 2024
1c87641
update test
vaivaswatha Oct 9, 2024
dc0c629
Merge branch 'master' into vaivaswatha/metadata
vaivaswatha Oct 16, 2024
32d83a1
Merge branch 'master' into vaivaswatha/metadata
sdankel Oct 21, 2024
fd2b8a3
Merge branch 'master' into vaivaswatha/metadata
vaivaswatha Oct 22, 2024
f644401
Change Metadata to Configurables offset. 256b -> 64b
vaivaswatha Oct 23, 2024
0dadcba
Merge branch 'vaivaswatha/metadata' of github.com:FuelLabs/sway into …
vaivaswatha Oct 23, 2024
7520959
Merge remote-tracking branch 'origin/master' into vaivaswatha/metadata
vaivaswatha Oct 23, 2024
566560d
fix bug in computing half_word_idx
vaivaswatha Oct 23, 2024
90a1b91
more fixes
vaivaswatha Oct 23, 2024
b8d4e96
some test updates
vaivaswatha Oct 23, 2024
67842f3
test updates
vaivaswatha Oct 23, 2024
9e99d74
more test updates
vaivaswatha Oct 23, 2024
f2fdaa1
update code size for configurable_dedup_decode
vaivaswatha Oct 23, 2024
b91bea7
Merge branch 'master' into vaivaswatha/metadata
vaivaswatha Oct 23, 2024
f281fc2
Merge branch 'master' into vaivaswatha/metadata
sdankel Oct 24, 2024
e185ef2
incorporate review comments. uniformly rename metadata
vaivaswatha Oct 24, 2024
326fac8
Merge branch 'vaivaswatha/metadata' of github.com:FuelLabs/sway into …
vaivaswatha Oct 24, 2024
e9f1d5e
another minor fix
vaivaswatha Oct 24, 2024
6c967b2
feat: add function for generating bytecode identifier
sdankel Oct 25, 2024
fdf3a64
Merge remote-tracking branch 'origin/master' into vaivaswatha/metadata
vaivaswatha Oct 28, 2024
d7f7a10
Merge branch 'master' into vaivaswatha/metadata
IGI-111 Oct 28, 2024
b5df275
Merge branch 'master' into vaivaswatha/metadata
IGI-111 Oct 28, 2024
0d54707
Merge branch 'vaivaswatha/metadata' into sophie/bytecode-id
sdankel Oct 28, 2024
1966323
Merge branch 'master' into sophie/bytecode-id
sdankel Oct 29, 2024
cf39072
clippy
sdankel Oct 29, 2024
dfecee8
Update forc-util/src/bytecode.rs
sdankel Oct 30, 2024
73f638b
Fixes find_method_for_type bail out with 1st match (#6606)
esdrubal Oct 30, 2024
92cad25
Bump to v0.66.3 (#6679)
IGI-111 Oct 30, 2024
fbb94d5
fix comment
sdankel Oct 30, 2024
d2ce3b1
Merge branch 'master' into sophie/bytecode-id
sdankel Oct 30, 2024
87257e9
feedback
sdankel Nov 3, 2024
d7e4868
cargo fmt
sdankel Nov 3, 2024
12efaa2
add constant for preamble size
sdankel Nov 3, 2024
048466b
add constant for preamble size
sdankel Nov 3, 2024
b9c80bf
InstructionWithBytesIterator
sdankel Nov 4, 2024
99ecf1c
Merge branch 'master' into sophie/bytecode-id
sdankel Nov 18, 2024
f337fbe
Merge branch 'master' into sophie/bytecode-id
sdankel Nov 19, 2024
5f3bda3
Merge branch 'master' into sophie/bytecode-id
sdankel Nov 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions forc-util/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,15 @@ clap = { workspace = true, features = ["cargo", "derive", "env"] }
dirs.workspace = true
fd-lock.workspace = true
forc-tracing.workspace = true
fuel-asm.workspace = true
fuel-tx = { workspace = true, optional = true }
hex.workspace = true
paste.workspace = true
regex.workspace = true
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
serial_test.workspace = true
sha2.workspace = true
sway-core.workspace = true
sway-error.workspace = true
sway-types.workspace = true
Expand Down
170 changes: 170 additions & 0 deletions forc-util/src/bytecode.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
use anyhow::anyhow;
use sha2::{Digest, Sha256};
use std::fs::File;
use std::io::{BufReader, Read};
use std::path::Path;

// The index of the beginning of the half-word (4 bytes) that contains the configurables section offset.
const CONFIGURABLES_OFFSET_INSTR_LO: usize = 4;
// The index of the end of the half-word (4 bytes) that contains the configurables section offset.
const CONFIGURABLES_OFFSET_INSTR_HI: usize = 5;
// The count of the beginning half-words that contain the configurables section offset.
const CONFIGURABLES_OFFSET_PREAMBLE: usize = CONFIGURABLES_OFFSET_INSTR_HI + 1;

/// A tuple of an instruction and its corresponding bytes. Useful when needing to access the raw bytes
/// of an instruction that is parsed as [fuel_asm::InvalidOpcode], such as metadata in the preamble.
pub type InstructionWithBytes = (
Result<fuel_asm::Instruction, fuel_asm::InvalidOpcode>,
Vec<u8>,
);

/// An iterator over each [fuel_asm::Instruction] or [fuel_asm::InvalidOpcode] with its corresponding bytes.
pub struct InstructionWithBytesIterator {
buf_reader: BufReader<File>,
}

impl InstructionWithBytesIterator {
/// Return a new iterator for each instruction parsed from raw bytes.
pub fn new(buf_reader: BufReader<File>) -> Self {
InstructionWithBytesIterator { buf_reader }
}
}

impl Iterator for InstructionWithBytesIterator {
type Item = InstructionWithBytes;

fn next(&mut self) -> Option<InstructionWithBytes> {
let mut buffer = [0; fuel_asm::Instruction::SIZE];
// Read the next instruction into the buffer
match self.buf_reader.read_exact(&mut buffer) {
Ok(_) => fuel_asm::from_bytes(buffer)
.next()
.map(|inst| (inst, buffer.to_vec())),
Err(_) => None,
}
}
}

/// Parses a bytecode file into an iterator of instructions and their corresponding bytes.
pub fn parse_bytecode_to_instructions<P>(path: P) -> anyhow::Result<InstructionWithBytesIterator>
where
P: AsRef<Path> + Clone,
{
let f = File::open(path.clone())
.map_err(|_| anyhow!("{}: file not found", path.as_ref().to_string_lossy()))?;
let buf_reader = BufReader::new(f);

Ok(InstructionWithBytesIterator::new(buf_reader))
}

/// Gets the bytecode ID from a bytecode file. The bytecode ID is the hash of the bytecode after removing the
/// condigurables section, if any.
pub fn get_bytecode_id<P>(path: P) -> anyhow::Result<String>
where
P: AsRef<Path> + Clone,
{
let mut instructions = parse_bytecode_to_instructions(path.clone())?;

sdankel marked this conversation as resolved.
Show resolved Hide resolved
// Collect the first six instructions into a temporary vector
let mut first_six_instructions = Vec::with_capacity(CONFIGURABLES_OFFSET_PREAMBLE);
for _ in 0..CONFIGURABLES_OFFSET_PREAMBLE {
if let Some(instruction) = instructions.next() {
first_six_instructions.push(instruction);
} else {
return Err(anyhow!("Incomplete bytecode"));
}
}
alfiedotwtf marked this conversation as resolved.
Show resolved Hide resolved

let (lo_instr, low_raw) = &first_six_instructions[CONFIGURABLES_OFFSET_INSTR_LO];
let (hi_instr, hi_raw) = &first_six_instructions[CONFIGURABLES_OFFSET_INSTR_HI];

if let Err(fuel_asm::InvalidOpcode) = lo_instr {
if let Err(fuel_asm::InvalidOpcode) = hi_instr {
// Now assemble the configurables offset.
let configurables_offset = usize::from_be_bytes([
low_raw[0], low_raw[1], low_raw[2], low_raw[3], hi_raw[0], hi_raw[1], hi_raw[2],
hi_raw[3],
]);

alfiedotwtf marked this conversation as resolved.
Show resolved Hide resolved
// Hash the first six instructions
let mut hasher = Sha256::new();
for (_, raw) in first_six_instructions {
hasher.update(raw);
}

// Continue hashing the remaining instructions up to the configurables section offset.
instructions
.take(
configurables_offset / fuel_asm::Instruction::SIZE
- CONFIGURABLES_OFFSET_PREAMBLE,
) // Minus 6 because we already hashed the first six
.for_each(|(_, raw)| {
hasher.update(raw);
});

let hash_result = hasher.finalize();
let bytecode_id = format!("{:x}", hash_result);
return Ok(bytecode_id);
}
}

Err(anyhow!("Configurables section offset not found"))
}

#[cfg(test)]
mod test {
use super::*;

#[test]
fn test_get_bytecode_id_happy() {
let bytecode_id: String =
get_bytecode_id("tests/fixtures/bytecode/debug-counter.bin").expect("bytecode id");
assert_eq!(
bytecode_id,
"e65aa988cae1041b64dc2d85e496eed0e8a1d8105133bd313c17645a1859d53b".to_string()
);

let bytecode_id =
get_bytecode_id("tests/fixtures/bytecode/release-counter.bin").expect("bytecode id");
assert_eq!(
bytecode_id,
"42ae8352cbc892d7c7621f1d6fb42b072a08ba5968508d49f54991668d4ea141".to_string()
);

let bytecode_id =
get_bytecode_id("tests/fixtures/bytecode/debug-configurable_constants.bin")
.expect("bytecode id");
assert_eq!(
bytecode_id,
"babc3d9dcac8d48dee1e5aeb3340ff098d3c1ab8b0a28341d9291d8ff757199e".to_string()
);

let bytecode_id =
get_bytecode_id("tests/fixtures/bytecode/release-configurable_constants.bin")
.expect("bytecode id");
assert_eq!(
bytecode_id,
"2adfb515b66763fd29391bdba012921d045a0be83d89be5492bcaacc429695e9".to_string()
);
}

#[test]
fn test_get_bytecode_id_missing_configurable_offset() {
let result = get_bytecode_id(
"tests/fixtures/bytecode/debug-configurable_constants-missing-offset.bin",
);
assert_eq!(
result.unwrap_err().to_string().as_str(),
"Configurables section offset not found"
);
}

#[test]
fn test_get_bytecode_id_bad_path() {
let result = get_bytecode_id("tests/fixtures/bytecode/blahblahblahblah.bin");
assert_eq!(
result.unwrap_err().to_string().as_str(),
"tests/fixtures/bytecode/blahblahblahblah.bin: file not found"
);
}
}
1 change: 1 addition & 0 deletions forc-util/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ use sway_error::{
use sway_types::{LineCol, LineColRange, SourceEngine, Span};
use sway_utils::constants;

pub mod bytecode;
pub mod fs_locking;
pub mod restricted;

Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
14 changes: 2 additions & 12 deletions forc/src/cli/commands/parse_bytecode.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
use anyhow::anyhow;
use clap::Parser;
use forc_util::bytecode::parse_bytecode_to_instructions;
use forc_util::ForcResult;
use std::fs::{self, File};
use std::io::Read;
use term_table::row::Row;
use term_table::table_cell::{Alignment, TableCell};
use tracing::info;
Expand All @@ -21,15 +19,7 @@ pub(crate) struct Command {
}

pub(crate) fn exec(command: Command) -> ForcResult<()> {
let mut f = File::open(&command.file_path)
.map_err(|_| anyhow!("{}: file not found", command.file_path))?;
let metadata = fs::metadata(&command.file_path)
.map_err(|_| anyhow!("{}: file not found", command.file_path))?;
let mut buffer = vec![0; metadata.len() as usize];
f.read_exact(&mut buffer).expect("buffer overflow");

let instructions = fuel_asm::from_bytes(buffer.iter().cloned())
.zip(buffer.chunks(fuel_asm::Instruction::SIZE));
let instructions = parse_bytecode_to_instructions(&command.file_path)?;

let mut table = term_table::Table::new();
table.separate_rows = false;
Expand Down
Loading