How to construct the data set

[zhipeng-web]

Hello author, I have some data collected by myself, what should I do to construct the same json file as you, thank you for telling me.

[FakeEyes2Wo]

I think u can do this
from ember.features import PEFeatureExtractor

extractor = PEFeatureExtractor(feature_version=2)
with open('test.exe', 'rb') as f:
bytez = f.read()
print(extractor.features)
feature_vector = extractor.feature_vector(bytez)

[zer0daysec]

This is an example of a generated feature I wrote

main.py

import os
import json
import ember
import argparse

from pathlib import Path

def get_file_size(file_path):
    return os.path.getsize(file_path) if file_path.exists() else 0

if __name__ == "__main__":
    prog = "sample feature"
    description = "Get Sample feature"
    parser = argparse.ArgumentParser(prog=prog, description=description)
    parser.add_argument("-v", "--featureversion", type=int, default=2, help="EMBER feature version")
    parser.add_argument("folder", type=str, default="./samples", help="samples folder path")
    args = parser.parse_args()

    folder_path = Path(args.folder)
    output_dir = Path("./data")

    max_file_size = 2 * 1024 * 1024 * 1024
    file_index = 0
    current_file = output_dir / f"train_features_{file_index}.jsonl"

    with open(current_file, "a") as f:
        for binary_path in folder_path.rglob("*"):
            if binary_path.is_file():
                with open(binary_path, "rb") as file:
                    file_data = file.read()
                extractor = ember.PEFeatureExtractor(args.featureversion)
                json_features = json.dumps(extractor.raw_features(file_data))
                if get_file_size(current_file) + len(json_features.encode("utf-8")) > max_file_size:
                    f.close()
                    file_index += 1
                    current_file = output_dir / f"train_features_{file_index}.jsonl"
                    f = open(current_file, "a")
                f.write(json_features + "\n")

but no md5, appeared, label, avclass attribute

I don't know how to generate the missing 4 attribute values. If you know, can you tell me?