From c2a7983cad40188fcd2ad490fca620d75f745580 Mon Sep 17 00:00:00 2001 From: Kamila Szewczyk Date: Tue, 15 Oct 2024 15:13:51 +0200 Subject: [PATCH] v0.2 --- .github/workflows/release.yml | 2 + ChangeLog | 9 +- Makefile.am | 2 + NEWS | 8 +- README.md | 2 +- configure.ac | 2 +- xpar.1 | 174 ++++++++++++++++++++++++++++++++++ 7 files changed, 193 insertions(+), 6 deletions(-) create mode 100644 xpar.1 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e2ee976..d390c07 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -67,6 +67,8 @@ jobs: run: | sudo apt-get update sudo apt-get install -y ${{ matrix.target[2] }} + - name: Install nasm + run: sudo apt-get install nasm -y - name: Configure run: ./configure --bindir=/ --program-suffix=-${{ matrix.target[0] }} ${{ matrix.target[1] }} - name: Make diff --git a/ChangeLog b/ChangeLog index 06f46c1..f2eb4b8 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,8 +1,11 @@ -2024-10-14 Kamila Szewczyk +2024-10-15 Kamila Szewczyk + + aarch64-machine-specific code - CI: binaries; configure: static linking mode +2024-10-14 Kamila Szewczyk - CI: tentative release process preparations + v0.1 release + update AUTHORS move read_header_from_map into a guard block diff --git a/Makefile.am b/Makefile.am index 9206978..92964f8 100644 --- a/Makefile.am +++ b/Makefile.am @@ -14,6 +14,8 @@ if XPAR_AARCH64 xpar_SOURCES += xpar-aarch64.S endif +dist_man_MANS = xpar.1 + # Developer convenience targets .PHONY: update-ChangeLog update-ChangeLog: diff --git a/NEWS b/NEWS index 35e05d1..2d95fe9 100644 --- a/NEWS +++ b/NEWS @@ -2,7 +2,13 @@ This file contains release notes for major and minor releases of xpar. For a complete list of source-level changes, consult the ChangeLog file. =============================================================================== -v0.1 (pending release) +v0.2 (15-10-2024) +- Provides a manual page for the xpar command. +- Provides platform-specific code for aarch64, which can be enabled via + the --enable-aarch64 configure option. + +=============================================================================== +v0.1 (14-10-2024) - Initial release. - Supports joint mode and sharded mode for error and erasure correction. - Provides platform-specific code for x86_64, which can be enabled via diff --git a/README.md b/README.md index fc8735f..cd268ad 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Licensed under the terms of GNU GPL version 3 or later - see COPYING. Report issues to Kamila Szewczyk . Project homepage: https://github.com/kspalaiologos/xpar -xpar in solid mode generates a slightly inflated (by about 12%) parity-guarded +xpar in joint mode generates a slightly inflated (by about 12%) parity-guarded file from a given data file. Such a file can be recovered as long as no more than about 6.2% of the data is corrupted. xpar internally uses a (255,223)-RS code over an 8-bit Galois field. diff --git a/configure.ac b/configure.ac index 5baaeea..d749aa0 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ m4_define([xpar_version_major], [0]) -m4_define([xpar_version_minor], [1]) +m4_define([xpar_version_minor], [2]) m4_define([xpar_version], [xpar_version_major.xpar_version_minor]) AC_PREREQ([2.69]) diff --git a/xpar.1 b/xpar.1 new file mode 100644 index 0000000..9fbdf8c --- /dev/null +++ b/xpar.1 @@ -0,0 +1,174 @@ +.TH XPAR 1 +.SH NAME +xpar \- an error/erasure code system guarding data integrity +.SH SYNOPSIS +.ll +8 +.B xpar +.RB [ " \-Je " / " \-Jd " ] +.RB [ " \-hfvqVc " ] +.RB [ " \-i/--interlacing\ # " ] +.RB [ " \--no-mmap\ " ] +[ +.I "names \&..." +] +.ll -8 +.br +.B xpar +.RB [ " \-Se " ] +.RB [ " \-hfvqV " ] +.RB [ " \--out-prefix\ # ", " \--dshards\ # ", " \--pshards\ # " ] +.RB [ " \--no-mmap\ " ] +[ +.I "names \&..." +] +.br +.B xpar +.RB [ " \-Sd " ] +.RB [ " \-hfvqV " ] +[ +.I "names \&..." +] +.SH DESCRIPTION +.PP +The +.B xpar +command +is the command-line interface to an error/erasure code system which +fortifies data and corrects errors in the data or reconstructs missing +shards. +.PP +.B xpar +in joint encoding mode ( +.B \-Je +) generates a slightly inflated (by about 12%) parity-guarded +file from a given data file with the extension +.I .xpar +. Such a file can be recovered as long as no more than about 6.2% of the +data is corrupted. xpar internally uses a (255,223)-RS code over an 8-bit +Galois field. At most two filenames intertwined with flags are expected +as arguments. +.B xpar +will by default not overwrite existing files. If this behaviour is intended, +use the +.B \-f +option. +If no file names are specified, +.B xpar +will process data from the standard input and write it to the standard output, +refusing to output binary data to a terminal. If two files are specified, +the first one is used in place of standard input, and the second one is used +in place of standard output. If the -c flag is present, +.B xpar +will read from the specified file and output data to standard output instead. +Otherwise, if decoding, +.B xpar +will try to guess the decompressed filename by removing the +.I .xpar +extension. If not present, an error will be reported. If encoding +the output filename will be generated by appending the +.I .xpar +extension to the input filename. +.PP +In joint mode, the flag +.B \-i/\-\-interlacing +can be used to specify the interlacing factor. This does not affect the nominal +error correction capability (in the sense of a noisy memory-less channel model), +but can be used to increase the robustness against burst errors. The default +interlacing factor is 1 and the maximum correctible burst error length is +16 (as given by the Singleton bound). The interlacing factor of 2 transposes +65025-byte blocks of data and hence can correct burst errors of length 4080. The +interlacing factor of 3 transposes approx. 16MB blocks of data and can correct +burst errors of length approx. 1MB. In theory, with the interlacing factor of +one, up to 32 burst errors could be corrected - this problem is however NP-hard +and reduces to polynomial reconstruction (the basis of e.g. Shamir's secret +sharing scheme). +.PP +.B xpar +in sharded encoding mode ( +.B \-Se +) generates a set of parity-guarded files from a given data file. The amount of +files generated is given by the sum of the mandatory parameters +.B \--dshards +and +.B \--pshards +- which specify the number of data and parity shards, respectively. The data +shards contain the file contents verbatim ( +.B xpar\ +uses a systematic code). The data is reconstructible as long as there are more +than +.B \--dshards +shards available. +.SH OPTIONS +.TP +.B \-S --sharded +Enable sharded processing mode. +.TP +.B \-J --joint +Enable joint processing mode. +.TP +.B \-e --encode +Enable encoding mode. +.TP +.B \-d --decode +Enable decoding mode. +.TP +.B \-h --help +Print a help message and exit. +.TP +.B \-f --force +Force operation. Ignore errors and try to recover from them, if possible. +Overwrite existing files. +.TP +.B \-v --verbose +Be verbose. Print additional information. +.TP +.B \-q --quiet +Be quiet. Suppress most output, except fatal errors. +.TP +.B \-V --version +Print version information of the package and exit. +.TP +.B \-c --stdout +Write resultant data to standard output. Joint mode only. +.TP +.B \-i --interlacing +Specify the interlacing factor. Joint mode only. +.TP +.B \--no-mmap +Disable memory mapping. Generally results in worse performance, as the fallback +method is to read the file in chunks using the standard I/O library. +.TP +.B \--out-prefix +Specify the prefix for the output files (shards). Sharded mode only. +.TP +.B \--dshards +Specify the number of data shards. Sharded mode only. Mandatory. +.TP +.B \--pshards +Specify the number of parity shards. Sharded mode only. Mandatory. +.SH REPORTING BUGS +Report bugs to: kspalaiologos@gmail.com or kszewczyk@acm.org, or open a ticket +at https://github.com/kspalaiologos/xpar. +.SH COPYRIGHT NOTICE +Copyright \(co 2022-2024 Kamila Szewczyk +.PP +Thanks to: Phil Karn (KA9Q), for the original implementation of the Reed-Solomon +code, licensed under the GPLv2-or-later license. Marshall Lochbaum for noticing +an optimisation to the syndrome computaton code. Rich Felker for the +implementation of +.B getopt_long +in musl libc that is included in this package, licensed under the terms of the +Expat/MIT license. +.PP +Permission is granted to make and distribute verbatim copies of +this manual provided the copyright notice and this permission notice +are preserved on all copies. Permission is granted to process this file through troff and print the +results, provided the printed document carries copying permission +notice identical to this one except for the removal of this paragraph +(this paragraph not being relevant to the printed manual). Permission is granted to copy and distribute modified versions of this +manual under the conditions for verbatim copying, provided that the entire +resulting derived work is distributed under the terms of a permission +notice identical to this one. Permission is granted to copy and distribute translations of this manual +into another language, under the above conditions for modified versions, +except that this permission notice may be stated in a translation approved +by the Foundation.