[pull] master from restic:master #201
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
backup: fix handling of files with type irregular
dump: add --compress flag to compress archives
fs: move NodeFromFileInfo into FS interface
Ignore AuthorizationFailure caused by using a container level SAS/SAT token when calling GetProperties during the Create() call. This is because the GetProperties call expects an Account Level token, and the container level token simply lacks the appropriate permissions. Supressing the Authorization Failure is OK, because if the token is actually invalid, this is caught elsewhere when we try to actually use the token to do work.
Add changelog entry in the 'unreleased' sub-folder for changes introduced when fixing issue #4004.
Add two new test cases, TestBackendAzureAccountToken and TestBackendAzureContainerToken, that ensure that the authorization using both types of token works. This introduces two new environment variables, RESTIC_TEST_AZURE_ACCOUNT_SAS and RESTIC_TEST_AZURE_CONTAINER_SAS, that contain the tokens to use when testing restic. If an environment variable is missing, the related test is skipped.
Fix Azure Container Token Auth
Previously, NodeFromFileInfo used the original file path to create the node, which also meant that extended metadata was read from there instead of within the vss snapshot.
OpenFile, Stat and Lstat should reuse the underlying FS implementation to avoid diverging behavior.
When using the VSS FS, then `f.Name()` contained the filename in the snapshot. This caused a double mapping when calling NodeFromFileInfo.
The only user was archiver.fileSaver.
Fix multiple restic processes executing concurrently and racing to remove obsolete snapshots. Co-authored-by: Michael Eischer <[email protected]>
restic cannot backup irregular files as those don't behave like normal files. Thus skip them with an error.
Bumps [github.com/spf13/pflag](https://github.com/spf13/pflag) from 1.0.5 to 1.0.6. - [Release notes](https://github.com/spf13/pflag/releases) - [Commits](spf13/pflag@v1.0.5...v1.0.6) --- updated-dependencies: - dependency-name: github.com/spf13/pflag dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) from 1.8.0 to 1.8.1. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](Azure/azure-sdk-for-go@sdk/azcore/v1.8.0...sdk/azidentity/v1.8.1) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>
…/Azure/azure-sdk-for-go/sdk/azidentity-1.8.1 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity from 1.8.0 to 1.8.1
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go) from 1.5.0 to 1.6.0. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](Azure/azure-sdk-for-go@sdk/azcore/v1.5.0...sdk/azcore/v1.6.0) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
…/spf13/pflag-1.0.6 build(deps): bump github.com/spf13/pflag from 1.0.5 to 1.0.6
ui/termstatus: Remove unused bytes.Buffer
…/Azure/azure-sdk-for-go/sdk/storage/azblob-1.6.0 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.5.0 to 1.6.0
doc: add edit on github link
) * feat(backends/s3): add warmup support before repacks and restores This commit introduces basic support for transitioning pack files stored in cold storage to hot storage on S3 and S3-compatible providers. To prevent unexpected behavior for existing users, the feature is gated behind new flags: - `s3.enable-restore`: opt-in flag (defaults to false) - `s3.restore-days`: number of days for the restored objects to remain in hot storage (defaults to `7`) - `s3.restore-timeout`: maximum time to wait for a single restoration (default to `1 day`) - `s3.restore-tier`: retrieval tier at which the restore will be processed. (default to `Standard`) As restoration times can be lengthy, this implementation preemptively restores selected packs to prevent incessant restore-delays during downloads. This is slightly sub-optimal as we could process packs out-of-order (as soon as they're transitioned), but this would really add too much complexity for a marginal gain in speed. To maintain simplicity and prevent resources exhautions with lots of packs, no new concurrency mechanisms or goroutines were added. This just hooks gracefully into the existing routines. **Limitations:** - Tests against the backend were not written due to the lack of cold storage class support in MinIO. Testing was done manually on Scaleway's S3-compatible object storage. If necessary, we could explore testing with LocalStack or mocks, though this requires further discussion. - Currently, this feature only warms up before restores and repacks (prune/copy), as those are the two main use-cases I came across. Support for other commands may be added in future iterations, as long as affected packs can be calculated in advance. - The feature is gated behind a new alpha `s3-restore` feature flag to make it explicit that the feature is still wet behind the ears. - There is no explicit user notification for ongoing pack restorations. While I think it is not necessary because of the opt-in flag, showing some notice may improve usability (but would probably require major refactoring in the progress bar which I didn't want to start). Another possibility would be to add a flag to send restores requests and fail early. See #3202 * ui: warn user when files are warming up from cold storage * refactor: remove the PacksWarmer struct It's easier to handle multiple handles in the backend directly, and it may open the door to reducing the number of requests made to the backend in the future.
…orb the error * sometimes, the report function may absorb the error and return nil, in those cases the bar.Add(1) method would execute even if the file deletion had failed
…rect-number-of-files-deleted forget: report count of deleted files correctly
Bump minimum go version to 1.22
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.204.0 to 0.219.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](googleapis/google-api-go-client@v0.204.0...v0.219.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
…ang.org/api-0.219.0 build(deps): bump google.golang.org/api from 0.204.0 to 0.219.0
…#5182) Enhancement: create ability to sort output of restic ls -l by name, size, atime, ctime, mtime, time(=mtime), X(=extension), extension --------- Co-authored-by: Michael Eischer <[email protected]>
…apshot. (#5185) Co-authored-by: Michael Eischer <[email protected]>
Refactor ls sorting
where possible (max is unknown).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )