Nydus Image Service v2.1.0

Binaries download mirror (sync within a few hours): https://registry.npmmirror.com/binary.html?path=nydus/v2.1.0/

What's New Features

• nydusd: production ready for nydus rafs v6 (erofs) format for fusedev and in-kernel fscache driver
• nydusd: support registry p2p acceleration with dragonfly in mirror mode.
• nydus-image: support nydus rafs to tar conversion (unpack nydus image)
• nydusify: support "view" subcommand to view nydus image lazily

What's Changed

• enable "erofs over fscache" feature by @nan1994 in #446
• cargo: upgrade dbs-uhttp to v0.3.0 by @imeoer in #447
• action: allow run CI for pr wants merge to any branch by @imeoer in #449
• storage: fix clippy warnings by @jiangliu in #455
• nydusd: clean up clippy warnings by @jiangliu in #456
• doc: update outdated information by @hsiangkao in #459
• nydusd: recover --shared-dir for fuse backend again by @hsiangkao in #460
• rafs/v6: fix wrongly calculated prefetch table size by @changweige in #458
• switch nydus-cached to mio by @uran0sH in #453
• refact: nydusd's state machine by @ccx1024cc in #439
• Prepare for publishing nydus-storage and nydus-rafs by @jiangliu in #452
• Tiny typo fix and improve nydusd help message by @changweige in #461
• fix: fscache and virtio fail to start by @ccx1024cc in #462
• storage: fix a bug related caused by merging code by @jiangliu in #464
• Improvements to the blob cache manager by @jiangliu in #466
• storage: fix ondemand read may cause IO err when enable prefetch by @kevinXYin in #468
• Refine utils/metrics related code by @jiangliu in #470
• nydus-image: fix some comments by @power-more in #476
• doc: refine README.md by @hsiangkao in #478
• doc: more fscache-related description by @hsiangkao in #479
• nydusify & nydus-image: v6 image conversion support by @yawqi in #473
• Introduce async framework and clean up prefetch related logic by @jiangliu in #467
• Prepare for releasing nydus-api v0.1.0 by @jiangliu in #474
• doc: nydus-fscache.md: fix up referenced RAFS v6 images by @hsiangkao in #487
• Doc: Correct fscache distribution steps by @yqleng1987 in #482
• Prepare for releasing nydus-utils-v0.3 by @jiangliu in #488
• nydus-image: supports multiple versions of merge by @zyfjeff in #489
• Add some comments by @power-more in #491
• nydusify: bug fix, check subcommand uses direct mode by @yawqi in #494
• stroage: garbage collect unused blob cache mgr by @jiangliu in #481
• doc: update expired dingtalk qr code by @imeoer in #499
• nydus-image: fixed a bug caused by empty blob and prefetched combined scenes by @zyfjeff in #495
• nydusify: add fs-version annotation by @yawqi in #497
• nydusify: add command line arg fs-version check by @yawqi in #501
• doc: fix broken dingtalk invitation url and others by @imeoer in #504
• blobfs: prepare for publishing v0.1 by @jiangliu in #500
• utils: introduce per-thread tokio current thread Runtime by @jiangliu in #498
• Prepare for publishing nydus-2.1.0-alpha1 by @jiangliu in #508
• api: prepare for publishing v0.1.1 by @jiangliu in #509
• github action: add top images v6 format conversion and check by @yawqi in #490
• Prepare for publishing v2.1.0-alpha2 by @jiangliu in #511
• Add documentation for overall architecture by @jiangliu in #510
• use precise command in nydus-fscache doc by @liubogithub in #517
• doc: fix docker daemon configuration path by @imeoer in #522
• api: reorganize storage API definitions by @bergwolf in #514
• Update containerd-env-setup.md by @aa624545345 in #505
• Support eStargz TOC conversion and run for RAFS v6 by @imeoer in #486
• ctr-remote: replace the appendDefaultLabelsHandlerWrapper function by @loheagn in #525
• Fixed several correctness issues with Nydus v6 at runtime. by @zyfjeff in #521
• fscache: remove warning when handle fscache read request by @imeoer in #532
• nydus-image: fix the prefetch table was out of bounds due to use the wrong index by @zyfjeff in #530
• nydus-image nydus-cached: builds on macos by @wanderxjtu in #523
• comments: modify comments of src_ino and i_ino by @power-more in #519
• nydus-image: nydus v6 support explicit uid and gid by @zyfjeff in #531
• FIX: miss blob header in inline-nydus-file by @ccx1024cc in #534
• github actions: bug fix, the build nydus-rs always gets cancelled by @yawqi in #537
• fscache: enable multi-threading to process fscache requests by @jiangliu in #536
• github actions: add integration tests for nydus by @yawqi in #535
• nydusd: wrap thread validator as func by @imeoer in #539
• feat: support convert Nydus image layers to tar files by @ccx1024cc in #538
• ctr-remote: note user this ctr is only enhanced for nydus by @changweige in #544
• makefile: add target install by @changweige in #545
• fscache: fix large stargz chunks processing by @imeoer in #546
• doc: add description about estargz and nerdctl support by @imeoer in #547
• README.md: fix up leftovers of stargz for fscache by @hsiangkao in #548
• Fix spelling mistakes by @changweige in #549
• nydus-image: remove redundant -o short option by @changweige in #550
• rafs: v6 must have local blobcache configured by @changweige in #551
• build(deps): bump github.com/containerd/containerd from 1.4.13 to 1.5.13 in /contrib/nydusify by @dependabot in #553
• Add a recording for fscache lazy pulling by @hsiangkao in #557
• fscache: no need to clone string by @changweige in #558
• nydus-backend-proxy reload blob objects map by @changweige in #556
• dep: upgrade packages to fix some security vulnerabilities by @imeoer in #562
• rafs/v6: pack rdev for rafs v6 image by @changweige in #564
• nydus-test: Add BackendProxy storage backend to simulate registry storage backend by @changweige in #563
• add basic nydusify conversion case by @changweige in #566
• nydus-test: ignore missing backend_proxy_url by @changweige in #567
• ci: don't run Convert Top Docker Hub Images job on forked repos by @liubin in #574
• Use nydusify to check file content for nydus v6 by @zyfjeff in #572
• nydusd: fixed getting the wrong chunk due to using the wrong blob index by @zyfjeff in #573
• nydus-test: Replace storage backend with BACKEND_PROXY and adapts github action by @changweige in #570
• nydus-image: use BtreeMap replace HashMap to guarante order of chunkinfo by @zyfjeff in #575
• nydusify: Add fs-version args for pack subcommand by @zyfjeff in #529
• Add layered image conversion to IT workflow and replace storage backend with BACKEND_PROXY by @changweige in #577
• chore: change trivial logs from INFO to DEBUG by @ccx1024cc in #579
• backend-proxy: don't panic if address already binded by @changweige in #578
• add metadata fields for nydusify filesystem check by @wraymo in #543
• Doc: make sure snapshotter's root dir exist by @liubogithub in #581
• fscache: print error message when nydusd can't start by @changweige in #582
• ci: ci/release support macos arm by @killagu in #584
• Restrain rafs v6 mount if its configuration has digest_validate by @changweige in #576
• docs: add overview for data deduplication by @jiangliu in #583
• docs: detail improvement for running nydus by @imeoer in #589
• docs: add documentation for Nydus data deduplication by @jiangliu in #588
• doc: use snapshotter --fs-driver in instrunctions by @changweige in #593
• prepare for nydusd daemonless by @kevinXYin in #540
• Update nydusd doc for bootstrap creation by @liubogithub in #591
• Display detailed error in FsCacheHandler constructor when cachefiles device not ready by @benjaminhuanghuang in #596
• rafs: use real fs root ino to trigger prefetch by @changweige in #597
• Add pre built nydus v6 image by @liubogithub in #598
• storage: fix io error may let the chunk map bit always pending by @kevinXYin in #601
• Enhance nydus-backend-proxy by @jiangliu in #600
• Update dependencies by @jiangliu in #605
• Nydusify: support "view" subcommand to view Nydus image by @YushuoEdge in #603
• Reduce network IOs by amplifying prefetch requests by @jiangliu in #609
• make copen always return right file size by @kevinXYin in #606
• Prefetch limit by @changweige in #611
• action: avoid print tons of logs by @changweige in #615
• Dump nydusd panic info and backtrace into logger and show nydusd git commit version by @changweige in #613
• storage: fetch blob.meta in asynchronous mode by @jiangliu in #610
• action: enable nydus integration tests by @bergwolf in #620
• nydusd: gc unused blob files after umounting by @jiangliu in #619
• storage: remove unnecessary debug_assert by @changweige in #622
• fscache: fix a typo by @changweige in #625
• nydusd: fix race window in copen handler by @kevinXYin in #623
• Optimize performance by fix a bug in prefetch by @jiangliu in #626
• unpack: fix empty username and groupname by @imeoer in #630
• storage: remove localfs based prefetch by @jiangliu in #628
• storage: unify method naming style for ChunkInfo by @changweige in #632
• remove dead code nydus-cached by @changweige in #635
• builder: fix broken symlink in unpack by @imeoer in #636
• Improve prefetch performance further by @jiangliu in #633
• ci: remove nydus-cached binary build by @imeoer in #637
• storage: convert chunk info per as blob meta regardless of by @changweige in #629
• storage: fix range prefetch algorithm from panic and io hang by @changweige in #614
• rafs: try best to reduce memory overhead by @changweige in #640
• builder: fix rafs v5 dir size for reproducible build by @imeoer in #642
• Clean up code for v2.1 by @jiangliu in #644
• registry: avoid unnecessary memory allocation by @changweige in #646
• Adapt further to rafs v6 format test by @changweige in #645
• nydus-image: move set_v5_dir_size() to Node by @jiangliu in #647
• nydus-image: fix incorrect node offset in rafs v6 for hardlinks by @yixiaojie0x in #643
• nydus-image: ensure has_attr flag is set correctly by @jiangliu in #648
• README.md: add description about community meeting by @hsiangkao in #652
• nydusify: skip tls verification by @imeoer in #616
• builder: remove diff build feature by @imeoer in #649
• storage: reflect prefetch state from each blob cache object by @changweige in #650
• storage: simplify BlobCache::is_prefetch_active() by @jiangliu in #654
• nydus-test: replace frame bin skopeo to static version by @changweige in #657
• Get rid of BlobIoChunk::V5 by @changweige in #656
• nydus-test: delay for some time before verifying prefetch metrics by @changweige in #659
• nydus: prepare for releasing v2.1.0-rc1 by @jiangliu in #660
• builder: fix panic on building big file by @imeoer in #663
• Fix test by @changweige in #665
• Rename nydusctl subcommand by @changweige in #666
• storage: use dyn BlobChunkInfo in storage layer by @changweige in #664
• storage/backend: use rate log to prevent tons of logs by @changweige in #669
• nydusify: infer BackendType and BackendConfig by @wraymo in #618
• storage: avoid allocating memory when requesting registry by @changweige in #673
• README.md: add description about CRI-O / Podman support by @hsiangkao in #677
• nydusctl: beautify backend metrics display by @changweige in #679
• nydusctl: print rafs read op latency distribution by @changweige in #680
• storage: clear some dead code by @changweige in #681
• builder: only reference the blob actually used in the chunk dict by @imeoer in #678
• storage: don't drop tokio runtime to non-blocking context by @changweige in #682
• rafs: optimize entry search in rafs v6 by @yixiaojie0x in #683
• release: prepare for v2.1.0-rc.2 by @imeoer in #685
• release: update Cargo.lock to v2.1.0-rc.2 by @imeoer in #688
• nydus-test framework supports fscache mode by @changweige in #692
• add log rotation by @sctb512 in #689
• deps: fix security vulnerability for lz4-sys by @imeoer in #696
• nydus-test: clean up readme as a simple usage guide by @changweige in #695
• nydus-test: don't assign nydusd to anchor by @changweige in #697
• nydusctl: fix panic when query nydusd as fscache daemon by @changweige in #698
• nydusify: fix incorrect http scheme for checker by @imeoer in #699
• nydus-test: use url builder to query by @changweige in #701
• nydus-test: add the missing parameter 'artifacts' by @sctb512 in #702
• storage: fix blob batch read error by @kevinXYin in #703
• builder: output blob list for merge subcommand by @imeoer in #700
• nydusd: remove unix socket before exiting by @sctb512 in #706
• misc: add more release archs by @imeoer in #690
• cargo: update fuse-backend-rs dependency by @bergwolf in #708
• Fix opendir cache issue for rafs, passthroughfs, and the hybrid mode by @zhangjaycee in #707
• Fix blob uncompressed size by @changweige in #712
• nydusd: increase default fscache threads by @kevinXYin in #715
• kernel-patches: introduce kernel patch for fscache ondemand IO path o… by @kevinXYin in #717
• storage: fix compile error by @wllenyj in #718
• add cases and enhance framework symlink verifier by @changweige in #713
• dep: update dbs-uhttp to v0.3.1 by @jiangliu in #719
• nydusctl: refine commandline help message by @jiangliu in #720
• nydusd: rename daemon subcommand to singleton by @bergwolf in #722
• Improve nydusify build subcommand by @jiangliu in #721
• storage: remove readahead configuration from LocalFsConfig by @jiangliu in #726
• rafs: introduce iterator to walk inode of a rafs filesystem by @jiangliu in #728
• Revert "makefile: fix a minor bug in Makefile" by @imeoer in #730
• nydusd: fix a bug in handling singleton subcommand by @jiangliu in #731
• api: rename blob cache entry config's fs_prefetch by @changweige in #709
• nydusd: fix a bug in handling singleton subcommand by @jiangliu in #732
• nydus-image: refactor subcommand "inspect" by @YushuoEdge in #675
• nydusify: abort oss blob upload if conversion failed by @imeoer in #710
• release: prepare for v2.1.0-rc.3.1 by @imeoer in #733
• action: enable fuse + rafsv5 integration test by @bergwolf in #734
• Improve help message for view, check and build subcommands by @jiangliu in #725
• nydusd: make thier names consistent in singleton mode by @changweige in #739
• Reduce dependency by @jiangliu in #738
• dependency: remove dependency on serde-with by @jiangliu in #617
• nydus-test: re-enable v5 nydusify test by @bergwolf in #744
• Unify fusedev and virtiofs frontend by @jiangliu in #743
• nydusify: remove dependency on docker to pull images when checking nydus images by @wraymo in #729
• Clean up fusedev and enhance makefile targets for macos by @jiangliu in #746
• storage: add mirror support by @sctb512 in #736
• tests: fix up integration tests by @bergwolf in #747
• test: replace java image with amazoncorretto by @imeoer in #748
• makefile: unify the makefiles for contrib components by @jiangliu in #749
• nydusify: output nydus & oci manifest index for check by @imeoer in #750
• workflow: refine release workflow for macos by @jiangliu in #752
• Adapt nydus test by @changweige in #754
• makefile: fix invalid "install -D" option on macos by @jiangliu in #753
• makefile: fix a typo in of INSTALL_DIR_PREFIC by @jiangliu in #756
• storage: support configure proxy use_http by @changweige in #759
• release: add npm mirror url into release note by @imeoer in #761
• contrib: upgrade to go 1.18 by @imeoer in #762
• fix mirror's legacy issues by @sctb512 in #758
• Enrich nydus-image print, fix prefetch table layout and other couple of fixes by @changweige in #767
• nydusify: support hook plugin by @imeoer in #763
• storage: print explicit error message by @changweige in #769
• nydusd: enlarge default fuse server threads by @changweige in #789
• nydus-image/v5: prefetch table should contain inode numbers rather its index by @changweige in #793
• nydus-image: fix inspect to get correct path of rafs v6 file by @sctb512 in #785
• Backport some patches for stable/v2.1 by @changweige in #799
• release: update version on build automatically by @imeoer in #804

New Contributors

• @uran0sH made their first contribution in #453
• @power-more made their first contribution in #476
• @yqleng1987 made their first contribution in #482
• @zyfjeff made their first contribution in #489
• @aa624545345 made their first contribution in #505
• @loheagn made their first contribution in #525
• @wraymo made their first contribution in #543
• @YushuoEdge made their first contribution in #603
• @yixiaojie0x made their first contribution in #643

Full Changelog: v2.0.0...v2.1.0