users/evgenybud/vhost rdma server migration v2

[budevg]

• Use new excludes format, remove old excludes usage
• Mark singleton accessors by returns_nonnull attribute
• YDB Import 503
• Count broken partitions instead of broken disks for placement groups
• Update ytexec resource.
• Intermediate changes
• External build system generator release 70
• Intermediate changes
• workaround for rdma test failing on left ci setup
• YDB Import 504
• Update contrib/restricted/boost/program_options to 1.84.0
• Intermediate changes
• rename some libs to fix building with cmake
• fix review issues
• Add warning for dublicated files in RESOURCE_FILES() macro
• Update Cython to 0.29.37
• Flake8 migrations clean up
• fix the value of _SRC_PYSRC_CMDLINE in global scope
• Intermediate changes
• Update contrib/python/yarl to 1.9.4
• Disk Manager: Make dry run catch case of missing templates in right acceptance tests
• Remove old semantics for excludes
• revert update curl to 8.2.1 and apply patch from ydb
• Intermediate changes
• fix race in endpoint file storage, support persistent endpoints in nfs
• Allow dont generate debug info for cpp proto libs
• Intermediate changes
• Revert commit rXXXXXX,Remove USE_OPENSOURCE_TEST_TOOL flag
• [Disk Manager]: Fix bug with incorrect context manager usage in eternal_acceptance_test_runner.py
• NOTICKET: splitting blockstore and filestore (currently empty) docs;
• implemented ReadBlob/WriteBlob/PatchBlob metrics for filestore tablet: Count, RequestBytes, Time
• [Disk Manager] release target units and slots on relocation abort
• Update contrib/restricted/boost/type_index to 1.84.0
• Fix hack for setuptools on Windows
• Add flock to "ext_mapping.conf.json"
• Flake8 migrations clean up
• Macro to search junit tests in separate jar
• Automatic release build for ymake, os_ymake
• Define TRUE and FALSE ya.make constants
• Patch for Integer overflow in Poco::UTF32Encoding
• Add DCAppAttestService to sysincl
• [Disk Manager] should check base disks state instead of slots state
• Bugfix for reflection-based Proto2Json: Always write default values in maps
• Reflection-based Proto2Json: Don't copy JSON when writing into a string
• fix rdma-test using nbsci user
• Change "ext_mapping.conf.json"
• clean dll after ci tests
• Flake8 migrations clean up
• Intermediate changes
• NOTICKET: fix typo in sessions history table
• Update contrib/libs/googleapis-common-protos to 1.62.0
• Update libcxx to 22 Jul 2022 1d057a6d43060eaf6c679e0593a43f99ecd70987
• Remove hack - do not add current directory
• YDB Import 506
• Intermediate changes
• YDB Import 507
• Flake8 migrations clean up
• fix the value of _SRC_ASM_CMDLINE in global scope
• Intermediate changes
• Fix macOS OS_SDK=local build
• Intermediate changes
• Add std::filesystem::path to stlfwd.
• Remove redundant slash
• : Add docs about filestore metrics to github.
• Optimize Json2Proto/Proto2Json for map fields
• Intermediate changes
• YDB Import 511
• Proper version of iterator_traits for pointers when concepts are not present.
• Flake8 migrations clean up
• move _ADD_HIDDEN_INPUTS to more appropriate conf file
• Use msvc STL only on windows
• Add compiler runtime sysincls for local Xcode toolchain
• Add Y_ABORT_IF
• Intermediate changes
• Automatic release build for ya_bin3, ya_bin, os_ya, test_tool, os_test_tool_3, test_tool3
• Use forwarding and move to avoid copies of input value in Future::Return function.
• fast path for reading resynced disks
• set server shutdown deadline to avoid useless spinning at shutdown
• Intermediate changes
• Support Python 3.12 for multidict
• Flake8 migrations clean up
• Update contrib/python/prompt-toolkit/py3 to 3.0.43
• Fix .gcno suffix for python libraries
• [contrib/ydb]: Add default copy assignment operator in TVDiskAvailabilityTiming
• Resolve clang-win toolchains to default-win
• Introduce --dont-resolve-moves cli option
• add example to start qemu
• Introduce new ssh client using openssh for left
• [Disk Manager] fix acceptance tests command line
• Introduce APPLE_SDK_LOCAL setting
• Intermediate changes
• disable some fs_posix_compliance tests and topic_api_impl tests
• Add needed functions in ipmath.h
• Add redirect in test results nginx
• Use ref-qualified call operator in TFuture::Apply
• rejecting requests related to FastPathRecords upon EvPoisonPill + some cleanup
• Intermediate changes
• YDB Import 512
• Fix forward declaration in llvm12 Support/Json/Value
• Flake8 migrations clean up
• call Allocate/DeallocateCheckpoint
• Cleanup Python 3
• fix bison/flex cmds
• Remove unnecessary patch from Python 3
• ydb-go-sdk/v3.54.3
• tc32 toolchain for macos
• Intermediate changes
• Consider SO_REUSEPORT as universally available
• caching DA config to a local file
• NBSSUPPORT-35 [nbs tools] event log stats: look at checksums with correct commit id
• fix style
• Enable disk manager tests
• CLOUDDUTY-23951 : Fix qemu timeouts.
• NBS-4414: improved resync failure ut; added a bit more logging;
• [Disk Manager]: Fix eternal and sync tests reporting
• Add templates generation for eternal and sync tests
• NBS-4414: resync fast path should work even if there are fresh devices; added a ut for the resync with fresh devices case
• NBS-3864: introduce fast data path external endpoint
• NBSNEBIUS-46: removed dead code
• NBSNEBIUS-46: removed dead code
• [Disk Manager]: Make disk manager acceptance tests scripts executable
• include go tests in search regex
• [Disk Manager]: Generate result reports xml even if the testcase fails
• [Disk Manager]: Fix stdout and stderr paths naming in acceptance tests
• [Disk Manager]: Fix s3 url in disk manager acceptance tests
• Add test-results to gitignore
• Fixed idle session logic in TCreateSessionActor + added a ut for this logic (Fixed idle session logic in TCreateSessionActor + added a ut for this logic #29)
• not building ydb protos for java (not building ydb protos for java #36)
• improved .gitignore (improved .gitignore #38)
• making plugin tests more stable (making plugin tests more stable #39)
• NBS-4410 fix cache use in tests, add mutes, fix packer build (NBS-4410 fix cache use in tests, add mutes, fix packer build #40)
• fix: return wrongfully removed line in pr check workflow (fix: return wrongfully removed line in pr check workflow #43)
• NBSNEBIUS-14: using custom version if available (NBSNEBIUS-14: using custom version if available #42)
• NEBDUTY-297: mute tests with qemu (NEBDUTY-297: mute tests with qemu #45)
• [Disk Manager] internal/pkg/tasks should not depend on Disk Manager public API
• [Disk Manager] internal/pkg/tasks should not depend on internal/pkg/common ([Disk Manager] internal/pkg/tasks should not depend on internal/pkg/common #51)
• NBS-4407: filestore-vhost should depend on libs/vfs_fuse/vhost, not libs/vfs_fuse (NBS-4407: filestore-vhost should depend on libs/vfs_fuse/vhost, not libs/vfs_fuse #53)
• [Disk Manager]: Fix mixed up stdout and stderr in disk manager tests
• make filestore tests in opensource less verbose (Make filestore tests in open-source less verbose #50)
• [Disk Manager] remove unused code ([Disk Manager] remove unused code #58)
• not building ydb protos for java (not building ydb protos for java #37)
• [Disk Manager] simplify metrics_mock.go ([Disk Manager] simplify metrics_mock.go #60)
• Add ssh client for nbs and nfs tests (Add ssh client for nbs and nfs tests #61)
• [Disk Manager] move metrics/mocks to tasks ([Disk Manager] move metrics/mocks to tasks #62)
• [Disk Manager] internal/pkg/tasks should not depend on internal/pkg/monitoring/metrics ([Disk Manager] internal/pkg/tasks should not depend on internal/pkg/monitoring/metrics #63)
• [Disk Manager] internal/pkg/tasks/errors should not depend on internal/pkg/persistence
• fix: pr check test types (fix: pr check test types #67)
• Change clang-format config file to default (Change clang-format config file to default #33)
• CLOUDDUTY-23935 : Fix fs_posix_complience tests. (CLOUDDUTY-23935 : Fix fs_posix_complience tests #15)
• NBS-3864: set vhost rdma backend device offset to 0
• NBS-3864: use volume block size for rdma backend
• [tests] set service_local timeout to 3min ([tests] set service_local timeout to 3min #65)
• allow not to pass identity file to ssh (allow not to pass identity file to ssh #71)
• use iamtoken-client in blockstore-client (NBS-4456: use iam-token-client in blockstore-client #22)
• get rid of unnecessary dependency (get rid of unnecessary dependency #73)
• NBSNEBIUS-36: remove stopped endpoint from storage (NBSNEBIUS-36: remove stopped endpoint from storage #68)
• NBS-4410 add cmake build for stable-23-3 (NBS-4410 add cmake build for stable-23-3 #77)
• NBS-4681: cache DA sessions in a file (NBS-4681: cache DA sessions in a file #41)
• add retry to upload_file (add retry to upload_file #80)
• NBSNEBIUS-46: TCreateSessionActor: fixed reconnect in case first connect attempt fails (NBSNEBIUS-46: TCreateSessionActor: fixed reconnect in case first connect attempt fails #86)
• CLOUDDUTY-23898 : Fix data race and uninitialized value (CLOUDDUTY-23898 : Fix data race and uninitialized value #74)
• Removed tab in ya.make (Removed tab in ya.make #89)
• NBSNEBIUS-60: revive filestore/bin directory (NBSNEBIUS-60: revive filestore/bin directory #79)
• NBS-4742 : Add canonical tests. (NBS-4742 : Add canonical tests. #88)
• cleanup (cleanup TStorageAdapter #92)
• [Disk Manager] internal/pkg/tasks should not depend on internal/pkg/headers ([Disk Manager] internal/pkg/tasks should not depend on internal/pkg/headers #76)
• Fix typo in cloud/filestore/ya.make (Fix typo in cloud/filestore/ya.make #97)
• move non-interface files from iface to impl
• remove .swp file
• Removed whitespace (Removed whitespace #98)
• NBS-4410 fixes for cmake build, adding ability to disable caching for nightly builds (NBS-4410 fixes for cmake build, adding ability to disable caching for nightly builds #99)
• Fix pr.yaml workflow call (Fix pr.yaml workflow call #105)
• added vim files to .gitignore (added vim files to .gitignore #106)
• Added removed files (Added removed files #100)
• NBSNEBIUS-70: loading compaction map lazily (NBSNEBIUS-70: loading compaction map lazily #87)
• [Disk Manager] internal/pkg/tasks should not depend on internal/pkg/errors ([Disk Manager] internal/pkg/tasks should not depend on internal/pkg/errors #104)
• NBS-4410 add ability to add sleep to workflow to inspect it, added ssh key to vm, added ncp to return ipv4 address (NBS-4410 add ability to add sleep to workflow to inspect it, added ssh key to vm, added ncp to return ipv4 address #108)
• log nfs-vhost control requests (log nfs-vhost control requests #90)
• NBS-4410 fix ncp installation (NBS-4410 fix ncp installation #113)
• NBSNEBIUS-65: add rdma documentation (NBSNEBIUS-65: add rdma documentation #116)
• NBS-4816 : Fix recipes for NFS (NBS-4816 : Fix recipes for NFS #112)
• Removed tests files (Removed tests files #117)
• NBS-4808 make build and easy (NBS-4808 make build NBS easy #96)
• remove useless kms tests (remove useless kms tests #114)
• Changed generate certs script (Changed generate certs script #125)
• [Disk Manager] internal/pkg/tasks should not depend on internal/pkg/logging ([Disk Manager] internal/pkg/tasks should not depend on internal/pkg/logging #127)
• [Disk Manager] internal/pkg/persistence should not depend on internal/pkg/monitoring/metrics ([Disk Manager] internal/pkg/persistence should not depend on internal/pkg/monitoring/metrics #128)
• NBS-4410 better sleep after fails, added organization members ssh keys to vm (NBS-4410 better sleep after fails, added organization members ssh keys to vm #131)
• Lowering link_threads/test_threads value to possibly mitigate OOMs while running nightly tests (Lowering link_threads/test_threads value to possibly mitigate OOMs while running nightly tests #135)
• NBS-4410 fix token issue (NBS-4410 fix token issue #136)
• [Disk Manager] move internal/pkg/persistence to internal/pkg/tasks ([Disk Manager] move internal/pkg/persistence to internal/pkg/tasks #132)
• NEBIUSNBS-70: implemented ability to get compaction map range stats via private api, added some uts regarding compaction map initialization (NEBIUSNBS-70: implemented ability to get compaction map range stats via private api, added some uts regarding compaction map initialization #137)
• [Disk Manager] use word 'ydb' instead of 'kikimr'
• Remove all CMakeLists (Remove all CMakeLists #139)
• [Disk Manager] internal/pkg/tasks should not depend on cloud/disk_manager/test/recipe ([Disk Manager] internal/pkg/tasks should not depend on cloud/disk_manager/test/recipe #142)
• Make gitignore files docs: unable to run example #1 (Make gitignore files #1 #140)
• NBSNEBIUS-36: use tmp file for endpoint storage (NBSNEBIUS-36: use tmp file for endpoint storage #133)
• fix endpoint storage reading (NBSNEBIUS-36: fix endpoint storage reading #147)
• fix: fix warnings for not used inputs, installed atop for in-vm memory stats, increased 5m timeout in vm creation (fix: fix warnings for not used inputs, installed atop for in-vm memory stats, increased 5m timeout in vm creation #152)
• NBS-4358: fix UnadjustedBlockCount; added test_override_storage_discovery_config (NBS-4358: fix UnadjustedBlockCount; added test_override_storage_discovery_config #85)
• [Disk Manager] move cloud/disk_manager/internal/pkg/tasks to cloud/tasks ([Disk Manager] move cloud/disk_manager/internal/pkg/tasks to cloud/tasks #153)
• [Disk Manager] move cloud/disk_manager/test/common to cloud/tasks/test/common ([Disk Manager] move cloud/disk_manager/test/common to cloud/tasks/test/common #155)
• NBS-4824: [Disk Manager] Add more logs for s3 (NBS-4824: [Disk Manager] Add more logs for s3 #115)
• NBSNEBIUS-37: don't register disk agent with no devices
• NBSNEBIUS-65: add RDMA read/write sequence diagrams
• [Disk Manager] move test/nemesis to cloud/tasks ([Disk Manager] move test/nemesis to cloud/tasks #156)
• NBSNEBIUS-45: track actual time of migrations (NBSNEBIUS-45: track actual time of migrations #103)
• [Disk Manager] fix cloud/tasks/persistence tests
• NBS-4773 [disk_manager] Support don't read from checkpoint feature (NBS-4773 [disk_manager] Support don't read from checkpoint feature #102)
• [Disk Manager] add cloud/tasks to github workflows
• add gitignores for test binaries ([Disk Manager] add gitignores for test binaries #164)
• NBSNEBIUS-76: RejectAgentTimeout dynamic calculation should deduplicate agent disconnects by rack - problems in a single rack should not make RejectAgentTimeout reach max values (NBSNEBIUS-76: RejectAgentTimeout dynamic calculation should deduplicate agent disconnects by rack - problems in a single rack should not make RejectAgentTimeout reach max values #159)
• NBS-4842 [Disk Manager] add errors/canceled metrics for s3 (NBS-4842 [Disk Manager] add errors/canceled metrics for s3 #162)
• [Tasks] rename env variable
• Add required resources for disk manager tests (Add required resources for disk manager tests #64)
• [Tasks] cloud/tasks should not depend on cloud/storage
• [Tests]: Do not allocate pseudo-terminal ([Tests]: Do not allocate pseudo-terminal #175)
• Disk manager check chunk contents for zeroes (Disk manager check chunk contents for zeroes #54)
• NBS-4358: a blocksize parameter for a pool config (NBS-4358: a blocksize parameter for a pool config #171)
• Launch everything under user, upload test artifacts, if test fails, to s3 (Launch everything under user, upload test artifacts, if test fails, to s3 #181)
• NBS-4789: [Disk Manager] refactor sre tools (NBS-4789: [Disk Manager] refactor sre tools #177)
• NBSNEBIUS-83: device config validation should not fail if Serial or PhysicalOffset changed from 0 to non-0 (NBSNEBIUS-83: device config validation should not fail if Serial or PhysicalOffset changed from 0 to non-0 #186)
• NBS-4358: added UT (NBS-4358: added UT #183)
• NBS-4808 add large and medium tests to gitignore (NBS-4808 add large and medium tests to gitignore #188)
• NBS-4787: enable ydb configs_dispatcher service (NBS-4787: enable ydb configs_dispatcher service #109)
• fix: remove debug command (fix: remove debug command #190)
• fix: remove debug step, remove upload of test results (fix: remove debug step, remove upload of test results #191)
• NBSNEBIUS-82: not counting broken placement group partitions for non-HDD disks; slightly improved ShouldHandleCmsRequestsForHostsWithDependentHddDisks ut (NBSNEBIUS-82: not counting broken placement group partitions for non-HDD disks; slightly improved ShouldHandleCmsRequestsForHostsWithDependentHddDisks ut #189)
• NBS-4826 send message about long running blob operations to TPartitionActor and TVolumeActor (NBS-4826 send message about long running blob operations to TPartitionActor and TVolumeActor #151)
• NBS-4748: migrate from solomon to monitoring (NBS-4748: migrate from solomon to monitoring #110)
• NBSNEBIUS-79: Updating build documentation (NBSNEBIUS-79: Updating build documentation #94)
• NBS-179: support using multiple devices from the same rdma endpoint
• [Disk Manager]: Ensure disk manager eternal tests are ran one at a time ([Disk Manager]: Ensure disk manager eternal tests are ran one at a time #205)
• fix fuzzing error in ReadContent (NBS-4720: fix fuzzing error in ReadContent #197)
• [Disk Manager] install qemu-utils on CI VMs because it is required for facade/image_service_*test ([Disk Manager] install qemu-utils on CI VMs because it is required for facade/image_service_*test #207)
• NBSNEBIUS-86: impld garbage FreshDeviceIds filtration in TVolumeState (NBSNEBIUS-86: impld garbage FreshDeviceIds filtration in TVolumeState #206)
• Don't use None version as custom (Don't use None version as custom #211)
• [Disk Manager] fix facade/image_service_*test ([Disk Manager] fix facade/image_service_*test #217)
• fix: log handling in chunk tests, added more logs to summary, append summary instead of add or modify, add flag to disable all info sync (fix: log handling in chunk tests, added more logs to summary, append summary instead of add or modify, add flag to disable all info sync #215)
• NBSNEBIUS-59: Return error on incorrect agentIds in UpdateAgentListParams request (NBSNEBIUS-59: Return error on incorrect agentIds in UpdateAgentListParams request #70)
• fix: trying to lower test_threads to limit concurrency and free up resources for large tests (fix: trying to lower test_threads to limit concurrency and free up resources for large tests #222)
• NBSNEBIUS-86: rendering raw volume config (pretty-printed proto) on volume monpage (NBSNEBIUS-86: rendering raw volume config (pretty-printed proto) on volume monpage #220)
• NBS-179: use backward compatible arguments to run vhost-server for local disks (NBS-179: use backward compatible arguments to run vhost-server for local disks #199)
• NBSNEBIUS-71: introduce alternating rw pattern to eternal-load binary (NBSNEBIUS-71: introduce alternating rw pattern to eternal-load binary #202)
• [Tasks] remove unneeded 'description' arg from scheduler.ScheduleRegularTasks ([Tasks] remove unneeded 'description' arg from scheduler.ScheduleRegularTasks #230)
• feature: add directory listing to s3 folder (feature: add directory listing to s3 folder #229)
• improve logging of aio failures in vhost-server (improve logging of aio failures in vhost-server #210)
• NBSNEBIUS-86: TDiskConfigs in DiskRegistry now store event history, history is displayed on DiskRegistry monpage, currently only ReplaceDevice events are added to history - will add other stuff in the next PRs (NBSNEBIUS-86: TDiskConfigs in DiskRegistry now store event history, history is displayed on DiskRegistry monpage, currently only ReplaceDevice events are added to history - will add other stuff in the next PRs #233)
• NBSNEBIUS-89: [Tasks] extract scheduleInterval and maxTasksInflight into TaskSchedule struct; rename tasks/storage.schedule to scheduleState (NBSNEBIUS-89: [Tasks] extract scheduleInterval and maxTasksInflight into TaskSchedule struct; rename tasks/storage.schedule to scheduleState #235)
• NBS-4859 Any errors in the mirrored disk replica should be retriable (NBS-4859 Any errors in the mirrored disk replica should be retriable #237)
• NBS-4736: report non repl hive metrics (NBS-4736: report non repl hive metrics #213)
• Fix disk manager eternal tests by calling exec_command in a thread pool executor (Fix slow ssh client in python tests #219)
• feature: cmake build summary in pr and junit reports (feature: cmake build summary in pr and junit reports #244)
• Fix execution of json requests (Fix execution of json requests in /example #234)
• [disk-manager] fix test on delete checkpoint data (NBS-4665: [disk-manager] fix test on delete checkpoint data #240)
• fix: nightly index rebuild, update: updated actions to fresher version using node20 instead of node16 (fix: nightly index rebuild, update: updated actions to fresher version using node20 instead of node16 #249)
• fix: vhost-server aio backend was incorrectly batching ios (fix: vhost-server aio backend was incorrectly batching ios #248)
• NBSNEBIUS-86: writing migration and replication-related events to disk history (NBSNEBIUS-86: writing migration and replication-related events to disk history #252)
• Mute more qemu tests for filestore (Mute more qemu tests for filestore #255)
• NBSNEBIUS-89: [Tasks] extract scheduleInterval and maxTasksInflight into TaskSchedule struct (for tasks.Scheduler) (NBSNEBIUS-89: [Tasks] extract scheduleInterval and maxTasksInflight into TaskSchedule struct (for tasks.Scheduler) #253)
• Show overridden AgentParams on monitoring page (Show overridden AgentParams on monitoring page #228)
• NBSNEBIUS-86: fixed stupid bug: FilteredFreshDeviceIds filtration should not drop all FreshDeviceIds for volumes created after 2023-08-30 (NBSNEBIUS-86: fixed stupid bug: FilteredFreshDeviceIds filtration should not drop all FreshDeviceIds for volumes created after 2023-08-30 #250)
• feature: remove images from TESTS_DATA_DIR to make results size smaller (feature: remove images from TESTS_DATA_DIR to make results size smaller #259)
• feature: add file sizes to directory listing (feature: add file sizes to directory listing #258)
• [Disk Manager]: Add remote tests binary ([Disk Manager]: Add remote tests binary #218)
• NBS-4787: select YDB metrics by host, remove cluster
• NBSNEBIUS-89: [Tasks] add 'crontab' parameters to TaskSchedule (not supported yet); refactor tasks/storage.CreateRegularTasks (NBSNEBIUS-89: [Tasks] add 'crontab' parameters to TaskSchedule (not supported yet); refactor tasks/storage.CreateRegularTasks #261)
• NBSNEBIUS-89: [Tasks] support 'crontab' parameters for regular tasks (NBSNEBIUS-89: [Tasks] support 'crontab' parameters for regular tasks #264)
• fix test artifact permissions (fix test artifact permissions #247)
• Fix inflight queue not closing ([Disk Manager]: Snapshot: Fix inflight queue not closing in ShallowCopySnapshot() #266)
• Make forbidden dependency check for model dirs (Make forbidden dependency check for model dirs #267)
• add go mods (adding go.mod's for tasks package and its dependencies  #270)
• NBS-4789: [Disk Manager] adding lister metrics for TaskStatusReadyToCancel and TaskStatusCancelling (NBS-4789: [Disk Manager] adding lister metrics for TaskStatusReadyToCancel and TaskStatusCancelling #271)
• ignoring TEvBootstrapper::TEvStatus in TVolumeActor::StateZombie (fixing rare crashes in tests) (ignoring TEvBootstrapper::TEvStatus in TVolumeActor::StateZombie (fixing rare crashes in tests) #272)
• improve go.mod (should use one go.mod file for whole repo #275)
• add logs for debugging (NBS-4843: [Disk Manager] add logs for debugging TestTasksRunningLimit #277)
• Revert "NBS-4787: enable ydb configs_dispatcher service (NBS-4787: enable ydb configs_dispatcher service #109)" (Revert "NBS-4787: enable ydb configs_dispatcher service (#109)" #274)
• NBS-4818 : Use ydb as a binary in filestore tests. (NBS-4818 : Use ydb as a binary in filestore tests. #273)
• NBS-4827 extract TNonreplicatedPartitionMigrationCommonActor (NBS-4827 extract TNonreplicatedPartitionMigrationCommonActor #194)
• issue [NBS] Ability to save block checksums in TBlobMeta to check them upon ReadBlob #196: implemented block checksum calculation and verification for the blobs stored in blobstorage (issue #196: implemented block checksum calculation and verification for the blobs stored in blobstorage #278)
• NBS-4759: [Disk Manager] remove unnecessary logic from generateBaseDisk (NBS-4759: [Disk Manager] remove unnecessary logic from generateBaseDisk #282)
• disabling java proto generation (disabling java proto generation to fix proto build #286)
• loadtest env tear_down should happen inside finally blocks because otherwise we may get an exception because blockstore-loadtest run failed and will never know that kikimr daemon crashed (loadtest env tear_down should happen inside finally blocks because otherwise we may get an exception because blockstore-loadtest run failed and will never know that kikimr daemon crashed #290)
• separate init and get token (NBS-4456: separate init and iam-token-client and get token #293)
• NBSNEBIUS-36: use socket path as a key in endpoint storage (NBSNEBIUS-36: use socket path as a key in endpoint storage #254)
• [CI]: Add ya cache garbage collection before scripts ([CI]: Add ya cache garbage collection before scripts #295)
• NBSOPSNEBIUS-25: report ExternalEndpointUnexpectedExit on vhost-server crashes (NBSOPSNEBIUS-25: report ExternalEndpointUnexpectedExit on vhost-server crashes #288)
• [CI]: Add Y_BENCHMARK and G_BENCHMARK to daily large tests ([CI]: Add Y_BENCHMARK and G_BENCHMARK to daily large tests #300)
• [CI]: Remove logs in nebius CI run_all.sh ([CI]: Remove logs in nebius CI run_all.sh #297)
• [CI]: Fix left python tests ssh client ([CI]: Fix left python tests ssh client #304)
• [CI]: Add missing temporary directory creation for the ssh client ([CI]: Add missing temporary directory creation for the ssh client #305)
• fix tests in cmake (Fix tests in cmake #299)
• NBS-4759: [Disk Manager] optimize relocation of overlay disks without alive pool (NBS-4759: [Disk Manager] optimize relocation of overlay disks without alive pool #289)
• issue [Filestore] Compaction/Cleanup/FlushBytes optimization #95 NBSNEBIUS-70: if Flush/Compaction/Cleanup/FlushBytes ops are enqueued before CompactionMap is completely loaded, their EOperationStates should not get stuck in the Enqueued state (issue #95 NBSNEBIUS-70: if Flush/Compaction/Cleanup/FlushBytes ops are enqueued before CompactionMap is completely loaded, their EOperationStates should not get stuck in the Enqueued state #303)
• [NBS] Errors in rdma metrics (Issue [NBS] Errors in rdma metrics #291) ( NBS-4867: Errors in rdma metrics NBS-4867 #292)
• fix config.proto in eternal-load to make compatible with previous config.sc (fix config.proto in eternal-load to make compatible with previous config.sc #307)
• NBSNEBIUS-36: fix keyring tests (NBSNEBIUS-36: fix keyring tests #311)
• NBSNEBIUS-71: make .proto config more compatible with previous .sc config (NBSNEBIUS-71: make .proto config more compatible with previous .sc config #313)
• Fix GITHUB_TOKEN issue in cmake tests (Fix GITHUB_TOKEN issue in cmake tests #316)
• Tests to run on .github folder update, to verify that all essential scripts are working correctly (Tests to run on .github folder update, to verify that all essential scripts are working correctly #285)
• [Disk Manager] fix imports in remote tests ([Disk Manager] fix imports in remote tests #319)
• remove arcadia paths from migration doc (remove arcadia paths from migration doc #317)
• issue [NBS] blockstore-server crashed in TExternalVhostEndpointListener #312: temporarily revert call to TrySwitchEndpoint (issue #312: temporarily revert call to TrySwitchEndpoint #321)
• allow to create underlay vm in eternal-tests; fix scp in test ssh client; fix running eternal-load command in background (NBSNEBIUS-100: allow to create underlay vm in eternal-tests; fix scp in test ssh client; fix running eternal-load command in background #308)
• Remove doc/ from pr build (Remove doc/ from pr build #320)
• fix: add more memory to instances and reduce test threads on nightly builds (fix: add more memory to instances and reduce test threads on nightly builds #327)
• fix remote tests build ([Disk Manager] fix remote tests build #326)
• NBS-1704: extract common code for unix domain sockets (NBS-1704: extract common code for unix domain sockets #283)
• [Filestore] NBSNEBIUS-91: Add ForcedCleanup handle for fs ([Filestore] NBSNEBIUS-91: Add ForcedCleanup handle for fs #268)
• issue [Filestore] Make Filestore read throughput scalable with respect to the number of clients #324: DescribeData API and implementation stub (issue #324: DescribeData API and implementation stub #329)
• tablet history entries should contain fqdn (tablet history entries should contain fqdn #325)
• feature: ttl-watcher is a script that removes old logs from s3 by setting up object acl (feature: ttl-watcher is a script that removes old logs from s3 by setting up object acl #334)
• NEBDUTY-504: save dmesg -T on the tear_down of blockstore loadtest (NEBDUTY-504: save dmesg -T on the tear_down of blockstore loadtest #309)
• fix: script path (fix: script path and order, so index files are also under those conditions #341)
• NBS-4808 improve example sh (NBS-4808 improve example sh #344)
• fixed English (fixed English #345)
• fix eol for some files (fix eol for some files #348)
• issue [Filestore] Make Filestore read throughput scalable with respect to the number of clients #324: DescribeData implementation and ut (issue #324: DescribeData implementation and ut #343)
• issue [Filestore] Make Filestore read throughput scalable with respect to the number of clients #324: cleanup: filesystem-id field is not needed for filestore-client executeaction, removed broken (and unneeded) json validation; fixed comments here and there, etc (issue #324: cleanup: filesystem-id field is not needed for filestore-client executeaction, removed broken (and unneeded) json validation; fixed comments here and there, etc #351)
• NBS-4829: [disk-manager]: add log level to dm logs (NBS-4829: [disk-manager]: add log level to dm logs #346)
• NBSNEBIUS-36: fix python nemesis-tests (NBSNEBIUS-36: fix python tests #347)
• NBSOPSNEBIUS-24: setup block count for disks; added configs for tests with local ssd (NBSOPSNEBIUS-24: setup block count for disks; added configs for tests with local ssd #315)
• issue [Filestore] Make Filestore read throughput scalable with respect to the number of clients #324: introduced TwoStageReadEnabled flag, returning this flag to filestore-vhost via TCreateSessionResponse::FileStore::Features; outputting StorageConfig overrides on monpage; minor cleanup (issue #324: introduced TwoStageReadEnabled flag, returning this flag to filestore-vhost via TCreateSessionResponse::FileStore::Features; outputting StorageConfig overrides on monpage; minor cleanup #352)
• report generation should properly handle unparsable test results - it should still process other test results and not just fail completely upon the first invalid result (report generation should properly handle unparsable test results - it should still process other test results and not just fail completely upon the first invalid result #357)
• issue Stopped to upload volume stats to some of the ydb tables #354: fix issue with pushing empty values to ydb tables + check this in integration test (issue #354: fix issue with pushing empty values to ydb tables + check this in integration test #355)
• NEBDUTY-547: dump sensors to the log when we couldn't wait for free space during test initialization (NEBDUTY-547: dump sensors to the log when we couldn't wait for free space during test initialization #363)
• [Disk Manager]: Tests: Increase nemesis restart timeout ([Disk Manager]: Tests: Increase nemesis restart timeout #361)
• [Disk Manager]: Tests: Use YDB binary from S3 instead of building it locally ([Disk Manager]: Tests: Use YDB binary from S3 instead of building it #359)
• feature: check if we have large-tests label and/or sleep label to launch large tests with sleep (feature: check if we have large-tests label and/or sleep label to launch large tests with sleep #371)
• fix deletion of rdma server session (fix deletion of rdma server session #336)
• [Disk Manager] fix journald_logger_test after NBS-4829: [disk-manager]: add log level to dm logs #346 ([Disk Manager] fix journald_logger_test after #346 #372)
• fix: move sleep to separate job and add correct condition to it (fix: move sleep to separate job and add correct condition to it #376)
• NBS-4838: [Disk Manager] refactor tasks storage (NBS-4838: [Disk Manager] refactor tasks storage #358)
• NBSNEBIUS-45: reset MaxMigrationTime on the last finished device migration; show Start migration timestamp on the volume's mon. page (NBSNEBIUS-45: fix MaxMigrationTime #374)
• NBS-4787: test ydb configs dispatcher service (NBS-4787: test ydb configs dispatcher service #225)
• [Disk Manager] recipe's stdin/stdout/stderr files should contain 'service name' ([Disk Manager] recipe's stdin/stdout/stderr files should contain 'service name' #375)
• issue [Filestore] Compaction/Cleanup/FlushBytes optimization #95: 1. deduplicating out-of-order compaction map chunk load requests in queue 2. limiting the number of out-of-order compaction map chunk load requests in queue (issue #95: 1. deduplicating out-of-order compaction map chunk load requests in queue 2. limiting the number of out-of-order compaction map chunk load requests in queue #382)
• remove launch of ttl-watcher (Remove launch of ttl-watcher #385)
• feature: add ability to run subset of tests based on labels (feature: add ability to run subset of tests based on labels #388)
• [Disk Manager] fix nemesis tests after [Disk Manager]: Tests: Increase nemesis restart timeout #361 ([Disk Manager] fix nemesis tests after #361 #390)
• issue-324: DescribeData: 1. BlobOffsets should be in bytes, not blocks 2. returning FileSize 3. added a ut that checks that we don't return data outside of the aligned superrange that contains the requested range (issue-324: DescribeData: 1. BlobOffsets should be in bytes, not blocks 2. returning FileSize 3. added a ut that checks that we don't return data outside of the aligned superrange that contains the requested range #399)
• issue [Filestore] Make Filestore read throughput scalable with respect to the number of clients #324: storing NProto::TFileStore in session state to use it in the two stage read implementation in TServiceActor (issue #324: storing NProto::TFileStore in session state to use it in the two stage read implementation in TServiceActor #400)
• [Tasks] get rid of unused counters ([Tasks] get rid of unused counters #398)
• issue [Filestore] Make Filestore read throughput scalable with respect to the number of clients #324: DescribeData: offset calculation shouldn't use TBlock::BlockIndex since this block might be not initialized (issue #324: DescribeData: offset calculation shouldn't use TBlock::BlockIndex since this block might be not initialized #402)
• [Disk Manager] use SET_APPEND(RECIPE_ARGS... instead of SET(RECIPE_ARGS... ([Disk Manager] use SET_APPEND(RECIPE_ARGS... instead of SET(RECIPE_ARGS... #401)
• NBS-4892: fix crash (NBS-4892: fix crash #393)
• show fs sessions in monitoring (show fs sessions in monitoring #397)
• Introduce PoisonPill helper (Introduce PoisonPill helper #384)
• resolve issues (NBS-4759: [Disk Manager] resolve issues from #289 #404)
• NBS-4876 Persistent save shadow disk state (NBS-4876 Persistent save shadow disk state #366)
• NBS-4759: [Disk Manager] hotfix and add test for NBS-4759: [Disk Manager] resolve issues from #289 #404 (NBS-4759: [Disk Manager] hotfix and add test for #404 #405)
• NBS-4684: [Disk Manager] use proxy overlay disks in relocation if possible (NBS-4684: [Disk Manager] use proxy overlay disks in relocation if possible #350)
• feature: add ttl removal for files, fix: flake8 for generate-summary.py (feature: add ttl removal for files, fix: flake8 for generate-summary.py #395)
• [Disk Manager]: Tests: Actualize README.md ([Disk Manager]: Tests: Actualize README.md #396)
• Remove spaces (Remove spaces #409)
• Use better place for deny_ydb_dependency.inc (Use better place for deny_ydb_dependency.inc #410)
• Add to gitignore more files (Add to gitignore more files #411)
• NBSNEBIUS-101: use vhost-side reads WIP. Issue: [Filestore] Compaction/Cleanup/FlushBytes optimization #95 (NBSNEBIUS-101: use vhost-side reads WIP. Issue: #95 #394)
• Add actors tests to build (Add actors tests to build #413)
• CLOUDDUTY-25033 (Remove unused #414)
• NEBDUTY-603: added git reset --hard before git pull for nightly builds on our ci vm (NEBDUTY-603: added git reset --hard before git pull for nightly builds on our ci vm #415)
• [Tasks] Use utc time when scheduling regular tasks by cron. (Use utc time when scheduling regular tasks by cron. #417)
• feature: added label to run rebase on pr to test it, fix: updated github-script version to migrate node16->node20 (feature: added label to run rebase on pr to test it, fix: updated github-script version to migrate node16->node20 #418)
• CLOUDDUTY-25033 (CLOUDDUTY-25033 #421)
• move call to .UTC() closer to the actual usage (Regular Crontab tasks: move call to .UTC() closer to the actual usage #420)
• fix: index generation arguments on build or tests (fix: index generation arguments on build or tests #422)
• Fix deny-deps (Fix deny-deps #423)
• NBS-4876 fix compilation in arcadia (NBS-4876 fix compilation in arcadia #426)
• add date&time to chaos monkey logs (add date&time to chaos monkey logs #330)
• issue-324: proper EvGet error handling, not outputting user data to logs, dependencies cleanup, code cleanup (issue-324: proper EvGet error handling, not outputting user data to logs, dependencies cleanup, code cleanup #419)
• [Disk Manager]: Test: Obtain the disk from the previous test ([Disk Manager]: Test: Obtain the disk from the previous test #408)
• issue-324: TServiceActor: ReadData fallback in case DescribeData or EvGet fail (issue-324: TServiceActor: ReadData fallback in case DescribeData or EvGet fail #428)
• [BB->arc sync] Libvhost support for MEM_SLOTS/VRING_ENABLE ([BB->arc sync] Libvhost support for MEM_SLOTS/VRING_ENABLE #337)
• add Executor to TEndpointService (add Executor to TEndpointService #427)
• Disabled TLogbrokerTest::ShouldWriteData (Disabled TLogbrokerTest::ShouldWriteData #433)
• NBS [NBS] Fast data path for NRD #179: use durable client to retry network errors in vhost-server rdma backend (NBS #179: use durable client to retry network errors in vhost-server rdma backend #276)
• [Disk Manager]: Test: Use per-disk suffix for acceptance tests ([Disk Manager]: Test: Use per-disk suffix for acceptance tests #429)
• enabled submodules in checkout actions (enabled submodules in checkout actions #439)
• NBS-4875: Always adjust blocks count (NBS-4875: Always adjust blocks count #339)
• issue-324: added ut and loadtest for the TwoStageRead feature, fixed a crash in TReadDataActor::HandleReadBlobResponse (issue-324: added ut and loadtest for the TwoStageRead feature, fixed a crash in TReadDataActor::HandleReadBlobResponse #441)
• use bucket-location param to locate s3 bucket (Use bucket-location param to locate s3 bucket #442)
• move checks of processing sockets to EndpointManager (#353: move checks of processing sockets to EndpointManager #335)
• cloud/contrib: replace vhost with a submodule (cloud/contrib: replace vhost with a submodule #432)
• FIx ASAN build (FIx ASAN build #447)
• fix nbd/bench (fix nbd/bench #450)
• disk agent should reject secure erase if there are inflight write or zero requests (disk agent should reject secure erase if there are inflight write or zero requests #362)
• Add missed #include (Add missed #include <thread> #391)
• fix rdma completions polling (fix rdma completions polling #449)
• ISSUE-353: fix race with RefreshEndpoint, tweak service_endpoint (ISSUE-353: fix race with RefreshEndpoint, tweak service_endpoint #446)
• fix test with address sanitizer (fix test with address sanitizer #460)
• backport missing changes from PR (disk agent should reject secure erase if there are inflight write or zero requests #454) (backport missing changes from PR (#454) #459)
• issue-122: ReadBlocks should use commit id to better understand whether it overlaps with unconfirmed blobs or not (issue-122: ReadBlocks should use commit id to better understand whether it overlaps with unconfirmed blobs or not #457)
• Get rid of EnableNodeIdentifier setting because of its deprecation (Get rid of EnableNodeIdentifier setting because of its deprecation #462)
• [NBS] log client_id in NBS go sdk ([NBS] log client_id in NBS go sdk #465)
• [Disk Manager] use YDB from trunk if not OPENSOURCE ([Disk Manager] use YDB from trunk if not OPENSOURCE #468)
• Build endpoint_manager fix (Build endpoint_manager fix #470)
• issue-324: TServiceActor: fixing TReadDataActor leak (issue-324: TServiceActor: fixing TReadDataActor leak #471)
• Update build instruction (Update build instruction #475)
• issue [Blockstore] should add blobs to index in parallel while writing them #122: DescribeBlocks should respect 'unconfirmed blobs' feature (issue #122: DescribeBlocks should respect 'unconfirmed blobs' feature #473)
• [Filestore] NBSNEBIUS-99: add counters for better introspective of data state ([Filestore] NBSNEBIUS-99: add counters for better introspective of data state #466)
• [CI]: Decrease large tests logs TTL from 30 days to 7 ([CI]: Decrease large tests logs TTL from 30 days to 7 #478)
• issue-95: 1. fixing a bug that caused FlushState to get stuck in the Enqueued state if FlushBytes got triggered before the complete initialization of CompactionMap 2. rebooting tablet after N backpressure errors to make it more resilient in case of similar bugs (issue-95: 1. fixing a bug that caused FlushState to get stuck in the Enqueued state if FlushBytes got triggered before the complete initialization of CompactionMap 2. rebooting tablet after N backpressure errors to make it more resilient in case of similar bugs #486)
• [Disk Manager]: Tests: Mute TestShallowCopySnapshotWithRandomFailure ([Disk Manager]: Tests: Mute TestShallowCopySnapshotWithRandomFailure #453)
• [CI]: Add logging to run_all.sh ([CI]: Add logging to run_all.sh #477)
• Add ASAN support (Add ASAN support #469)
• NBS-4905 simplify SendRequestToPartitionWithUsedBlockTracking (NBS-4905 simplify SendRequestToPartitionWithUsedBlockTracking #490)
• NBS-4683 tests for multiple block size (NBS-4683 tests for multiple block size #456)
• [Disk Manager] improve nbs transfer logging ([Disk Manager] improve nbs transfer logging #483)
• issue [Blockstore] should add blobs to index in parallel while writing them #122: GetChangedBlocks should respect 'unconfirmed blobs' feature (issue #122: GetChangedBlocks should respect 'unconfirmed blobs' feature #484)
• NBSNEBIUS-41: don't release NRD session on retriable errors (NBSNEBIUS-41: don't release NRD session on retriable errors #430)
• [Disk Manager]: Tests: Clean up disk manager snapshots before each test ([Disk Manager]: Tests: Clean up disk manager snapshots before each te… #383)
• issue-324: added tablet-level metrics for WriteData/ReadData/DescribeData requests (issue-324: added tablet-level metrics for WriteData/ReadData/DescribeData requests #496)
• v1 (NBS-4488: [Disk Manager] remove unused code from migration task #503)
• Improve clang-format. Prefer return type on its own line (Improve clang-format. Prefer return type on its own line #510)
• Generate vscode workspace (Generate vscode workspace #511)
• v1 (NBS-4814: [Disk Manager] introduce vhd format footer/header #500)
• [Disk Manager] improve nemesis logging ([Disk Manager] improve nemesis logging #485)
• NBS-4827 Prepare for shadow disk actor (NBS-4827 Prepare for shadow disk actor #474)
• issue [Blockstore] should add blobs to index in parallel while writing them #122: get rid of OverlapsUnconfirmedBlobs overloading; fix naming inconsistency (issue #122: get rid of OverlapsUnconfirmedBlobs overloading; fix naming inconsistency #494)
• NBS-4488: [Disk Manager] remove unnecessary migration config setting (NBS-4488: [Disk Manager] remove unnecessary migration config setting #517)
• [Disk Manager] fix typo in acceptance test disk policy
• issue-324: removed IBlockBuffer::GetContent and GetContentRef methods, extracted CopyFileData implementation, rewritten it without GetContent/GetContentRef usage, added uts for it (preparation to implement a lazy-alloc block buffer to use it in HandleDescribeData) (issue-324: removed IBlockBuffer::GetContent and GetContentRef methods, extracted CopyFileData implementation, rewritten it without GetContent/GetContentRef usage, added uts for it (preparation to implement a lazy-alloc block buffer to use it in HandleDescribeData) #520)
• add EOL ([Disk Manager] add EOL to the file #525)
• v1 (NBS-4902: [Disk Manager] fix race condition in migrate disk task #513)
• [Filestore] NBSNEBIUS-121: TwoStageRead: add logging for ReadData fallback reason + fix ServiceActor not reporting ReadData counters ([Filestore] NBSNEBIUS-121: TwoStageRead: add logging for ReadData fallback reason + fix ServiceActor not reporting ReadData counters #521)
• issue-324: implemented LazyBlockBuffer and using it in TIndexTabletActor::HandleDescribeData (issue-324: implemented LazyBlockBuffer and using it in TIndexTabletActor::HandleDescribeData #523)
• [CI]: Write run_all.sh output to the logs_dir ([CI]: Write run_all.sh output to the logs_dir #527)
• Tune gitignore (Tune gitignore #529)
• Change notify-mock tu py3.12 (Change notify-mock to py3.12 #519)
• NBSNEBIUS-41: simplify the process of acquiring a disk (NBSNEBIUS-41: simplify the process of acquiring a disk #516)
• Make go version in go.mod and in ya tool the same (Make go version in go.mod and in ya tool the same #531)
• mute nemesis test (mute nemesis test #528)
• NEBINC-82: [Disk Manager] add readOnCacheMiss to url/common/cache constructor (NEBINC-82: [Disk Manager] add readOnCacheMiss to url/common/cache constructor #536)
• NEBINC-82: [Snapshot] reader.ReadBinary should use cache (NEBINC-82: [Snapshot] reader.ReadBinary should use cache #537)
• NBS-4814: [Disk Manager] intoduce vhd image_map_reader; implement ReadHeader and ReadFooter (NBS-4814: [Disk Manager] intoduce vhd image_map_reader; implement ReadHeader and ReadFooter  #515)
• track small number of recently completed requests; resolves [NBS] Race between send and recv completions #481 (track small number of recently completed requests; resolves #481 #489)
• NEBINC-82: [Snapshot] should use 'url source forbidden' message when encounter 403 http status code (NEBINC-82: [Snapshot] should use 'url source forbidden' message when encounter 403 http status code #542)
• [CI]: Enable cleanup in disk manager acceptance tests ([CI]: Enable cleanup in disk manager acceptance tests #545)
• [CI]: Move cleanup for a single disk test under a feature flag ([CI]: Move cleanup for a single disk test under a feature flag #544)
• clarify callers and interfaces (clarify callers and interfaces #518)
• Suppress E_NOT_FOUND error in GetDependentDisks (Suppress E_NOT_FOUND error in GetDependentDisks #543)
• issue-541: added counters: StatefulSessionsCount, StatelessSessionsCount, SessionTimeouts (issue-541: added counters: StatefulSessionsCount, StatelessSessionsCount, SessionTimeouts #550)
• issue-95: not counting backpressure errors until compaction state becomes fully loaded (issue-95: not counting backpressure errors until compaction state becomes fully loaded #553)
• issue-541: added DescribeSessions private api action (issue-541: added DescribeSessions private api action #554)
• [CI]: Fix acceptance tests issues ([CI]: Fix acceptance tests issues  #548)
• add CMS action to the private API (add CMS action to the private API #555)
• issue-541: added createsession and resetsession commands to filestore-client, testing create+reset+describe sessions in client tests (issue-541: added createsession and resetsession commands to filestore-client, testing create+reset+describe sessions in client tests #556)
• NBS [NBS] Fast data path for NRD #179: switch fast path volume to regular path during migrations
• CR
• CR2